Skip to main content

Healthcare system inputs and patient-reported outcomes: a study in adults with congenital heart defect from 15 countries



The relationship between healthcare system inputs (e.g., human resources and infrastructure) and mortality has been extensively studied. However, the association between healthcare system inputs and patient-reported outcomes remains unclear. Hence, we explored the predictive value of human resources and infrastructures of the countries’ healthcare system on patient-reported outcomes in adults with congenital heart disease.


This cross-sectional study included 3588 patients with congenital heart disease (median age = 31y; IQR = 16.0; 52% women; 26% simple, 49% moderate, and 25% complex defects) from 15 countries. The following patient-reported outcomes were measured: perceived physical and mental health, psychological distress, health behaviors, and quality of life. The assessed inputs of the healthcare system were: (i) human resources (i.e., density of physicians and nurses, both per 1000 people) and (ii) infrastructure (i.e., density of hospital beds per 10,000 people). Univariable, multivariable, and sensitivity analyses using general linear mixed models were conducted, adjusting for patient-specific variables and unmeasured country differences.


Sensitivity analyses showed that higher density of physicians was significantly associated with better self-reported physical and mental health, less psychological distress, and better quality of life. A greater number of nurses was significantly associated with better self-reported physical health, less psychological distress, and less risky health behavior. No associations between a higher density of hospital beds and patient-reported outcomes were observed.


This explorative study suggests that density of human resources for health, measured on country level, are associated with patient-reported outcomes in adults with congenital heart disease. More research needs to be conducted before firm conclusions about the relationships observed can be drawn.

Trial registration NCT02150603. Registered 30 May 2014,

Peer Review reports


Healthcare expenditures have never been higher and medical costs are expected to continue to increase in all parts of the world [1]. In 2013, 7.8 trillion US $ were spent on healthcare worldwide, and this number is expected to increase up to 18.3 trillion US $ by 2040 [1]. This strong increase can be attributed to an ageing population and medical innovation [1]. Hence, more than ever, it is important to critically evaluate financial investments in health care and appraise the importance of ‘inputs’, such as human resources and the infrastructure of the healthcare system. Human resources of the healthcare system are “all people engaged in actions whose primary intent is to enhance health” [2]. Healthcare system infrastructure refers to “formal and enduring structures that support health”, such as hospitals and the number of available hospital beds [3].

In the late 1960s, a behavioral model of healthcare services use was developed by Andersen and colleagues to facilitate an understanding of which factors influenced patients’ use of healthcare services [4]. This model has expanded with the growth of supporting empirical evidence [5]. In the latest versions of the model (see eFig. 1), healthcare system inputs are considered part of the organization of the healthcare system and are included as contextual characteristics driving health consumption by patients [5]. The model assumes that contextual characteristics are both directly and indirectly associated with patient outcomes and that these relationships can be bidirectional.

Prior studies have investigated the role of human resources and infrastructure for health in predicting patient outcomes [6,7,8,9]. Physician and nurse staffing ratios and the availability of hospital beds per population have been associated with various outcomes including mortality, stage of diagnosis, prognosis, and quality of care [7,8,9,10]. However, minimal attention has been devoted to the associations between the human resources and the infrastructure, and outcomes which are particularly relevant from the patients’ perspective, namely patient-reported outcomes (PROs) [11,12,13,14,15]. Yet, in light of the current priority on patient-centered, holistic, and comprehensive care, the patient perspective should surely be incorporated in healthcare system evaluations [16]. Therefore, the relationship between country-level healthcare system inputs and PROs warrants investigation in public health and health services research. PROs are “measurements based on a report that comes directly from the patient about the status of a patient’s health condition, without amendment or interpretation of the patient’s response by a clinician or anyone else” [17]. Examples of PROs are self-reported health status or quality of life, these outcomes are included as such in the Andersen behavioral model (see eFig. 1) [5]. One study has investigated the association between the density of healthcare workforce (on country-level) and psychosomatic and mental health symptoms (on individual-level), but found no robust relationship [15]. As the Andersen behavioral model presumes a relationship between country-level healthcare system inputs and PROs, such investigations are warranted.

For the present study, we use data of adults with congenital heart disease (CHD). With a birth prevalence up to 9.3 per 1000 newborns, CHD is the commonest birth defect in neonates [18]. This population is of great interest to investigate the association between healthcare system inputs and PROs, for several reasons. First, in Western countries, due to significant medical advances in recent decades, over 90% of children with CHD now survive into adulthood, thus yielding an exponential growth of the population of adults with CHD [19]. Hence, this population is putting an increased burden on the healthcare system, with exponentially increasing healthcare utilization and costs [20]. From this perspective, CHD can be seen as an exemplar for life-cycle diseases. Second, CHD comprises a range of mild, moderate and complex heart defects. This variability in complexity reflects the broader spectrum of chronic diseases which also differ in complexity and severity. Third, as in other chronic conditions, a comprehensive understanding of adult CHD extends beyond medical outcomes and includes PROs. Inter-country variations in PROs among adults with CHD have previously been observed [13]. It is possible that certain healthcare system characteristics at the national level may play a role in this matter. Indeed, among adults with CHD, total healthcare expenditures per capita and healthcare system performance have been associated with PROs [13, 21]. The explained variance of these factors, however, ranged between 0.001 and 0.25%, suggesting that other aspects of the organization of the healthcare system require scrutiny. When exploring the relationship between healthcare system inputs and PROs in this heterogeneous chronic population, we can obtain valuable information that can be transferred, at least to a certain extent, to other chronic patient populations.

We aimed to test the presumed relationship between the inputs of the overall healthcare system and PROs of the Andersen behavioral model and to obtain a better understanding of the mechanism of inter-country differences in PROs. To achieve this aim, we investigated whether human resources and infrastructure for health on country level, operationalized as physician density, nurse density, and hospital bed density, were predictors of PROs in a large international sample of adults with CHD.


Study population and procedure

The present study was part of an overarching project entitled ‘Assessment of Patterns of Patient-Reported Outcomes in Adults with Congenital Heart disease – International Study’ (APPROACH-IS). A total of 4028 adults with CHD from 15 countries, comprising five continents, were included in this project [13, 22, 23].

Inclusion criteria were: (i) diagnosis of CHD, defined as “a structural abnormality of the heart and/or intra-thoracic great vessels that is present at birth and of actual or potential functional significance” [24]; (ii) aged 18 years or older; (iii) diagnosis established before the age of 10 years; (iv) continued follow-up at a CHD center or included in a national/regional registry; and (v) possessing physical, cognitive, and language capabilities required to complete self-reported questionnaires. Exclusion criteria were: prior heart transplantation and primary pulmonary hypertension [22]. Data collection was carried out from April 2013 through March 2015. In a previous paper, rationale, design, and methods of APPROACH-IS have been detailed [22]. The study protocol was registered at NCT02150603.

Measures (individual-level data)

For the present study, data on five PROs were analyzed: perceived physical health and mental health status (12-item Short Form Health Survey), psychological distress (Hospital Anxiety and Depression Scale), health behavior (Health Behavior Scale–Congenital Heart Disease), and quality of life (Linear Analog Scale). Further details on the used set of questionnaires have been reported previously [22] and can be found online in supplementary material (see eTable 1).

Healthcare system inputs (country-level data)

In order to capture inputs of the healthcare system, human resources and infrastructure measurements were assessed. Country-level human resources for health were operationalized in terms of physician density as well as density of nurses. Physician density per 1000 people was defined as the number of physicians, including generalists and specialist medical practitioners, per 1000 citizens in the country’s population [25, 26]. Nurse density per 1000 people was operationalized as the number of professional nurses, professional midwives, auxiliary nurses, auxiliary midwives, enrolled nurses, enrolled midwives and other associated personnel, such as dental nurses and primary care nurses, per 1000 people within a country’s population [26]. Country-level health infrastructure was expressed as the hospital bed density per 10,000 population, which included the total number of hospital beds excluding labour and delivery beds per 10,000 population [26]. These data were obtained from the World Bank and the World Health Organization (WHO) [25, 26]. Since data from Taiwan are lacking in the WHO repository, the Taiwan Statistical Data Book was consulted to obtain the required data for Taiwan [27]. For most countries, data from 2015 have been selected. If this score was unavailable, the nearest-to-2015 score has been selected (see eTable 2).

Statistical analyses

Continuous data are presented as medians and interquartile ranges. Categorical variables are presented as absolute numbers and percentages.

The association of healthcare system inputs and PROs was estimated through general linear mixed models (GLMM). GLMM are an extension to the generalized linear model (GLM) with random effects in addition to the usual fixed effects. In the analyses, a 2-level hierarchical structure was taken into account, in which patients were nested within countries. Thereby, random effects were included at the country-level and the model corrects for differences between countries, so called ‘unmeasured country differences’. We modeled five PROs as dependent variable with the healthcare system inputs as predictors. Only patients for whom full data were available for all variables of interest (n = 3588) were included, as only a relatively small proportion of patients had missing data for PROs (0.05–1.89%). Therefore, it was unnecessary to perform multiple imputation. The GLMM analyses were performed in three steps:

First, separate univariable general linear mixed models were applied to estimate the associations between density of physicians, density of nurses and density of hospital beds and the five PROS, being physical health status, mental health status, psychological distress, health behavior, and quality of life.

Second, multivariable general linear mixed models were performed for the three health care system inputs and the five PROs, with adjustments for patient-specific variables that previously have been found to predict PROs (i.e., sex, age, employment status, marital status, highest educational level, complexity of the health disease and the patient’s functional ability (New York Heart Association classification) [13]) and unmeasured country differences.

Third, in order to estimate the robustness of the results, we performed sensitivity analyses in which we excluded countries with an outlying density of more than two standard deviations from the mean. The multivariable general linear mixed models were repeated while excluding these centers.

Data analysis was performed using IBM SPSS Statistics for Windows, version 24 (Armonk, NY: IBM Corp.). Tests were two-sided and a p < 0.05 was considered as statistical significant.


Sample characteristics

In all, 3588 adults with CHD from 15 countries were included in the analyses. The country with the greatest number of included patients was the USA (666; 18.6%). Most patients had moderate disease complexity (1740; 48.5%), were women (1876; 52.3%) and reported no limitations during physical activity (NYHA Class 1) (1951; 54.4%). Detailed information on the demographic and clinical characteristics of the sample is given in Table 1.

Table 1 Demographic and clinical characteristics of the sample (n = 3588)

Healthcare system inputs

In Figure 1, we summarize country-level healthcare system inputs data from the different participating countries. Density of physicians was highest in Norway (4.4 per 1000 people) and lowest in India (0.8 per 1000 people). Norway had the highest density of nurses (17.9 per 1000 people), whereas India had the lowest (2.1 per 1000 people). With 134 beds per 10,000 people, Japan had the highest density of hospital beds, whereas India had the lowest density (7 beds per 10,000 people). For all countries, the mean density of physicians was 3.2 per 1000 people (SD: 1.0), the mean density of nurses was 9.8 per 1000 people (SD: 4.4) and the mean density of hospital beds was 48.1 per 10,000 people (SD: 28.9). Hence, two outlying values were noticed, being the density of physicians in India and the density of hospital beds in Japan. In eTable 2 of the supplementary material, a more detailed overview of these data, the year these data were collected, and a brief description of the overall healthcare system, can be found.

Fig. 1
figure 1

Human resources and infrastructure of the healthcare system of countries included in APPROACH-IS. Size of the bubbles and number next to bubbles indicate the density of hospital beds per 10,000 people; Sources: World Bank [25], World Health Organization [26] and Taiwan Statistical Data Book 2016 [27]

Patient-reported outcomes

PROs of this sample are described in eFig. 2 and have been reported and discussed in more detail in a prior publication [13]. Patients from Malta had the best self-reported physical and mental health, whereas patients from India and France had the worst self-reported physical and mental health, respectively. The lowest scores of psychological distress (i.e., depressive and anxiety symptoms) were found in patients living in the Netherlands, whereas the highest scores were reported by patients from India. Moreover, Indian patients demonstrated higher prevalence of health risk behaviors (i.e., smoking, drinking, drugs, lack of physical activity, and problematic oral care), whereas Swedish patients demonstrated the lowest prevalence of health risk behaviors. Patients from Switzerland demonstrated the best quality of life, and Japanese patients the worst.

Association between healthcare system inputs and PRO

In the univariable general linear mixed models, higher physician density was associated with better self-reported physical and mental health, less psychological distress, and better quality of life (Table 2). A higher number of nurses per population was associated with better physical health status and less risky health behavior. A high hospital bed density was associated with lower quality of life.

Table 2 General Linear Mixed Models with healthcare system inputs as predictors of PROs (n = 3588)

Multivariable general linear mixed models, adjusted for patient characteristics and unmeasured country differences, showed very similar results (Table 2). However, the association between density of physicians and quality of life was no longer significant and the association between density of nurses and psychological distress turned significant in the multivariable analyses.

Because the density of physicians in India and the density of hospital beds in Japan deviated more than two standard deviations from the mean in the sample, we performed sensitivity analyses excluding these countries on an individual basis (Table 2). After removing India, higher density of physicians remained associated with better self-reported physical and mental health status, less psychological distress, and better quality of life. When Japan was excluded from the analyses, no associations between bed density and PROs were observed.


We explored the relationship between healthcare system inputs and PROs, as proposed in the Andersen behavioral model. The results of the present study demonstrated that country-level density of physicians and nurses were associated with PROs among adults with CHD. Although replication in future studies is needed, our results suggested that human resources for health might directly or indirectly affect PROs and might partly explain the inter-country variations in PROs reported previously [13].

Physician density

In the current study, univariable, multivariable, and sensitivity analyses indicated that a higher country-level physician density was associated with favourable PROs in adults with CHD. More precisely, we found that a higher number of physicians was associated with better self-reported health, less psychological distress, and better quality of life. This is not in line with the study of Riehm and colleagues, in which no association between the density of physicians and psychiatrists (on country-level) and better mental health of adolescents was found [15]. However, most previous research, which investigated the relationship between the density of physicians and outcomes at a national level, both patient-reported and other patient outcomes, are in line with our findings. In Canada, the variation of physician density within the country determined self-reported general health status [12]. It was estimated that adding an additional family physician per 10,000 population would yield a 2 to 4% increase in self-reported general health [12]. Similarly, an American study demonstrated that a higher supply of primary care physicians was associated with a higher probability of reporting good self-rated health [28]. In addition, a study from Hungary showed that more physicians per 100,000 inhabitants was associated with better recognition of depression and a lower suicide rate [29]. It can be hypothesized that earlier diagnosis and timely treatment of health problems may explain the associations between higher physician density with better health status and less psychological distress. Indeed, greater physician density has been shown to be associated with lower mortality rates, earlier stage of diagnoses, and better prognosis [7,8,9]. As yet another example, the geographical variation in breast cancer mortality in the United States was shown to be partly explained by the number of physicians [8]. Our study suggested that improved outcomes do not only pertain to mortality and morbidity, but also to PROs.

Nurse density

We also found that a higher country-level nurse density was associated with better perceived physical health, less psychological distress, and less risky health behaviors. A higher nurse density is likely to yield more absolute hours available for direct nursing care. A prior study in the United States showed that more hours of provision of direct nursing care on a shift was related to lower levels of pain and better self-reported health status of patients [30]. Hence, the time that a nurse has for direct patient care is probably a mediator for the association between nurse density and self-reported health, as it can be assumed that when the nurse density increases, the patient-to-nurse ratio decreases. Our finding that a higher density of nurses was associated with less psychological distress may initially seem to contradict those of an American study conducted in nursing homes, in which optimal staffing ratios were related to higher levels of depression [11]. However, this finding was determined to be due to the fact that higher nurse density led to improved recognition (i.e., diagnosis) of depression and anxiety [11]. This was confirmed in another study in which the predicted probability of a diagnosis of depression or anxiety increased from 14 to 16% when the ratio changed from over five patients per nurse to under three [31].

Hospital bed density

Higher hospital bed density was associated with lower quality of life in uni- and multivariable analyses. However, the hospital bed capacity of Japan (i.e., 134 per 10,000 people) is by far the highest in the world [32], and is an outlier in our sample. In addition, we previously showed that Japanese patients enrolled in APPROACH-IS had the lowest average quality of life. We thus suspected that the association between bed density and quality of life could be fully explained by the outlying hospital bed capacity of Japan. This was confirmed by sensitivity analysis, which showed no associations between bed density and PROs after Japan was excluded. The number of hospital beds is an indicator for the capacity and availability of hospital infrastructure. But, it does not seem to be of great importance for patients with CHD. It could be that hospital beds are not the best indicator for this population, as most patients make use of the outpatient clinics. Therefore, in future studies in patients with CHD, it could be interesting to bring PROs of adults with CHD in association with the number of outpatient clinics or outpatient clinic visits.

Methodological considerations

To date, most research on the association between health system inputs and PROs have been performed on a national level [8, 11, 12]. These studies investigated regional or institutional variance, but did not allow us to appraise variability between countries. Hence, the unique contribution of our study was the examination of healthcare system inputs (on country level) and PROs (on individual level) from an international perspective. Nonetheless, there are some methodological limitations to consider.

First, we used healthcare input data at country level and PROs at the level of individual patients, who were recruited from one or more centers in the respective countries. Although we performed multilevel analyses to control for unmeasured country differences and to take into account that patients are nested in countries, it would have been interesting to investigate human resources and PROs both measured at country or center level.

Second, although we included data from 15 countries around the globe, all but two (i.e., India and Argentina) were high income countries. To gain a richer understanding of the impact of healthcare system inputs at the international level, it will be important that future studies include more low- and middle-income countries. Furthermore, in most countries, there is regional variation in healthcare system inputs. Hence, it could be that the results have been influenced by the fact that some centers are located in high resources areas. Also this issue should be examined in future studies.

Third, although the Andersen behavioral model suggested both a direct and indirect link between contextual organization of the healthcare system and PROs (see eFigure 1) [5], the variables are distal from each other. Indeed, it is highly unlikely that the associations are direct. It can be assumed that other healthcare system characteristics (e.g., accessibility to care, including aspects of financing and insurance) or country specific characteristics (e.g., educational level or overall wealth of the population) might be confounding the association. The present study, therefore, warrants future research in which the underlying mechanisms and the intermediate variables are investigated thoroughly.

Fourth, the Andersen behavioral model assumes that the relations are bidirectional [5]. Because our study used an ecological cross-sectional design [22], we cannot draw conclusions in terms of the direction of effects, nor could causality be tested. Since this cross-sectional study provided strong indications about this association, subsequent studies should investigate the associations longitudinally by using multiple time points to confirm the presumed bidirectional association of the Andersen behavioural model.

Fifth, it is unknown whether our results pertaining to adults with CHD may be generalized to adults with other health conditions. As a sample case, CHD does represent a broad spectrum of mild, moderate, and complex chronic diagnoses. However, it would be interesting to investigate whether our observations could be replicated among patients with other chronic health conditions. Furthermore, there was no control group available for this study, in which the same patient-reported outcomes were measured using a similar methodology. But, it would be interesting in future studies to add a healthy control group or general population normative data to increase the generalizability and transferability of the findings.

In addition, this study focused on the density of physicians, nurses, and hospital beds. These are important indicators of the healthcare system, since they have been shown to be associated with the national performance of the system [33]. However, other aspects of the comprehensive organization of the healthcare system, such as the overall performance, accessibility to care and quality of care metrics should also be considered in future studies. Moreover, for the population of adults with CHD, it would be interesting to study the association between PROs and the density of human resources that is most relevant to them, namely the density of congenital cardiologists or specialized nurses. Additionally, it would be interesting to subsequently look at the combination of these professions and their impact on PROs, since they interact highly with each other and are both needed in the provision of good care.

Moreover, it is contradictory that midwives have been included in the nurse density and maternity beds have been excluded in the bed density. However, this is how the data were represented by the World Health Organization and the World Bank. Leaving out certain data for our analyses would have impacted the comparability and reliability of the data, since the available information on the number of midwives was incomplete or outdated for several countries.

Finally, valid and reliable self-reported instruments were used to measure PROs. Consequently, patients with cognitive impairment and learning difficulties were excluded in the study, which might also influence the generalizability of the findings. However, a study on the Swedish data of APPROACH-IS has shown that differences between participants and non-participants in demographic, clinical, and health status characteristics were small [34].

Understanding the influence of density of human resources and infrastructure on outcomes is an important consideration for health policy and planning [12]. Indeed, the study should be seen as a preliminary step for researchers to investigate the influence of healthcare system inputs on PROs. It is likely only with the availability of such scientific data that we will be able to advise policy makers about aspects of the healthcare system impacting PROs.


This explorative study provides a first indication that country-level density of healthcare system inputs are associated with PROs in adults with CHD. More specifically, higher physician density was associated with better self-reported physical and mental health, less psychological distress, and better quality of life. Higher density of nurses was associated with better physical health status, less psychological distress, and less risky health behaviors. Density of hospital beds was not associated with PROs. Before firm conclusions about the relationships observed can be drawn and before recommendations for policy-making can be made, more research ought to be conducted in order to replicate the present findings and to scrutinize the mediating mechanisms between characteristics of the healthcare system and outcomes as perceived by the patients.

Availability of data and materials

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.



Assessment of Patterns of Patient-Reported Outcomes in Adults with Congenital Heart disease – International Study


Congenital Heart Disease


Generalized Linear Model


General Linear Mixed Models


Interquartile Range


New York Heart Association Classification


Patient Reported Outcomes


Standard Deviation


World Health Organization


  1. Dieleman JL, Templin T, Sadat N, et al. National spending on health by source for 184 countries between 2013 and 2040. Lancet. 2016;387(10037):2521–35.

    Article  Google Scholar 

  2. World Health Organization. The world health report 2006: working together for health. Geneva, Switerland: World Health Organization; 2006. Accessed 5 June 2019.

  3. World Health Organization. Public Health Infrastructure and Knowledge. Accessed 5 June 2019.

  4. Andersen RM. Behavioral Model of Families' Use of Health Services. Chicago: University of Chicago; 1968.

    Google Scholar 

  5. Andersen RM, Davidson PL, Baumeister SE. Improving access to care in America. In: Kominski EF, editor. Changing the U.S. health care system: key issues in health services, policy and management. 4th ed. San Francisco: Jossey-Bass; 2013. p. 33–69.

    Google Scholar 

  6. Sakai R, Fink G, Kumamaru H, Kawachi I. The Impact of Pediatrician Supply on Child Health Outcomes: Longitudinal Evidence from Japan. Health Serv Res. 2016;51(2):530–49.

    Article  PubMed  Google Scholar 

  7. Ananthakrishnan AN, Hoffmann RG, Saeian K. Higher physician density is associated with lower incidence of late-stage colorectal cancer. J Gen Intern Med. 2010;25(11):1164–71.

    Article  Google Scholar 

  8. Fleisher JM, Lou JQ, Farrell M. Relationship between physician supply and breast cancer survival: a geographic approach. J Community Health. 2008;33(4):179–82.

    Article  Google Scholar 

  9. Gulliford MC. Availability of primary care doctors and population health in England: is there an association? J Public Health Med. 2002;24(4):252–4.

    Article  Google Scholar 

  10. Farahani M, Price N, El-Halabi S, et al. Impact of Health System Inputs on Health Outcome: A Multilevel Longitudinal Analysis of Botswana National Antiretroviral Program (2002-2013). PLoS One. 2016;11(8):e0160206.

    Article  Google Scholar 

  11. Cassie KM, Cassie WE. Organizational and Individual Conditions Associated With Depressive Symptoms Among Nursing Home Residents Over Time. Gerontologist. 2012;52(6):812–21.

    Article  Google Scholar 

  12. Sarma S, Peddigrew C. The relationship between family physician density and health related outcomes: the Canadian evidence. Cah Sociol Demogr Med. 2008;48(1):61–105.

    PubMed  Google Scholar 

  13. Moons P, Kovacs AH, Luyckx K, et al. Patient-reported outcomes in adults with congenital heart disease: Inter-country variation, standard of living and healthcare system factors. Int J Cardiol. 2018;251:34–41.

    Article  Google Scholar 

  14. Riehm KE. Human Resources for Health: A Cross-national, Multilevel Study of Adolescent Health Outcomes and Inequalities: Department of Psychiatry, McGill University, Montreal; 2017.

  15. Riehm KE, Latimer E, Quesnel-Vallée A, Stevens GWJM, Gariépy G, Elgar FJ. Does the density of the health workforce predict adolescent health? A cross-sectional, multilevel study of 38 countries. J Public Health. 2018;41(1):e35–43.

    Article  Google Scholar 

  16. Tzelepis F, Sanson-Fisher RW, Zucca AC, Fradgley EA. Measuring the quality of patient-centered care: why patient-reported measures are critical to reliable assessment. Patient Prefer Adherence. 2015;9:831–5.

    Article  Google Scholar 

  17. US Department of Health and Human Service Food and Drug Administration. Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims. 2009. UKEwi8pseFg43eAhUEyIUKHTllBxEQFjAAegQIChAC& Accessed 5 June 2019.

  18. van der Linde D, Konings EE, Slager MA, et al. Birth prevalence of congenital heart disease worldwide: a systematic review and meta-analysis. J Am Coll Cardiol. 2011;58(21):2241–7.

    Article  Google Scholar 

  19. Mandalenakis Z, Rosengren A, Skoglund K, Lappas G, Eriksson P, Dellborg M. Survivorship in Children and Young Adults With Congenital Heart Disease in Sweden. JAMA Intern Med. 2017;177(2):224–30.

    Article  Google Scholar 

  20. Willems R, Werbrouck A, De Backer J, Annemans L. Real-world healthcare utilization in adult congenital heart disease: a systematic review of trends and ratios. Cardiol Young. 2019;29(5):553–63.

    Article  PubMed  Google Scholar 

  21. Van Bulck L, Luyckx K, Goossens E, Apers S, Kovacs AH, Thomet C, et al. Patient-reported outcomes of adults with congenital heart disease from eight European countries: scrutinising the association with healthcare system performance. Eur J Cardiovasc Nurs. 2019;18(6):465–73.

    Article  Google Scholar 

  22. Apers S, Kovacs AH, Luyckx K, et al. Assessment of Patterns of Patient-Reported Outcomes in Adults with Congenital Heart disease - International Study (APPROACH-IS): rationale, design, and methods. Int J Cardiol. 2015;179:334–42.

    Article  Google Scholar 

  23. Apers S, Kovacs AH, Luyckx K, et al. Quality of Life of Adults With Congenital Heart Disease in 15 Countries: Evaluating Country-Specific Characteristics. J Am Coll Cardiol. 2016;67(19):2237–45.

    Article  Google Scholar 

  24. Mitchell SC, Korones SB, Berendes HW. Congenital heart disease in 56,109 births. Incidence and natural history. Circulation. 1971;43(3):323–32.

    Article  CAS  Google Scholar 

  25. World Bank. World Bank Open Data. Accessed 15 May 2020.

  26. World Health Organization. Global Health Observatory data repository. Accessed 15 May 2020.

  27. National Development Council R.O.C. (Taiwan). Taiwan Statistical Data Book 2016. Accessed 28 May 2020.

  28. Shi L, Starfield B. Primary care, income inequality, and self-rated health in the United States: a mixed-level analysis. Int J Health Serv. 2000;30(3):541–55.

    Article  CAS  Google Scholar 

  29. Rihmer Z, Rutz W, Barsi J. Suicide rate, prevalence of diagnosed depression and prevalence of working physicians in Hungary. Acta Psychiatr Scand. 1993;88(6):391–4.

    Article  CAS  Google Scholar 

  30. Potter P, Barr N, McSweeney M, et al. Identifying nurse staffing and patient outcome relationships: a guide for change in care delivery. Nurs Econ. 2003;21(4):158–66.

    PubMed  Google Scholar 

  31. Hodgson A, Morgan D, Peterson R. Does Better Nurse Staffing Improve Detection of Depression and Anxiety As Secondary Conditions in Hospitalized Patients with Pneumonia? Nurs Econ. 2016;34(3):134–43.

    PubMed  Google Scholar 

  32. Zhang X, Oyama T. Investigating the health care delivery system in Japan and reviewing the local public hospital reform. Risk Manag Healthc Policy. 2016;9:21–32.

    Article  Google Scholar 

  33. Gonçalves F. A cross country explanation of performance of heath care systems: The consumer point of view using the Euro Health Consumer Index. Econ Model. 2011;28(1):196–200.

    Article  Google Scholar 

  34. Berghammer MC, Mattsson E, Johansson B, Moons P, Dellborg M. Comparison of participants and non-participants in patient-reported outcome surveys: the case of Assessment of Patterns of Patient-Reported Outcomes in Adults with Congenital Heart disease - International Study. Cardiol Young. 2017;27(3):427–34.

    Article  Google Scholar 

  35. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310:2191–4.

    Article  Google Scholar 

  36. Wet medisch-wetenschappelijk onderzoek met mensen (WMO) 1998. Retrieved from Accessed 15 May 2020.

  37. Central Committee on Research Involving Human Subjects. Your research: Is it subject to the WMO or not? Accessed 15 May 2020.

Download references


The authors would like to thank all individuals at the participating centres who made substantial contributions to APPROACH-IS and all APPROACH-IS participants.


This work was supported by the Research Fund – KU Leuven (Leuven, Belgium) through grant OT/11/033; by the Swedish Heart-Lung Foundation (Sweden) through grant number 20130607; by the University of Gothenburg, Centre for Person-centred Care (Gothenburg, Sweden); by the Cardiac Children’s Foundation (Taiwan) through grant number CCF2013_02. Dr. Erwin Oechslin currently holds the Bitove Family Professorship of Adult Congenital Heart Disease. Furthermore, this work was endorsed by and conducted in collaboration with the International Society for Adult Congenital Heart Disease. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Open access funding provided by University of Gothenburg.

Author information

Authors and Affiliations




The project is conducted in 24 centers from 15 countries around the world, and is led by a steering committee that designed and managed the project. In each center, a researcher or clinician was in charge for the data collection. Because of their substantial role in the acquisition of the data and substantial intellectual contribution while reviewing this manuscript, they were included as co-author. The contributions are as follows: Substantial contributions to the conception or design of the work: PM, KL, SA. Substantial contributions to the acquisition and interpretation of data for the work: EO (Toronto, Canada), CT (Bern, Switzerland), WB (Leuven, Belgium), JE (Chiba, Japan), MAS (Amsterdam, the Netherlands), CWL (Taipei, Taiwan), JLJ (Columbus, Ohio, United States), PK (Montréal, Canada), S Cook (Pittsburgh, United States), S Chidambarathanu (Chennai, India), LA (Cordoba, Argentina), KE (Oslo, Norway), MD (Gothenburg, Sweden), MB (Stockholm, Sweden), BJ (Umeå, Sweden), ASM (Edmonton, Canada), SM (Melbourne, Australia), MC (Msida, Malta), GV (Cincinnati, United States), AS (Lyon, France), SMF (Stanford, United States), KW (Saint-Louis, United States), EC (Milan, Italy) and SK (Omaha, United States). Substantial contributions to the analysis of data for the work: LVB, EG, KL, PM. In conclusion, all 29 authors meet the requirements of authorship as specified by the International Committee of Medical Journal Editors (ICMJE). All co-authors have read the manuscript and approved its submission to BMC Health Services Research. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Philip Moons.

Ethics declarations

Ethics approval and consent to participate

All participants provided written informed consent to participate and procedures were in accordance with the declaration of Helsinki [35]. The study has been approved by the following institutional review boards: University Hospitals Leuven/KU Leuven Belgium (University Hospitals Leuven, Leuven, Belgium; the coordinating center), Comité de Ética de Hospital de Niños de la Santísima Trinidad (Hospital de Niños de la Santísima Trinidad, Cordoba, Argentina), Monash Health HREC Low Risk Review Panel (Monash University, Clayton, Australia), Research Ethics Board of the University of Alberta (University of Alberta, Edmonton, Canada), Comité d’éthique de la recherche et du développement des nouvelles technologies - Institut de Cardiologie de Montréal (Montreal Heart Institute, Montreal, Canada), University Health Network Research Ethics Board (University of Toronto, Toronto, Canada), Comité d’Éthique de CHU de Lyon (Hospices civils de Lyon, Lyon, France), Instutional Ethics Committee of the Frontier Lifeline Hospital (IEC - FLLH) (Frontier Lifeline Hospital, Chennai, India), Il Comitato Etico dell’Ospedale San Raffaele Milano (IRCCS Ospedale San Raffaele, Milan, Italy), Instutional Review Board of Chiba Cardiovascular Center (Chiba University Hospital, Chiba, Japan), Research Ethics Committee Of the University of Malta (University of Malta, Msida, Malta), Regionale komiteer for medisinsk og helsefaglig forskningsetikk - Oslo universitetssykehus (Oslo University Hospital – Rikshospitalet, Oslo, Norway), Regionala etikprövningsnämnden i Göteborg (Göteborgs Universitet, Göteborg, Sweden), Kantonale Ethikkommission Bern (Swiss Ethics, Bern, Switzerland), National Taiwan University Hospital (NTUH) Research Ethics Committee (REC) (National Taiwan University, Taipei, Taiwan), Institutional Review Board of Cincinnati Childrens Hospital Med Center (Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, United States), Institutional Review Board of the Nationwide Children’s Hospital (Nationwide Children’s Hospital, Columbus, OH, United States), Institutional Review Board of the University of Pittsburgh (University of Pittsburgh, Pittsburgh, PA, United States), Institutional Review Board of Nebraska’s Health Science Center (University of Nebraska Medical Center, Omaha, NE, United States), The Washington University in St. Louis Institutional Review Board (Washington University, Saint-Louis, MO, United States) and Institutional Review Board of Stanford University (Stanford University, Stanford, CA, United States). In the Netherlands, ethics approval from the IRB was not required for survey research according to national regulations [36, 37]. The approval letters are available from the corresponding author upon request.

Consent for publication

Not applicable.

Competing interests

All authors declare to have no conflicts of interest regarding this manuscript.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Table S1.

Overview of variables and measurements. Table S2. Detailed overview of the healthcare system inputs and other healthcare system characteristics of countries included in APPROACH-IS. Figure S1. Andersen behavioral model of health services use, sixth revision (Reproduced from [13]). Permission to reuse figure was obtained from John Wiley and Sons. Figure S2. Mean and standard deviation of patient-reported outcomes in 3588 adults with congenital heart disease.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Van Bulck, L., Goossens, E., Luyckx, K. et al. Healthcare system inputs and patient-reported outcomes: a study in adults with congenital heart defect from 15 countries. BMC Health Serv Res 20, 496 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: