Int. J. Environ. Res. Public Health 2009, 6(9), 2387-2396; doi:10.3390/ijerph6092387

Article
Life Years at Risk: A Population Health Measure from a Prevention Perspective
Yuejen Zhao 1,2,* and Rosalyn Malyon 2
1
Institute of Advanced Studies, Charles Darwin University, Darwin NT 0909, Australia
2
Health Gains Planning Branch, Department of Health and Families, Northern Territory 0909, Australia; E-Mail:rosalyn.malyon@nt.gov.au
*
Author to whom correspondence should be addressed; E-Mail:yuejen.zhao@nt.gov.au; Tel.: +61-8-89858077; Fax: +61-8-89858075.
Received: 29 July 2008 / Accepted: 2 September 2009 / Published: 3 September 2009

Abstract

: This paper aims to present life years at risk (LYAR), a new measure of population health needs for primary, secondary and tertiary prevention, which classifies health outcomes by care type and distinguishes between positive and negative outcomes. It is determined by the probability of ill-health event, population size and life years lost, based on expected incidence, prevalence and mortality. The LYAR consists of two components: the observed LYAR, available using disability adjusted life years, and the avoided LYAR. Three examples are given to illustrate the calculation and application of the measure. The advantages, disadvantages and policy implications are also discussed.
Keywords:
health services needs and demand; risk assessment; health status indicators

1. Introduction

Population health refers to overall level and dispersion of health status in a defined population [1]. The assessment of population health needs is central to planning and providing health services to improve population health and utilize health resources effectively and efficiently [2]. Improving population health means both increasing overall level of health and reducing health gaps by eliminating or alleviating health hazards and preventing the occurrence of disease, injury, disability and premature death [3,4]. Accordingly, the purpose of health services can be considered to be loss aversion, and the extent of the risk of ill-health in the population indicates the level of needs for health care.

Healthy life years were proposed as a measure for population health in the 1960s [5], and further developments have included health indices [69], quality adjusted life years [1012] and disability adjusted life years (DALYs) [1315]. Of these measures, the DALY has become widely known as the standard unit of measure in burden of disease and injury studies (referred to hereafter as burden of disease studies). It can also be applied in the economic evaluation of health interventions [16]. The DALY aggregates the years of life lost (YLL) due to premature death and the years lost due to disability (YLD) into a single number. The YLD is calculated on the basis of the duration and severity weight of the disability (the relative preference for the disability state compared with full health and death) to enable compatibility with YLL [13].

Although there has been much criticism of the DALY [1719], it represents a significant improvement in accuracy of health needs measurement, because DALYs combine both fatal and non-fatal outcomes and change the measurement from counting number of deaths to calculating years of healthy life lost [20]. DALYs have been widely used to quantify and compare population health needs as an indicator of where resources should be directed to, given the current level of access to health care. A recent burden of disease study for Indigenous Australians identified the Indigenous population as having a disproportionately large share of total DALYs [21]. While this suggests that they should be a priority for health care services, the full extent of their needs is unclear as the DALY only represents the negative health outcomes (mortality and morbidity) that arise despite past and continuing health investments. The DALY provides no information on the extent of positive health outcomes (avoided disease, injury, disability and death). The negative health outcomes are insufficient for resource allocation [22]. The inclusion of positive outcomes forms a legitimate part of health needs assessment, because continued investment may be needed to keep negative outcomes low. For example, renal dialysis can postpone death for patients with renal failure and reduce DALYs, but it does not mean less need for health care. Measuring total health needs would better inform decisions on resource allocation and identify whether gaps in health status between sub-groups in the population arise from a lack of investment, relative differences in investment or level of access to the services.

A further limitation of the DALY is its inability to distinguish between needs for different types of health care [17,23]. A disease may have a high share of total DALYs, but this does not inform on the relative impact of different care types. For example, acute hospital care may provide medical and surgical acute care to reduce mortality, but more efforts may be needed to prevent the occurrence of the disease and disability [24]. Such information could facilitate prioritization between different sectors of health care and assist in the comparison of different types of intervention.

This study seeks to explore the use of the DALY as a measure of relative health needs particularly where there are differences in levels of access to health care. The objective of this paper is to present a new measure: life years at risk (LYAR) that classifies DALYs in accordance with different health care types from a prevention perspective, and incorporates both positive and negative health outcomes. Examples of the calculation and application of the LYAR are provided, and its limitations and policy implications discussed.

2. Method

Concepts from preventive medicine were used to construct the LYAR measure. From this perspective, health services encompass three types of care: primary prevention, which reduces incidence of disease or injury; secondary prevention, which lessens and avoids disability; and tertiary prevention, which saves lives and delays deaths [25,26]. Table 1 summarizes the function, target population and intervention features for the three types of health care. As a measure of health needs for the three care types, the LYAR represents the expected number of healthy life years at risk of disease, injury, disability and premature death over a given period of time in a specified population (refer to last column of Table 1).

The LYAR is based on the notion of risk, which combines the probability of an event occurring and the consequence of its occurrence [27]. In terms of health, risk can be expressed as the probability of illness or injury occurring and the fatal or non-fatal health outcomes from that event [28,29]. Thus, the magnitude of the LYAR depends on the proportion of the population falling ill or being injured, the level and duration of disability or years of life prematurely lost as a result of those diseases and injuries. Thus, the LYAR can be determined quantitatively by

LYAR = probability of the event × population × expected healthy life years lost due to the event

Like DALYs, the expected values for these variables can be ascertained from clinical and epidemiological evidence, disease and death registers, service activity data and expert opinion [13]. In regard to the type of health care, the LYAR consists of three components:

Total LYAR = LYAR of incidence + LYAR of disability + LYAR of mortality

LYAR of incidence (LYAR(I)) for a disease or injury reflects health needs for primary prevention, which equals expected incidence of an uncomplicated condition multiplied by the duration of the condition times the severity weight. Expected incidence includes two elements: prevented and observed incidences. Thus,

LYAR ( I ) = expected incidence × duration × disability weight = ( prevented incidence + observed incidence ) × duration × disability weight = observed incidence × duration × disability weight / ( 1 α ( I ) )
where attainment α (I) = prevented incidences/(prevented incidences + observed incidences) for 0 ≤ attainment ≤ 1, under the assumption that the prevented cases would have the same duration and disability weight as the actual cases observed. The observed part of LYAR(I) could be estimated using the YLD for uncomplicated cases from the burden of disease studies.

LYAR of disability (LYAR(D)) represents health needs for secondary prevention, which equals expected incidence of a disabling complication of the condition multiplied by the duration of the complication times the disability weight. For a steady-state population, prevalence is approximately equal to incidence times duration [30]. The expected disability prevalence comprises two elements: prevented and observed disability prevalence. Thus,

LYAR ( D ) = expected incidence of disabilities × duration × disability weight = expected prevalence of disabilities × disability weight = ( prevented disabilities + observed disabilities ) × disability weight = observed disabilities × disability weight / ( 1 α ( D ) )
where attainment α (D) = prevented disabilities/(prevented disabilities + observed disabilities), assuming the prevented disabilities would have the same duration and disability weight as the actual disabilities observed. The observed LYAR(D) could be estimated using the YLD for complicated cases from the burden of disease studies.

LYAR of mortality (LYAR(M)) describes health needs for tertiary prevention, which is the expected YLL attributable to the condition and incorporates two elements: prevented and observed YLL. Thus,

LYAR ( M ) = expected YLL = ( prevented YLL + observed YLL ) = observed YLL / ( 1 α ( M ) )
where attainment α (M) = prevented YLL/(prevented YLL + observed YLL). The observed LYAR(M) could be estimated by the actual YLL from burden of disease studies.

While observed LYAR can draw largely on information from burden of disease studies, prevented LYAR is less directly observable. The following data sources may be helpful for estimating these components:

  • Historical incidence, prevalence and mortality data;

  • Epidemiological expectation, for example, dengue fever and influenza incidences in a neighboring state of similar climate or global epidemic pattern;

  • Empirical estimation, for example, inferior health outcome in a neighboring population group without intervention;

  • Service activity data;

  • Risk factor analysis; and

  • Early detection and screening data.

Burden of disease studies have been used to compare the health status of sub-groups within the population. If attainment (the proportion of prevented cases) is assumed to be equal across these sub-groups, each group’s share of total LYAR will be the same as its share of total DALYs. It can be shown that:

LYAR Share = expected LYAR expected LYAR = ( prevented LYAR + observed LYAR ) ( prevented LYAR + observed LYAR ) = observed LYAR / ( 1 α ) observed LYAR / ( 1 α ) = DALYs DALYs
where attainment α =prevented LYAR/(prevented LYAR + actual LYAR). If, however, attainment is less in some population groups, for example, there is less access to services or a greater degree of unmet needs, the DALY will be oversensitive to negative outcomes for these groups. This oversensitivity occurs because the actual LYAR prevented is smaller than that derived from uniform α, and the sub-group’s DALY share would be greater. Table 2 gives a hypothetical example to demonstrate the rationale why the DALY is oversensitive to negative health outcomes. The total true amount of ill-health is shown by the expected LYAR (column 1) and its share (column 2). If not all members of groups A and C have access to preventive care, but all of Group B does (column 3), then assuming equal effectiveness of the care, a smaller proportion of life years will be prevented in groups A and C (column 4). The observed number of actual life years, as captured in the DALY (column 5), would be greater for groups A and C and their DALY share (column 6) greater than it would have been (column 2), had access to preventive care being equal across the groups. In this sense, the DALY is more generous to disadvantaged groups with inadequate access to health care services.

Existing data for ischemic heart disease (IHD) and type 2 diabetes were used to illustrate the calculation and application of the LYAR. For IHD, incidence and mortality data were taken from Australian hospital morbidity and mortality databases [31]. The standard life expectancy and the disability weight for angina pectoris (0.105) were obtained from the 2003 Australian burden of disease study [32] with the expected duration by sex and age ranging from 2.4 to 24.5 years. For type 2 diabetes, the observed LYAR rates were derived from the Northern Territory (NT) and Australian burden of disease studies [32,33]. The disability weights and disease sequelae for type 2 diabetes were identical between the two studies. Due to multiple data sources in burden of disease studies, it was necessary to use DISMOD software to check and maintain the internal plausibility and mathematical consistency between incidence, prevalence, mortality and duration estimates [34]. The prevented component of LYAR was excluded as it only sought to demonstrate the split of LYAR by types of care for type 2 diabetes. The observed LYAR(I) was based upon uncomplicated cases of type 2 diabetes. The LYAR(D) was derived using the prevalence of retinopathy, cataract, glaucoma, diabetic foot, amputations, renal failure, neuropathy, peripheral vascular disease, IHD and stroke attributable to type 2 diabetes. The LYAR(M) was determined by underlying cause of death. The NT Indigenous population for the period 1999–2003 was derived on the basis of the total estimated resident population (ERP) and the experimental Indigenous ERP [35,36]. The national population was the Australian ERP in 2003 [37]. The rate ratios and their significances were inferred by using conventional statistical methods [38].

3. Results

During 2001–2002, the observed IHD incidences in Australia were estimated to be 48,700 and the prevented incidences were about 12,000 (based on historical incidence level prior to intervention) [31]. This data generated an observed IHD LYAR(I) during 2001–2002 of 33,600 and a prevented LYAR(I) of 3,800. Thus, total LYAR(I) would be 37,400 and the attainment factor for IHD primary prevention about 20%.

In 2006, the observed number of actual deaths attributable to IHD was about 23,000 with the prevented deaths approaching 73,000 [39]. This data generated an observed IHD LYAR(M) in 2006 of about 177,900 and a prevented LYAR(M) of 564,600. Thus, total IHD LYAR(M) in 2006 would be 742,500 and the attainment factor for IHD tertiary prevention about 76%.

The next example demonstrates the application of observed LYAR on type 2 diabetes. Table 3 shows the type 2 diabetes rate ratios of NT Indigenous LYAR per 1000 population compared to the Australian average by age groups and three types of care. The total LYAR per 1000 population for both NT Indigenous males and females were over four times the Australian rates. At all levels (age group, gender and type of care), the observed LYAR per 1000 population for the NT Indigenous population was greater than that for the Australian general population. The differences were of statistical significance (P < 0.01) except for LYAR(I) at ages 70 years and over. The rate ratios increased with descending age. The rate ratios for secondary prevention were greatest, over three times those for primary and tertiary prevention. The most striking differentials were in the age group 0–29 years for secondary and tertiary prevention.

4. Discussion

In line with the DALY, the LYAR measures the health needs of a defined population based on the probability of ill-health, the expected duration of illness, the presence of disability and reduced longevity [40]. When comparing sub-groups within the population, the DALY share for groups with relatively poorer access to health care is likely to be higher than might otherwise be the case using LYAR, which considers the whole need (positive and negative health outcomes). Such amplification would give greater emphasis to their relative circumstances and may be desirable depending upon the purpose of the evaluation. If policy makers are seeking evidence to spend additional resources, then the DALY share will direct resources towards the population with more negative outcomes, but it will offer no insight about what type of care is needed. If policymakers are evaluating the effectiveness of current spending, LYARs may be the better measure. By including both prevented and observed instances, the LYAR provides insight into the remaining burden of disease as well as the achievements from on-going investments by types of care. Effective investments in prevention would be expected to lead to a lower DALY because they will lower incidence, disability and/or mortality. This does not, however, mean a lesser need for health services if on-going services are required to maintain the current level of prevented cases. In contrast, the LYAR would not be affected because it shows the whole health need by including the prevented instances.

The division of the LYAR according to the three types of health care would allow assessment of the relative contributions of pre-emptive, curative and life-saving interventions [41]. This dimension of the measure may be valuable for comparing the benefits of interventions in different sectors. It would also show the effect that interventions may have on other care types. Effective primary prevention may also reduce the LYAR for secondary and tertiary prevention because it lowers the occurrence of the disease. Effective secondary prevention (i.e. treatment of disease and injury) can reduce the LYAR for tertiary care. Capturing these dynamics could better inform on the relative impact of interventions in different care sectors.

The prevention perspective of the LYAR can shift the focus beyond medical solutions and capture the impact on population health of other intercessions to improve education, employment, community infrastructure and other socio-economic factors, including poverty and the social and economic participation of people with disabilities. Furthermore, the LYAR can capture improvements in the functionality of disabled persons by reassessing the disability weight to reflect the change in the level and sequelae of disability associated with a disease or injury. Although rehabilitative services improve quality of life and prevent premature death, they do not, however, prevent the condition.

Health funding in Australia has largely overlooked mortality and morbidity [42]. This omission may be due to concerns that the mortality and DALY measures do not recognize the full needs for health care. The LYAR is a step toward an alternative measure that can more inform funding decisions by presenting a full picture of health needs.

There are some limitations to the LYAR measure. First, it would require information on both observed and prevented disease and injury. Thus, its data needs are considerable. While data on observed cases may be available, estimating prevented cases may be more difficult as it may not be possible to readily observe their numbers. Historical incidence may be also difficult to estimate accurately if appropriate and reliable records have not been kept. Second, LYAR will be vulnerable to errors in assumptions and interpretations of the occurrence and experience of disease and injury. For example, rehabilitative services may not change the LYARs if the disability weights only reflect diagnoses, not functionality and quality of life. Third, the LYAR applies the concept of disability weights from the DALY so may be subject to similar criticisms to the DALY [18]. Further research is needed to consider the prospect of wider application and refinement of the population health needs measure that covers both positive and negative health outcomes.

References

  1. Kindig, D; Stoddart, G. What is population health? Am. J. Public Health 2003, 93, 380–383.
  2. Wright, J; Williams, R; Wilkinson, JR. Development and importance of health needs assessment. BMJ 1998, 316, 1310–1313.
  3. Hancock, T; Labonte, R; Edwards, R. Indicators that count! Measuring population health at the community level. Can. J. Public Health 1999, 90, S22–26.
  4. Murray, CJL; Salomon, JA; Mathers, C. A critical examination of summary measures of population health. Bull. World Health Organ 2000, 78, 981–994.
  5. Sanders, BS. Measuring community health levels. Am. J. Public Health 1964, 54, 1063–1070.
  6. Grogono, AW; Woodgate, DI. Index for measuring health. Lancet 1971, 2, 1024–1026.
  7. Ware, JE. Scales for measuring general health perceptions. Health Serv. Res 1976, 11, 396–415.
  8. Martini, CHM; McDowell, I. Health status: patient and physician judgments. Health Serv. Res 1976, 11, 508–515.
  9. Hunt, SM; McEwen, J. The development of a subjective health indicator. Soc. Health Ill 1980, 2, 231–246.
  10. Klarman, HE; Francis, JO; Rosenthal, GD. Cost-effectiveness analysis applied to the treatment of chronic renal disease. Med. Care 1968, 6, 48–54.
  11. Packer, A. Applying cost-effectiveness concepts to the community health system. Oper. Res 1968, 16, 227–253.
  12. Fanshel, S; Bush, JW. A health status index and its application to health services outcomes. Oper. Res 1970, 18, 1021–1066.
  13. Murray, CJL; Lopez, AD. The Global Burden of Disease: a Comprehensive Assessment of Mortality and Disability from Diseases, Injuries and Risk Factors in 1990 and Projected to 2020; Harvard School of Public Health: Boston, MA, USA, 1996.
  14. Murray, CJL; Lopez, AD. Progress and directions in refining the global burden of disease approach: a response to Williams. Health Econ 2000, 9, 69–82.
  15. World Health Organization. The World Health Report 2000—Health Systems: Improving Performance; WHO: Geneva, Switzerland, 2000.
  16. Murray, CJL; Evans, DB; Acharya, A; Baltussen, MPM. Development of WHO guidelines on generalized cost-effectiveness analysis. Health Econ 2000, 9, 235–251.
  17. Mooney, G; Leeder, S. Priority setting in health care: unburdening from the burden of disease. Editorial. Aust. N. Z. J. Public Health 1997, 21, 680–681.
  18. Anand, S; Hanson, K. Disability-adjusted life years: a critical review. J. Health Econ 1997, 16, 685–702.
  19. Reidpath, DD; Allotey, PA; Kouame, A; Cummins, RA. Measuring health in a vacuum: examining the disability weight of the DALY. Health Policy Plan 2003, 18, 351–356.
  20. Gold, MR; Stevenson, D; Fryback, DG. HALYs and QALYs and DALYs, Oh My: Similarities and differences in summary measures of population health. Ann. Rev. Public Health 2002, 23, 115–134.
  21. Vos, T; Barker, B; Stanley, L; Lopez, AD. The burden of disease and injury in Aboriginal and Torres Strait Islander peoples 2003; The University of Queensland: Brisbane, Australia, 2007.
  22. Carr-Hill, R; Sheldon, T. Rationality and the use of formulae in the allocation of resources to health care. J. Publ. Health 1992, 14, 117–126.
  23. Mont, D. Measuring health and disability. Lancet 2007, 369, 1658–1663.
  24. Sullivan, DF. Disability components for an index of health; National Center for Health Statistics: Hyattsville, MD, USA, 1971.
  25. Jekel, JF; Katz, DL; Elmore, JG. Epidemiology, Biostatistics, and Preventive Medicine, 2nd ed ed.; Elsevier: New York, NY, USA, 2001.
  26. National Public Health Partnership. The Language of Prevention; NPHP: Melbourne, Australia, 2006.
  27. Lupton, D. Risk; Routledge: New York, NY, USA, 1999.
  28. Froberg, DG; Kane, RL. Methodology for measuring health-state preferences–III: Population and context effects. J. Clin. Epidemiol 1989, 42, 585–592.
  29. Gakidou, EE; Murray, CJL; Frenk, J. Defining and measuring health inequality: an approach based on the distribution of health expectancy. Bull. World Health Organ 2000, 78, 42–54.
  30. Freeman, J; Hutchison, GB. Prevalence, incidence and duration. Am. J. Epidemiol 1980, 112, 707–723.
  31. Australian Institute of Health and Welfare. Heart, Stroke and Vascular Diseases—Australian Facts 2004; AIHW: Canberra, Australia, 2004.
  32. Begg, S; Vos, T; Barker, B; Stevenson, C; Stanley, L; Lopez, AD. The Burden of Disease and Injury in Australia 2003; AIHW: Canberra, Australia, 2007.
  33. Zhao, Y; You, J; Guthridge, S. Burden of Disease and Injury in the Northern Territory, 1999–2003; Department of Health and Families: Darwin, Australia, 2009. (in press).
  34. World Health Organisation. DISMOD II; WHO: Geneva, Switzerland, 2001.
  35. Australian Bureau of Statistics. Population by Age and Sex, Australian States and Territories; ABS: Canberra, Australia, 2003.
  36. Australian Bureau of Statistics. Experimental Projections of Aboriginal and Torres Strait Islander Australians, ATSIC regions, 2001 to 2009; ABS: Canberra, Australia, 2004.
  37. Australian Bureau of Statistics. Population by Age and Sex, Australian States and Territories; ABS: Canberra, Australia, 2003.
  38. Armitage, P; Berry, G; Matthews, JNS. Statistical Methods in Medical Research; Blackwell Science: Malden, MA, USA, 2002.
  39. Australian Institute of Health and Welfare. Impact of Falling Cardiovascular Disease Death Rates: Deaths Delayed and Years of Life Extended; AIHW: Canberra, Australia, 2009.
  40. Reid, RJ; Roos, NP; MacWilliam, L; Frohlich, N; Black, C. Assessing population health care need using a claims-based ACG morbidity measure: A validation analysis in the province of Manitoba. Health Serv. Res 2002, 37, 1345–1364.
  41. Bennett, J. Investment in Population Health in Five OECD Countries; OECD: Paris, France, 2003.
  42. Commonwealth Grant Commission. Assessing Community and Other Health Services for the 2010 Review. Discussion Paper 2007/21-S; CGC: Canberra, Australia, 2007.
Table Table 1. Health needs measurement from a prevention perspective.

Click here to display table

Table 1. Health needs measurement from a prevention perspective.
Health care typeHealth care aimTarget populationIntervention featuresProposed health needs measure
Primary preventionReduce incidencePopulation at riskEliminate or alleviate causes, risk factors or determinants of the condition; disease surveillance; immunization, education, health legislationLYAR of incidence of disease or injury
Secondary preventionReduce disabilityPatients with early stage of the conditionCure disease or mitigate impairments by early detection, prompt diagnosis and treatmentLYAR of disability
Tertiary preventionReduce mortalityPatients with late stage of the conditionMaintain function or prolong survival; life-saving interventions, surgeries and procedures; rehabilitative and palliative careLYAR of mortality

Note: LYAR=life years at risk. Modified from Reference [25].

Table Table 2. An hypothetical example (in life years, assuming effectiveness α = 50%).

Click here to display table

Table 2. An hypothetical example (in life years, assuming effectiveness α = 50%).
Expected LYAR (‘000)LYAR shareTreated LYAR (‘000)Prevented LYAR (‘000)Actual DALYs (‘000)DALY share
(1)(2)(3)(4) = (3) × α(5) = (1) – (4)(6)
Group A305%105258%
Group B50083%50025025076%
Group C7012%36185216%
Table Table 3. Observed life years at risk of type 2 diabetes per 1,000 population by age group and type of care, Northern Territory Indigenous population 1999–2003 vs Australia 2003.

Click here to display table

Table 3. Observed life years at risk of type 2 diabetes per 1,000 population by age group and type of care, Northern Territory Indigenous population 1999–2003 vs Australia 2003.
NT IndigenousAustraliaRate ratio

MaleFemaleMaleFemaleMaleFemale

LYAR (I) for primary prevention
  0–294.98.00.20.528.9*17.6*
  30–4917.421.56.45.12.7*4.2*
  50–6918.520.59.57.92.0*2.6*
  70+8.110.26.68.01.21.3

  Total9.012.64.44.12.0*3.1*

LYAR (D) for secondary prevention
  0–292.72.40.00.01279.2*163.8*
  30–4944.836.10.60.773.9*51.9*
  50–69113.693.24.23.227.2*29.1*
  70+49.742.79.79.75.1*4.4*

  Total22.220.11.81.912.3*10.7*

LYAR (M) for tertiary prevention
  0–290.20.40.00.0134.9*113.6*
  30–4918.415.80.50.338.6*58.6*
  50–6976.369.86.93.011.0*23.6*
  70+122.5117.536.633.73.4*3.5*

  Total12.413.04.54.22.8*3.1*

LYAR for total health care
  0–297.810.80.20.545.4*22.6*
  30–4980.673.37.56.110.8*12.0*
  50–69208.4183.520.614.010.1*13.1*
  70+180.3170.352.951.43.4*3.3*

  Total43.645.710.710.24.1*4.5*

Estimated resident population
  0–29945858918041283843966432
  30–49339033580929335082974180
  50–69111741280820304352022961
  70+210928937793151046254

  Total141771140690987164210009827

Note: LYAR = life years at risk, I = Incidence, D = Disability, M = Mortality; NT = Northern Territory, Australia;*P < 0.01.

Int. J. Environ. Res. Public Health EISSN 1660-4601 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert