Next Article in Journal
Acute Effects of Kinesiology Taping Stretch Tensions on Soleus and Gastrocnemius H-Reflex Modulations
Previous Article in Journal
Use of Digital Media for Home-Based Sports Activities during the COVID-19 Pandemic: Results from the German SPOVID Survey
Previous Article in Special Issue
Paving the Way for Outdoor Play: Examining Socio-Environmental Barriers to Community-Based Outdoor Play
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Can the Timed and Targeted Counseling Model Improve the Quality of Maternal and Newborn Health Care? A Process Analysis in the Rural Hoima District in Uganda

1
Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, 6229 GT Maastricht, The Netherlands
2
Independent Researcher, 627 Ntinda, Kampala 10302, Uganda
3
World Vision International, Hoima Plot 15B, Nakasero Road, Kampala 5319, Uganda
4
Lutheran World Federation, Kyangwali Plot 1401 Gaba Road, Kampala 5827, Uganda
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(9), 4410; https://doi.org/10.3390/ijerph18094410
Submission received: 1 February 2021 / Revised: 16 April 2021 / Accepted: 16 April 2021 / Published: 21 April 2021
(This article belongs to the Special Issue Child-friendly Planning and Public Health)

Abstract

:
Each year, more than half a million women die worldwide from causes related to pregnancy and childbirth, and nearly 4 million newborns die within 28 days of birth. In Uganda, 15 women die every single day from pregnancy and childbirth-related causes, 94 babies are stillborn, and 81 newborn babies die. Cost-effective solutions for the continuum of care can be achieved through Village Health Teams to improve home care practices and health care-seeking behavior. This study aims at examining the implementation of the timed and targeted counseling (ttC) model, as well as improving maternal and newborn health care practices. We conducted a quantitative longitudinal study on pregnant mothers who were recruited on suspicion of the pregnancy and followed-up until six weeks post-delivery. The household register was the primary data source, which was collected through a secondary review of the ttC registers. All outcome and process variables were analyzed using descriptive statistics. The study enrolled 616 households from 64 villages across seven sub-counties in Hoima district with a 98.5% successful follow-up rate. Over the course of the implementation period of ttC, there was an increase of 29.6% in timely 1st antenatal care, 28.7% in essential newborn care, 25.5% in exclusive breastfeeding, and 17.5% in quality of antenatal care. All these improvements were statistically significant. The findings from this study show that the application of the ttC model through Village Health Teams has great potential to improve the quality of antenatal and newborn care and the health-seeking practices of pregnant and breastfeeding mothers in rural communities.

1. Background

Each year, more than half a million women die worldwide from causes related to pregnancy and childbirth, and nearly 4 million newborns die within 28 days after birth [1]. In Uganda, 15 women die every day from pregnancy and childbirth-related causes, 94 babies are stillborn, and 81 newborn babies die. This equates to 695,701 deaths each year due to complications during pregnancy, childbirth, and in the first month [2]. Therefore, the link between a mother’s health and that of her newborn is crucial for addressing maternal and infant mortality and morbidity, since the majority of preventable maternal and infant deaths occur during pregnancy, childbirth, and the immediate postpartum period [3]. Ensuring that mothers and newborns receive timely, high-quality care during these critical periods is essential. At baseline, attending four or more ANC was 55.1% [4]; therefore, this means almost half of the mothers miss goal oriented services such as screening for, preventing, and treating HIV, anemia, malaria, urinary tract infections, pre-eclampsia/eclampsia, preterm delivery, and other complications that can be life-threatening to both mother and baby [5]. Supporting women to deliver under the supervision of a skilled birth attendant and having the necessary supplies is key to preventing leading causes of maternal and newborn deaths [6].
One of the globally accepted approaches aimed at reducing maternal and infant mortality is the continuum of care approach. This approach advocates individuals having access to key services throughout the life cycle, including pregnancy, childbirth, and the postnatal period [7], to prevent maternal and newborn mortality [8]. The goal of the continuum of care approach is to provide women with the reproductive health care they need during and after pregnancy and delivery, and to provide newborns with the opportunity to grow into healthy children [9]. Emphasis is placed on a unified integrated approach for mothers and newborns and the role of facility-based care, outreach, and community-based or home care [10].
Further still, when dealing with pregnant and lactating women at the household level, an appropriate approach needs to be selected that includes the promotion of access to care through the life cycle. This approach should also focus on the involvement of male partners in household based care. This could potentially further contribute to the reduction of gender-based violence during the reproductive health period. Studies in Tanzania showed that intimate partner violence is one of the main challenges to women’s health and wellbeing during the pregnancy period. Especially young married women are vulnerable, affecting their maternal health and pregnancy outcomes. Therefore, this calls for the proper implementation of community-based interventions to support pregnant women to seek antenatal care services and to raise awareness regarding the reduction of intimate partner violence [11].
Cost-effective solutions such as working with Uganda’s community health worker cadre, called the Village Health Teams (VHTs) [12], are available. This could produce rapid improvements, but urgency and commitment are required to implement them [13]. Continuum of care through VHTs can be achieved through a combination of policies and strategies to improve home care practices, health care-seeking behavior, and health care services at the health facility throughout life, building on existing programs and packages. Home visits by VHTs together with community mobilization activities have been associated with reduced neonatal mortality and stillbirths in settings with high neonatal mortality and poor access to facility-based health care [14]. Such home visits by VHTs have been recommended by the WHO, UNICEF, and partners since 2009 [15]. In Uganda, a previous study has shown that pregnancy and newborn home visits by VHTs combined with health facility strengthening is feasible and can improve maternal and newborn care practices in a typical rural Ugandan setting [16]. The use of locally recruited, trained VHTs linked to a health facility-based supervision system to deliver home-based preventive care, was found to be acceptable and effective in rural geographical locations [17].
One of the approaches VHTs can use, the timed and targeted counselling (ttC), [18] is a family-inclusive behavior change communication approach, targeting households with pregnant women through the continuum of care. The timed and targeted counselling model encompasses a wide range of life-saving health practices through appropriately timed messages, delivered using interactive storytelling. It applies a dialogue counseling methodology, based on the assessment of current needs and practices and negotiation of progressive improvements [19]. The VHTs seek to engage parents and family decision-makers, leading to a family-inclusive and gender-transformative model of pregnancy and newborn care, in which the positive contribution of male partners is emphasized. In practical terms, the VHTs organize immunization outreaches, perform home visits and various other activities that are involved with ttC, and attend other activities in the district concerning the community. The home visits are implemented for pregnant mothers, and after birth, the children are followed up and registered. VHTs also monitor the use of certain services, for instance whether mothers come for antenatal care at the health facility and whether they have had an HIV test [4].
The behavioral change communication model of choice for this study, ttC, fits into the Healthy Places for Children [20] that underpins the critical role of engagement, common vision, and collaboration. These are seen in the involvement of the district local government: how the health providers—in this regard, the health workers and the VHT—are at the center of providing service and demand creation. The model also focuses on social inclusion at the family level where the discussions and negotiations include all influencers such as the husband, in-laws, and grandmothers.
In the current study, ttC targeted pregnancy outcomes such as referring to goal-oriented ANC, preparedness for birth, male involvement in pregnancy and newborn care, and delivery at a health facility, which are considered as contributing to positive birth outcomes [21]. This study aimed at assessing the maternal and newborn health care before and after the implementation of ttC in the rural Hoima district in Uganda. This was done by comparing the quality of care and essential services during the pregnancy and newborn period.

2. Methods

2.1. Study Design

This was a longitudinal study, with an evaluation of the implementation of ttC as the subject of follow-up and changes overtime. Quantitative measures were used to evaluate the implementation process and to investigate change over time in pregnancy and newborn care. The study team continues to work tirelessly on plans to publish a paper focusing on the effects of ttC implementation.

2.2. Study Area

The study was carried out in the Hoima district, located in the western part of Uganda. Hoima has 15 sub-counties and has a population of over 500,000 [22]. The area used to be predominantly inhabited by the native Banyoro, but since the discovery of oil, new economic developments such as road construction, and the establishment of factories and fishing activities, people from other locations in Uganda have been moving to the district. However, most of the households depend on subsistence agriculture and small-scale cash crops such as tobacco and coffee [23]. ttC was implemented in the Hoima district, because this district has one of the highest maternal mortality rates in Uganda [24].

2.3. Study Population

The study population consisted of pregnant mothers registered by VHTs from the time they suspected they were pregnant. The study followed up these participants from pregnancy until six weeks post-delivery. To be included, women had to have either an ANC card (which is used to record vital information about the pregnant woman’s service provision.) [25] or a mother–child passport [26]. After delivery, women with an ANC card would additionally be provided with a Child Health Card [27]. The VHTs [28] were considered the secondary study subjects, since they were the ones responsible for performing the ttC household visits and the counseling as well as documenting in the registers. There were no exclusion criteria for the included women, other than that the pregnant women had to have lived in the project area until at least six weeks after childbirth. Data were collected by 65 VHTs from eight sub-counties in Hoima, with a total of 616 participating women and households. These included Buseruka, Upper and Lower Kabwoya, Busisi, Kigorobya, Bugambe, Kitoba and Mparo.

2.4. Ethics Approval

Ethical approval and clearance were received from the Institutional Review Board (IRB) for the School of Public Health, Makerere University College of Health Sciences, the Higher Degrees, Research and Ethics Committee (HDREC) protocol 730 on February 2nd, 2020. In line with the Ugandan government regulations, the approved protocol was then submitted to the Uganda National Council for Science and Technology (UNCST) for approval. This was approved with a research registration number: HS574ES. Since this study was regarded as an anonymized desk review, there was no need for the consent of the pregnant women or mothers (Trial registration: PACTR, PACTR202002812123868. Registered on 25 February 2020—Retrospectively registered, http://www.pactr.org/ PACTR202002812123868, accessed on 25 February 2020.

2.5. Intervention

The ttC intervention [29] was rolled out as a behavioral change communication model for pregnancy and newborn care. For ttC to be implemented, a five-day central hands-on training was given for the VHTs by the district health team (these are district health officials qualified as public health assistants that offer supervision and guidance to the VHT), using a two-way counseling approach (participatory approach). Then, these VHTs translated the acquired knowledge to the expectant mothers and mothers with neonates, as they delivered the ttC package during their home visits.
VHTs were taught how to communicate with the pregnant women and mothers with neonates, as well as the household heads (mostly the fathers) and any other influential individuals in the family. During the training, the VHTs were equipped with a participant’s manual, and a ttC-specific household register to record participants and kick-start the timely follow-up process [30].
The ttC visits pass on a particular message at set times to trigger the household to encourage the mother to go for a given service at the health center, which is relevant at that moment in pregnancy or the neonatal period. The visits that the VHT makes are in line with the Ministry of Health’s goal-oriented ANC [24] services package and the institutional delivery framework of the road to the reduction of maternal and newborn death in Uganda [31]. For this study, the focus is on the four visits made during pregnancy, the three visits made when the child is in the newborn period for essential newborn care practices, and finally, a fourth postpartum visit at six weeks to enhance exclusive breastfeeding and routine immunization. The VHTs were convened by the sub-county assistant on a monthly basis to ensure they adhered to the prescribed household visits and registered each action in the ttC household register. The visits are further specified in Table 1.
In a ttC pilot study among 1556 mothers in Palestine, [32] CHWs targeted mothers with timely key messages and support for positive feeding and caring practices during organized home visits throughout 12 months. The pilot study showed that practices improved significantly among participating mothers: exclusive breastfeeding until six-months increased from 33% to 48.4%; the proportion of mothers who report having four or more antenatal visits increased from 58.4% to 63.6%, and the proportion of mothers who received at least two post-natal visits increased from 26.8% to 52.4%.

2.6. Study Instrument and Variables

The ttC household register (this is the register that the VHT uses to tick off or confirm the key services or behaviors the household has implemented at each visit) was the primary data source. Data were collected through secondary review of the existing filled out ttC household registers for mothers who had been followed from the time the VHT identified them as pregnant to at least six weeks after childbirth. These data were collected by the VHTs during their household visits. While the VHTs are doing household counseling, they record information in the household register. The study variables consist of two groups: those that were measured at baseline as well as follow-up, and those that are specific to ttC and therefore were not applicable during the baseline timeframe and thus only collected at follow-up (see Table 2 for details on the study variables).
The principle data collectors were the VHTs who collected the data during household visits at given times within the pregnancy and newborn period [28]. The VHTs were established by the Ministry of Health to empower communities to take part in the decisions that affect their health, mobilize communities for health programs, and strengthen the delivery of health services at the household level. The role of the VHTs included ensuring community participation and empowerment, which is a strategy that enables communities to take responsibility for their own health and wellbeing and to participate actively in the management of their local health services. The VHTs help to reach community participation in health and link the communities to the formal health service delivery system. This helps to bridge the current health human resource gap, especially in rural or peripheral areas, where the majority of the people live. The household counseling procedure facilitated by the VHTs on a household visit was estimated to take up to 45 minutes per session.

2.7. Data Management and Analysis

Access to all data was limited to the research team and secured with a strong password. The principal investigator (GB) had overall responsibility for data management over the course of the study project and monitored compliance with the protocol.
A data entry template was created in Microsoft Excel to enter the longitudinal data regarding the participants. Each household page included was given a unique code to be able to identify participants and link baseline to follow-up data. The process of unique coding included sub-county, parish, village, and the house number. Data from Excel were exported to SPSS version 20 (IBM, New York, NY, USA). Then, correctness and consistency were checked before final data files were created for analysis. Descriptive analyses (frequencies and percentages) were used for analyzing all outcome and quality of health care variables. This was done for the four outcome variables: quality of ENC, quality of ANC services at the HF, childbirth practices, and quality of ANC care at home.
Cross-tabulation using the McNemar’s Chi-square test was used to compare the percentage of the five key outcome variables before and after implementation: first ANC in first trimester, quality of ENC, quality of ANC, exclusive breastfeeding, and four or more ANC visits. p-values < 0.05 were considered statistically significant.

2.8. Relevance of the Study

Analysis in preparation for the implementation of the sustainable development goals related to maternal and newborn health [37] revealed a slow progress in the reduction of maternal and newborn mortality by the Ugandan government. Evidence from other community-based interventions, for example a previous large-scale CHW intervention, showed that CHWs were effective in identifying pregnant women in their homes early in pregnancy and before they had attended ANC. The ttC intervention is meant to fulfill some of the conditions that are necessary for CHW to improve timely ANC uptake and newborn care [38] and therefore contribute positively to healthy pregnancy outcomes and reduction of maternal mortality in Uganda.
Furthermore, ttC is in line with renewed interest in the potential contribution of strengthening community health systems, including case management of childhood illnesses (e.g., pneumonia, malaria, and neonatal sepsis), delivery of preventive interventions such as immunization, promotion of healthy behavior, and mobilization of communities [39]. Therefore, the results from this study are to be used to asses if any progress in intermediate outcomes has been made in the study district. The ttC model is a World Vision initiative that has not been evaluated before. This study will provide evidence for scale up of ttC and adaptation by the Ministry of Health, as a model of choice for community-based maternal and newborn care at the policy level. The evidence gathered can be used to encourage more expectant mothers to continue using the established government structures, while advocating for scale up to the rest of the country. If implemented accordingly, widespread dissemination to other parts of the country can improve pregnancy outcomes nationwide.

3. Results

3.1. Social Demographic Characteristics of Participants

The study enrolled 616 households with pregnant women from 64 villages, across seven sub-counties reached by 65 VHTs. In total, 53.8% of the VHTs were female. On average, each VHT was responsible for the follow up of 10 pregnant women per village. Of the 616 pregnant women enrolled, 98.5% (n = 607) were successfully followed up to the end of the 6 weeks after childbirth. As regards the women who dropped out, 0.8% lost the newborn, 0.3% had a miscarriage during the pregnancy period, 0.2% migrated to places outside of the project coverage, while another 0.2% were not followed up by the VHTs. Further details have been included in Supplementary Table S1.

3.2. Changes in Key Outcomes

In comparison to before the implementation of ttC (see Figure 1 below), there was a general improvement in pregnancy and newborn care practices at follow-up. There was an increase of 29.6% in timely 1st ANC visit, of 28.7% in quality essential newborn care, of 25.5% in exclusive breastfeeding, and of 17.5% in quality of ANC care. These improvements were all statistically significant (p-values < 0.001). Attendance at the fourth or further ANC visit was declined by 4.6%, which is a non-significant (p = 0.153) difference.

3.3. Quality of Care

Supplementary Table S2 presents some components of quality of care at baseline and follow-up. Contributing factors to the increased quality of ANC care were an increase in iron folic supplementation of 67.0% compared to baseline and a 29.5% increase in attendance of the first ANC in the first trimester. Attendance at four or more ANC visits and intermittent presumptive treatment (IPT) during ANC showed no substantial change. In addition, a decrease in HIV testing during pregnancy by 15.0% was shown. As regards quality of care at home, there was a decrease in the percentage of women sleeping in a mosquito net by 24.7% from the baseline.
Contributing factors to appropriate childbirth practices improved by supporting the pregnant women to discuss the birth plan by 62.8%, and there was an increase of deliveries at the health facility by 10%. However, there was a decrease in the possession of a clean birthing kit of 40%. There was a steep increase in mothers practicing exclusive breastfeeding by 24.6% but also a drop in mosquito net utilization for the newborn by 11.9%.

3.4. Timely ANC Visits Recorded by the VHT

After implementation of ttC, 97.2% of all pregnant women attended the first ANC, with 52.2% having this service in the first trimester. Less women went for a second ANC visit (85.9%). The majority of these second visits (59.8%) occurred in the second trimester. The third ANC visit was attended by 69.3% of all pregnant women, with 40.6% having this visit in the second trimester. A 4th or further ANC visit to the HF was done by only 52.3% of all pregnant women, with 48.6% happening in the third trimester. The details of the period of each ANC visit are detailed in Figure 2 below.

3.5. ANC Appropriate Services and Home Practices

The provision of iron–folic supplementation to pregnant women at all ANC visits was practiced most often, with an average of 80% throughout the pregnancy visits. This showed an increasing trend from the first ANC visit through to the last before childbirth. The provision of IPT was given to 44% of pregnant women on average. As seen in Supplementary Table S3 below, the majority of pregnant women took IPT during the 2nd ANC visit, and it was given to very few mothers during the 4th or further ANC visits.
HIV counseling and testing were provided to 43.3% of the pregnant women, with the majority receiving the service early on in their pregnancy. On average, 15.6% of the pregnant women were sleeping under an LLITN, with the highest reported prevalence during the second ANC visit, and the poorest reported use during the fourth or further ANC visit.
A total of 44% of the pregnant women reported having adequate rest while at home, with the majority of the women at the fourth or further ANC visit reporting resting more, compared to 18.3% at the first ANC visit. Having adequate rest showed a progressive trend with more resting when approaching the childbirth period. On average, 42.5% of pregnant women had extra meals during pregnancy, which progressively increased with pregnancy duration. Only 23.4% of the pregnant women had a hand washing facility at the latrine, with more pregnant women having the facility as childbirth approached. More details can be accessed in Supplementary Table S3.

3.6. Essential Newborn Care during the First Week after Childbirth

Of the three visits required to be made by the VHT during the first week of the baby’s life, the majority of the mother–newborn pairs were visited on the 7th day after birth, followed by 60.6% visited on day 3 while those visited on day 1 was 58.2%. Of the newborns that were visited on day 1, 73.4% had been given the required vaccines. The majority (83.7%) of the mothers were supported by the VHTs to exclusively breastfeed their newborns on the third day after childbirth. Appropriate newborn suckling was supported in the majority (70.7%) on the 7th day (Figure 3). Fewer mothers were supported with regard to the newborns’ cord care. VHTs focused on cord care progressively, with fewer mothers being supported on the 1st day and more mothers being supported on the 7th day.

3.7. Newborn Care at Six Weeks after Childbirth

Figure S1 in the supplementary shows that 84.8% of the newborns were visited by the VHTs at 6 weeks after childbirth. Of the visited newborns, 79.6% were sleeping under an insecticide-treated mosquito net, 61.4% still received exclusive breastfeeding, and 87.2% had been vaccinated for polio1 and DPT1. Furthermore, 84.4% of newborns had their child health card plotted at immunization, whereas only 43.3% of the mothers were prepared to use family planning, and 29.9% had been taught how to prepare ORS for diarrhea care. Supplementary Table S4 provides more details of these figures.

3.8. Discussion

The current study examined the improvement of maternal and newborn health care and practices before and during implementation of ttC. Evaluations included the quality of care and the implementation of essential services during the pregnancy and newborn period. As observed from the results, there were noticeable and significant improvements in pregnancy and childbirth care after the implementation of ttC by the VHTs, especially in timely first ANC attendance, quality of ANC services, quality of essential newborn care, and exclusive breastfeeding. This achievement is attributed to the role of the VHTs, where community mobilization and household follow-up was done promptly and routinely to ensure pregnant mothers visit the HF to get appropriate care. The study results are consistent with evidence from Rwanda, where community health workers were deployed to improve maternal health, undertake health promotion activities, and connect people, especially those living in rural areas, with the health facilities as necessary [40]. The percentage of births assisted by skilled birth attendants in this Rwandan study increased from 26.7% to 69.0%, which is in line with the increase in deliveries in an HF in the current study [40]. In addition, the rate of contraceptive use increased from 4% to 45.1% in the study from Rwanda [40], which is in line with the 43.3% that indicated having decided on a family planning method after birth following the implementation of ttC in the current study.
However, the present study did show that after ttC implementation, attendance at the fourth and further ANC visits did not show an increase. This is surprising, since the study did show an improvement of 29.6% in early attendance of ANC, which is attributed to the role the VHTs played in ensuring pregnant women are supported at the household level. The WHO guidelines encourage pregnant women to have eight ANC contacts with the health system during each pregnancy, including the more familiar model of clinic-based ANC visits, as well as ANC care and/or counselling sessions for pregnant women at the household and community levels [41]. Attendance at the fourth and further ANC visits is often lowest among poor women with little education and who are living alone [42]. Accessibility-related factors potentially influencing low ANC attendance include long distance from facilities providing services, mode of transport, working hours, booking appointments, length of stay at the health facility, and the quality of care given in the previous visits [43].
The current study showed that recommended home-based practices during pregnancy were adhered to more often as the pregnancy approached childbirth. This can be seen for both the trends of the pregnant woman having adequate rest as well as for the pregnant woman having an extra meal during pregnancy. An extra meal provides the nutrients for the needs of the growing baby [42]. For example, extra calcium helps make and keep bones and teeth stronger than before the woman got pregnant [44]. The generally poor link between good nutrition during pregnancy and pregnancy outcomes can be seen in Uganda, where 10% of all newborns have a low birth weight at less than 2.5 kilograms [45].
Results from the study revealed that 30% of all newborns were not visited by the VHT at the household level by the close of the first week after childbirth. This is a missed opportunity for these newborns, since three-quarters of all neonatal deaths globally occur during the first week of life, contributing to the critical gaps in the continuum of care [46]. It is important to note that even if birth occurs at a health facility, in many settings, mothers and newborns are discharged within a few hours and have no further contact with a health provider until the 6-week postpartum and immunization visit [43]. This stresses the importance of the VHT visiting during the first week after childbirth [46].
This study had several strengths and limitations that need to be considered. The study provided unique insights in an understudied population. However, the study relied on secondary data, which meant that our scope was limited to the data that were previously gathered. This included lacking baseline data for a number of outcomes. Moreover, the data were collected by the VHTs, who are semi-literate in Uganda. Therefore, this meant that our data depend on what the VHTs collected according to their ability. During data analysis, variables that were transferred from the ANC card or CHC to the ttC household register by the VHT presented substantial missing data due to the lack of the ability to properly read and write, and this could have affected the percentages presented in the study. In addition, the current data did not allow for advanced statistical analyses. Therefore, this calls for a household-focused study, collecting data directly from the mother–baby pairs who have gone through ttC. Furthermore, the lack of a control group in the study brings about uncertainty of whether the changes can be attributed solely to ttC, as other concurrent factors in the region might have played a role.

4. Conclusions

Results from the study show an overall substantial improvement in the quality of care during the pregnancy and newborn period. Therefore, this indicates that ttC delivered through VHTs can play an integral role in ensuring the continuum of care for pregnancy and newborn care linking the community interventions to the health facilities. However, there is a need for the health system to eliminate health facility barriers that are still creating obstacles to making this process a reality.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph18094410/s1, Figure S1: Newborn care at 6 weeks after childbirth, Table S1: Characteristics of the pregnant women (N = 616) and VHTs (N = 64) included in the study, Table S2: Quality of care during pregnancy and childbirth, Table S3: Services and practices during pregnancy, Table S4: Results of a Chi-Square test on the 5 outcome variables.

Author Contributions

Conceptualization, G.B., S.G. and J.G.; Methodology, G.B., S.G., AM., I.C.B., A.M., G.R., S.P.J.K. and J.G.; Validation, A.M., J.G. and A.M.; Formal Analysis, G.B., S.G. and J.G.; Investigation, G.B. and I.C.B.; Resources, S.G., S.P.J.K. and J.G.; Data Curation, A.M., I.C.B. and G.R.; Writing—Original Draft Preparation, G.B. and J.G.; Writing—Review and Editing, G.B., S.G., AM., I.C.B., A.M., G.R., S.P.J.K. and J.G.; Visualization, G.B. and J.G.; Supervision, S.P.J.K.; Project Administration, I.C.B.; Funding Acquisition, S.P.J.K. and J.G. All authors have read and agreed to the published version of the manuscript.

Funding

The APC was funded by Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, The Netherlands.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the School of Public Health, Makerere University College of Health Sciences, the Higher Degrees, Research and Ethics Committee (HDREC) protocol 730 on 2 February 2020.

Informed Consent Statement

Patient consent was waived since this study was regarded as an anonymized desk review, there was no need for the consent of the pregnant women or mothers.

Data Availability Statement

The raw data will be made available upon reasonable request from the corresponding author.

Acknowledgments

The content is solely the responsibility of the authors and does not necessarily represent the official views of the university. A special acknowledgement goes to the district health offices of Hoima for providing support during the study, most especially the Health Assistants and VHTs in the seven sub-counties. We thank the World Vision Uganda Directorate of Health, Nutrition and HIV, and the Western Region Programme officer for health Doreen Mukiiza, for facilitating the ttC implementation track. Our gratitude goes to the Project Coordinator of the Hoima maternal newborn and child health Nangosha Elisha. We would like to thank Syabughangise Kebrine and Allan Baba Magala for their good work with data entry. Thanks to all the mothers who were willing to participate in the study.

Conflicts of Interest

The authors declare that they have no competing interest.

References

  1. World Health Organization. More Women and Children Survive Today than Ever Before—UN Report; WHO: Geneva, Switzerland, 2019. [Google Scholar]
  2. Stella, K. Childbirth in Uganda: Stories from Women and Health Workers; The Health newborn network: Kampala, Uganda, 2017. [Google Scholar]
  3. Hodgins, S.; Tielsch, J.; Rankin, K.; Robinson, A.; Kearns, A.; Caglia, J. A New Look at Care in Pregnancy: Simple, Effective Interventions for Neglected Populations. PLoS ONE 2016, 11, e0160562. [Google Scholar] [CrossRef] [Green Version]
  4. Babughirana, G.; Gerards, S.; Mokori, A.; Nangosha, E.; Kremers, S.; Gubbels, J. Maternal and newborn healthcare practices: Assessment of the uptake of lifesaving services in Hoima District, Uganda. BMC Pregnancy Childbirth 2020, 20, 1–10. [Google Scholar] [CrossRef]
  5. Ganle, J.K.; Kombet, M.L.; Baatiema, L. Factors influencing the use of supervised delivery services in Garu-Tempane District, Ghana. BMC Pregnancy Childbirth 2019, 19, 141. [Google Scholar] [CrossRef]
  6. Kate, J.K.; Joseph, E.d.G.-J.; Zulfi, q.A.B.; Pius, O.; Ann, S.; Joy, E.L. The Maternal, Newborn, and Child Health Continuum of Care; WHO: Geneva, Switzerland, 2018. [Google Scholar]
  7. Singh, K.; Story, W.T.; Moran, A.C. Assessing the Continuum of Care Pathway for Maternal Health in South Asia and Sub-Saharan Africa. Matern. Child Health J. 2016, 20, 281–289. [Google Scholar] [CrossRef] [Green Version]
  8. United Nations International Children’s Emergency Fund. The State of the World’s Children: Maternal and Newborn Health; UNICEF: New York, NY, USA, 2009. [Google Scholar]
  9. Moller, A.; Patten, J.H.; Hanson, C.; Morgan, A.; Say, L.; Diaz, T.; Moran, A.C. Monitoring maternal and newborn health outcomes globally: A brief history of key events and initiatives. Trop. Med. Int. Health 2019, 24, 1342–1368. [Google Scholar] [CrossRef] [Green Version]
  10. Bowser, D.; Okunogbe, A.; Oliveras, E.; Subramanian, L.; Morrill, T. A Cost-Effectiveness Analysis of Community Health Workers in Mozambique. J. Prim. Care Community Health 2015, 6, 227–232. [Google Scholar] [CrossRef] [Green Version]
  11. Kabir, R.; Majumder, A.A.; Arafat, S.Y.; Chodwhury, R.K.; Sultana, S.; Ahmed, S.M.; Monte-Serrat, D.M.; Chowdhury, E.Z. Impact of Intimate Partner violence on ever married women and utilization of antenatal care services in Tanzania. J. Coll. Med. Sci. 2018, 14, 7–13. [Google Scholar] [CrossRef] [Green Version]
  12. Perry, S.; Fair, C.D.; Burrowes, S.; Holcombe, S.J.; Kalyesubula, R. Outsiders, insiders, and intermediaries: Village health teams’ negotiation of roles to provide high quality sexual, reproductive and HIV care in Nakaseke, Uganda. BMC Health Serv. Res. 2019, 19, 1–12. [Google Scholar] [CrossRef]
  13. Gogia, S.; Sachdev, H.P.S. Home-based neonatal care by community health workers for preventing mortality in neonates in low- and middle-income countries: A systematic review. J. Perinatol. 2016, 36, S55–S73. [Google Scholar] [CrossRef] [Green Version]
  14. Guenther, T.; Nsona, H.; Makuluni, R.; Chisema, M.; Jenda, G.; Chimbalanga, E.; Sadruddin, S. Home visits by community health workers for pregnant mothers and newborns: Coverage plateau in Malawi. J. Glob. Health 2019, 9, 010808. [Google Scholar] [CrossRef]
  15. Namazzi, G.; Waiswa, P.; Nakakeeto, M.; Nakibuuka, V.K.; Namutamba, S.; Najjemba, M.; Namusaabi, R.; Tagoola, A.; Nakate, G.; Ajeani, J.; et al. Strengthening health facilities for maternal and newborn care: Experiences from rural eastern Uganda. Glob. Health Action 2015, 8, 24271. [Google Scholar] [CrossRef] [Green Version]
  16. Besada, D.; Eagar, D.; Rensburg, R.; Shabangu, G.; Hlahane, S.; Daviaud, E. Resource requirements for community-based care in rural, deep-rural and peri-urban communities in South Africa: A comparative analysis in 2 South African provinces. PLoS ONE 2020, 15, e0218682. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Michele, G.; Polly, W. Timed and Targeted Counselling for Health and Nutrition (2nd Edition): A Comprehensive Training Course for Community Health Workers; World Vision International: Middlesex, UK, 2015. [Google Scholar]
  18. Reithinger, R.; Megazzini, K.; Durako, S.J.; Harris, D.R.; Vermund, S.H. Monitoring and evaluation of programmes to prevent mother to child transmission of HIV in Africa. BMJ 2007, 334, 1143–1146. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Health & Nutrition Technical Brief: Timed and Targeted Counselling (TTC). A Service Package of the CHW Project Model. 2020. Available online: https://reliefweb.int/report/afghanistan/health-nutrition-technical-brief-timed-and-targeted-counselling-ttc-service (accessed on 6 February 2020).
  20. Warner, M.E.; Zhang, X. Healthy Places for Children: The Critical Role of Engagement, Common Vision, and Collaboration. Int. J. Environ. Res. Public Health 2020, 17, 9277. [Google Scholar] [CrossRef]
  21. Uganda Bureau of Statistics. The National Population and Housing Census 2014—Area Specific Profile Series for Hoima District; Uganda Bureau of Statistics: Kampala, Uganda, 2017. [Google Scholar]
  22. Stephen, B.; Laura, G. Livelihood Mapping and Zoning Exercise: Uganda. A special Report by the Famine Early Warning System Network (Fewsnet); United States Agency for International Development: Washington, DC, USA, 2010. [Google Scholar]
  23. Kawungezi, P.C.; AkiiBua, D.; Aleni, C.; Chitayi, M.; Niwaha, A.; Kazibwe, A.; Sunya, E.; Mumbere, E.W.; Mutesi, C.; Tukei, C.; et al. Attendance and Utilization of Antenatal Care (ANC) Services: Multi-Center Study in Upcountry Areas of Uganda. Open J. Prev. Med. 2015, 5, 132–142. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Babughirana, G.; Gerards, S.; Mokori, A.; Musasizi, B.; Isabirye, N.; Baigereza, I.C.; Rukanda, G.; Bussaja, E.; Kremers, S.; Gubbels, J. Effects of Implementing the Timed and Targeted Counselling Model on Pregnancy Outcomes and Newborn Sur-vival in Rural Uganda: Protocol for a Quasi-Experimental Study. Methods Protoc. 2020, 3, 73. [Google Scholar] [CrossRef]
  25. Ministry of Health Uganda. The Reproductive Health, Maternal, Newborn and Child Health Sharpened Plan; Ministry of Health Uganda: Kampala, Uganda, 2013. [Google Scholar]
  26. Kaphle, H.P.; Poudel, S.; Gupta, N.; Neupane, N. Maternal Awareness on Child Health Card and Factors Associated with its Retention in Salyan Village Development Committee of Kaski, Nepal. Int. Biol. Biomed. J. 2016, 2. Available online: http://ibbj.org/article-1-75-en.pdf (accessed on 16 March 2016).
  27. Al–Rabadi, H.S.; Sharif, N.E. Effectiveness of timed and targeted counselling about feeding and caring practices for infants and young children to mothers in West Bank: A randomised controlled trial. Lancet 2017, 390, S23. [Google Scholar] [CrossRef]
  28. Ministry of Health. Village Health Teams, Strategy and Operational Guidelines; Ministry of Health: Kampla, Uganda, 2010. [Google Scholar]
  29. World Vision. World Vision’s Time and Targeted Counselling: The World’s most Widespread Family Inclusive Maternal Health Project (28 Countries). 2017. Available online: https://familyincluded.com/world-vision-ttc/ (accessed on 17 February 2010).
  30. Benova, L.; Dennis, M.L.; Lange, I.L.; Campbell, O.M.R.; Waiswa, P.; Haemmerli, M.; Fernandez, Y.; Kerber, K.; Lawn, J.E.; Santos, A.C.; et al. Two decades of antenatal and delivery care in Uganda: A cross-sectional study using Demographic and Health Surveys. BMC Health Serv. Res. 2018, 18, 1–15. [Google Scholar] [CrossRef] [Green Version]
  31. McPherson, R.A.; Khadka, N.; Moore, J.M.; Sharma, M. Are Birth-preparedness Programmes Effective? Results from a Field Trial in Siraha District, Nepal. J. Health Popul. Nutr. 2006, 24, 479–488. [Google Scholar]
  32. Han’a Salim Al, R. Effectiveness of Timed and Targeted Counselling on Maternal, Newborn and Child Health and Nutrition (MNCH/N) Outcomes in Palestine: Assessment of Intervention. 2015. Available online: https://www.mhtf.org/document/effectiveness-of-timed-and-targeted-counselling-on-maternal-newborn-and-child-health-and-nutrition-mnchn-outcomes-in-palestine-assessment-of-intervention/ (accessed on 20 May 2015).
  33. Monitoring the Situation of Children and Women, Antenatal Care; UNICEF: New York, NY, USA, 2020.
  34. Genç, M.; Ledger, W.J. Syphilis in pregnancy. Sex. Transm. Infect. 2000, 76, 73–79. [Google Scholar] [CrossRef] [PubMed]
  35. World Health Organization. WHO Recommendations on Antenatal Care for a Positive Pregnancy Experience. 20 Avenue Appia; WHO: Geneva, Switzerland, 2016. [Google Scholar]
  36. Kimario, F.F.; Festo, C.; Shabani, J.; Mrisho, M. Determinants of Home Delivery among Women Aged 15–24 Years in Tanzania. Int. J. MCH AIDS (IJMA) 2020, 9, 191–199. [Google Scholar] [CrossRef] [PubMed]
  37. United Nations Development Programme. Sustainable Development Goal 3. Ensure Healthy Lives and Promote Well-Being for All at All Ages. 2015. Available online: https://sustainabledevelopment.un.org/sdg3 (accessed on 20 January 2015).
  38. Lema, I.A.; Sando, D.; Magesa, L.; Machumi, L.; Mungure, E.; Sando, M.M.; Geldsetzer, P.; Foster, D.; Kajoka, D.; Naburi, H.; et al. Community Health Workers to Improve Antenatal Care and PMTCT Uptake in Dar es Salaam, Tanzania. JAIDS J. Acquir. Immune Defic. Syndr. 2014, 67, S195–S201. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Haines, A.; Sanders, D.; Lehmann, U.; Rowe, A.K.; E Lawn, J.; Jan, S.; Walker, D.G.; Bhutta, Z. Achieving child survival goals: Potential contribution of community health workers. Lancet 2007, 369, 2121–2131. [Google Scholar] [CrossRef]
  40. Venyuy, M.A.; Cumber, S.N.; Nkfusai, C.N.; Bede, F.; Ijang, Y.P.; Wepngong, E.; Bama, S.N.; Tsoka-Gwegweni, J.M.; Tebeu, P.M. Determinants to late antenatal clinic start among pregnant women: The case of Saint Elizabeth General Hospital, Shisong, Cameroon. Pan Afr. Med. J. 2020, 35, 112. [Google Scholar] [CrossRef] [PubMed]
  41. Cook Children’s Health Care System. Eating during Pregnancy; Cook Children’s Health Care System: Fort Worth, TX, USA, 2020. [Google Scholar]
  42. Elana Pearl Ben-Joseph. Eating during Pregnancy. 2020. Available online: https://kidshealth.org/en/parents/eating-pregnancy.html (accessed on 3 June 2020).
  43. Uganda Bureau of Statistics (UBOS); ICF. Uganda Demographic and Health Survey 2016; UBOS: Kampala, Uganda; ICF: Rockville, MD, USA, 2018. [Google Scholar]
  44. World Health Organization. Newborns: Improving Survival and Well-Being; WHO: New York, NY, USA, 2019. [Google Scholar]
  45. Charlotte, W.; Pat, D.; Lalla, T.; Pyande, M. Postnatal Care; World Health Organization: Geneva, Switzerland, 2019. [Google Scholar]
  46. Ministry of Health Uganda. A Handbook to Improve Health in Communities; Village Health Team: Kampala, Uganda, 2010. [Google Scholar]
Figure 1. Change in key outcome variables. Ijerph 18 04410 i001 = baseline values; Ijerph 18 04410 i002 = post-implementation values; ANC = Antenatal care; ENC = Essential Newborn care; EBF = Exclusive Breastfeeding.
Figure 1. Change in key outcome variables. Ijerph 18 04410 i001 = baseline values; Ijerph 18 04410 i002 = post-implementation values; ANC = Antenatal care; ENC = Essential Newborn care; EBF = Exclusive Breastfeeding.
Ijerph 18 04410 g001
Figure 2. ANC visits per Trimester. ANC = Antenatal care.
Figure 2. ANC visits per Trimester. ANC = Antenatal care.
Ijerph 18 04410 g002
Figure 3. Essential newborn care during the first week. VAC = Newborns reported being vaccinated immediately after childbirth; EBF = Support to exclusively breastfeed; Suckling = Support to appropriate positioning and attachment of the newborn on the breast during feeding; Cord care = Support to practice appropriate cord care.
Figure 3. Essential newborn care during the first week. VAC = Newborns reported being vaccinated immediately after childbirth; EBF = Support to exclusively breastfeed; Suckling = Support to appropriate positioning and attachment of the newborn on the breast during feeding; Cord care = Support to practice appropriate cord care.
Ijerph 18 04410 g003
Table 1. ttC visits by VHTs to mothers during pregnancy or after childbirth [33].
Table 1. ttC visits by VHTs to mothers during pregnancy or after childbirth [33].
VisitTimingServices Delivered by the VHTs
1Before or at 3 months of pregnancyFocuses on suspected pregnant mothers registered as ttC candidates. They are counseled on how to care for themselves during pregnancy, a number of danger signs during pregnancy are explained, and the family is encouraged to notify the VHT in the case of any of these danger signs. Families are counseled on what to do during pregnancy and when giving birth. Women are advised to start ANC before or at three months pregnancy.
24 months of pregnancyThis visit is to affirm that the pregnant mother went for the first ANC visit and to encourage them to go for the second ANC visit. Pregnant women are educated on the advantages of exclusive breastfeeding of children until 6 months and continuous breastfeeding until they reach the age of 2. In addition, advantages of giving birth in a health facility and hand washing are highlighted.
36 months of pregnancyDiscussion of the birth plan, timing, and place of the birth, exploring potential challenges faced by expectant mothers in reaching the health facility and ways of overcoming these challenges. Expectant mothers are given information on the different family planning methods and where to go for services, and they are encouraged to utilize one of the methods after delivery.
48 months of pregnancyThe mother and family are educated about the early signs of true labor, danger signs during labor, and appropriate action to take in the case of these danger signs. The mother is also encouraged to deliver at a health facility and have emergency money available. In addition, the VHTs check whether the mother has a clean birthing kit available (the “Mama Kit”).
5–7The newborn period (Days 1, 3, and 7)This visit is done three times in the first week after birth, on days 1, 3, and 7. The VHT ensures that exclusive breastfeeding and essential maternal and newborn care are understood and practiced. Mothers are educated on the need to seek health care if the neonate develops fever and/or a cough, and the need to go for routine growth monitoring and immunization.
8When the baby is 6 weeks oldMalaria prevention is discussed, as well as other illnesses, danger signs in the baby, immunization for the baby, and hygiene practices. The VHT checks whether the child’s growth card has been plotted, gives support for exclusive breastfeeding, checks if the child has had its HIV confirmatory test as part of the prevention of mother-to-child transition of the HIV program, and whether the mother has thought about family planning methods.
Note: ANC = antenatal care, ttC = timed and targeted counseling, VHT = village health team, HIV = Human Immunodeficiency Virus.
Table 2. Study variables.
Table 2. Study variables.
VariableDescriptionAssessed at Baseline (Y/N) a
Key outcome variables
Quality of ANC servicesThe pregnant woman is able to visit a health facility for ANC and receive the eight key services in the goal-oriented ANC recommendations.Y
Quality of ANC care at homeThe pregnant woman performs the key four practices at home appropriate for the pregnancy period [34]. These include: woman having adequate rest, extra meal, sleeping LLTIN, and having a hand-washing facility at the home.N
Appropriate childbirth practicesThe pregnant woman is supported to practice the four key lifesaving actions prior to or during the time of birth [35]. These include plans to deliver at a health facility, discussion of a birth plan, delivery at the health facility, and provision of a clean birthing kit.N
Timely ANC attendanceThe pregnant woman attends ANC visits in a timely manner as recommended by the WHO and the Ugandan goal-oriented ANC [36].Y
Quality of essential newborn careMothers performing the six key essential newborn care practices both at the health facility and at home within the newborn period of 28 days after birth. These include mother was visited by the VHT during the 1st week of birth, baby sleeping under LLTIN, practicing exclusive breastfeeding, suckling well, birth weight of the baby recorded in the CHC, and the baby has been vaccinated.Y
Exclusive breastfeedingMother was able to sustain exclusive breastfeeding to the baby up to 6 weeks.Y
Quality of care variables
Quality of ANC services
Woman given folic acid during ANCPregnant woman is given iron–folic acid tablets at all ANC visits to the health facility.Y
Woman given Fansidar during ANCPregnant woman taking a directly observed dose of Fansidar at the health facility during ANC.Y
Woman tested for HIV and results givenPregnant woman does a routine HIV test at the health facility during the ANC visit.Y
Woman given deworming during pregnancyPregnant woman is given a dewormer at the health facility during the ANC visit.N
Woman vaccinated for Tetanus Toxoid (TT) onceThe pregnant woman is given at least one dose of TT vaccine during the pregnancy period by the health facility.N
ANC first trimesterPregnant woman visiting the health facility for her first ANC visit within the first 12 weeks of pregnancy.Y
4th ANC or morePregnant women has attended the health facility for four or more ANC visits.Y
Woman tested for syphilisPregnant woman takes a syphilis [26] test at least once during the pregnancy period during ANC and receives the results of the test.N
Quality of ANC care at home
Woman having adequate restThe pregnant woman is deliberately taking rest from laborious work during the day, and this is due to the pregnancy condition.N
Woman having an extra mealThe pregnant woman consumes an extra meal on top of her usual meal frequency when not pregnant.N
Woman sleeping in a long-lasting insecticide treated net (LLITN)The pregnant woman slept under an insecticide-treated bed net the night before the visit made by the VHT.Y
Hand washing facility at the homeThere is a visible hand washing facility on the walkway to the latrine of the household. N
Appropriate childbirth practices
Woman plans to deliver at health facilityThe pregnant woman is planning to deliver at the nearest health facility.N
Woman discussed a birth planDuring the 3rd trimester, the pregnant women has discussed a birth plan with the VHT on one of the visits.Y
Delivery at the health facilityThe pregnant woman actually delivered at the health facility.Y
Woman has a clean birthing kitThe pregnant woman is given a clean birthing kit by the health facility.Y
Quality of Essential Newborn Care (ENC)
Mother visited during the 1st weekThe VHT performs at least one visit to the household during the 1st newborn week.N
Baby sleeping under an LLITNDuring the newborn period visits, the newborn is sleeping under an insecticide-treated bed net.Y
Practicing exclusive breastfeedingFor the 24 hours previous to the household visit, the mother has not given anything to the newborn except for breast milk apart from medication prescribed by medical personnel to the baby.Y
Baby suckling wellThe VHT is observing breastfeeding and notices that the baby is suckling well.N
Birth weight recordAt discharge from the health facility, the newborn is weighed, and the weight is recorded on the child health card.N
VaccinationThe newborn is given the vaccines appropriate for that time period.N
Infant visited at 6 weeks
Visitation at 6 weeksThe VHT checked on the 6-week vaccinations, whether the baby was taken for growth monitoring and promotion, and whether the mother knew how to prepare oral rehydration salts in the case of a diarrheal episode. The VHT also checked whether the mother had a discussion at the HF about family planning choices.N
Notes: a—Y = Yes the variable was measured at baseline, N = No, the variable was not measured at baseline. ANC = Antenatal care; HF = Health facility; VHT = Village Health Team member; IPT = Intermittent presumptive treatment; TT = Tetanus Toxoid vaccine; LLITN = Long-lasting insecticide-treated net.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Babughirana, G.; Gerards, S.; Mokori, A.; Baigereza, I.C.; Mukembo, A.; Rukanda, G.; Kremers, S.P.J.; Gubbels, J. Can the Timed and Targeted Counseling Model Improve the Quality of Maternal and Newborn Health Care? A Process Analysis in the Rural Hoima District in Uganda. Int. J. Environ. Res. Public Health 2021, 18, 4410. https://doi.org/10.3390/ijerph18094410

AMA Style

Babughirana G, Gerards S, Mokori A, Baigereza IC, Mukembo A, Rukanda G, Kremers SPJ, Gubbels J. Can the Timed and Targeted Counseling Model Improve the Quality of Maternal and Newborn Health Care? A Process Analysis in the Rural Hoima District in Uganda. International Journal of Environmental Research and Public Health. 2021; 18(9):4410. https://doi.org/10.3390/ijerph18094410

Chicago/Turabian Style

Babughirana, Geoffrey, Sanne Gerards, Alex Mokori, Isaac Charles Baigereza, Alex Mukembo, Grace Rukanda, Stef P. J. Kremers, and Jessica Gubbels. 2021. "Can the Timed and Targeted Counseling Model Improve the Quality of Maternal and Newborn Health Care? A Process Analysis in the Rural Hoima District in Uganda" International Journal of Environmental Research and Public Health 18, no. 9: 4410. https://doi.org/10.3390/ijerph18094410

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop