Safety Climate Evaluation in Primary Health Care: A Cross-Sectional Study

The perception of the existence of deficits in patient safety, the associated costs and the limitation of resources have made it essential to define improvement strategies. Important concepts have emerged, such as safety climate, which evaluates the perceptions of safety status held by professionals in relation to their organization. The aim of this study is to characterize the safety climate in primary health care (PHC) using the Safety Attitudes Questionnaire (SAQ)—Short Form 2006 PT and to assess associations between SAQ-Short Form 2006 PT scores and demographic and professional characteristics. A cross-sectional study was conducted in all public PHC units in the northern region of Portugal. Data were collected through an online questionnaire shared via institutional emails and by means of a snowball approach. Descriptive and inferential statistical analysis were performed. Statistical significance set at p ≤ 0.05. A total of 649/7427 (8.7%) responses were included in the analyses. The mean and median total SAQ-Short Form 2006 PT scores were 69.23 (SD = 15.73, range 22.22–100.00) and 71.53 [59.03; 79.86], respectively. This is the first study to assess the safety climate in PHC in Portugal. The median obtained total SAQ-Short Form 2006 PT score was 71.53 [59.03; 79.86], which is below the threshold of ≥75, indicating safety deficits.


Introduction
The first works highlighting the importance of patient safety appeared in the 20th century with the publication of the results of the Harvard Medical Practice Study (1991) [1,2], To Err is Human: Building a Safer Health System (2000) [3] and An organization with a memory (2002) [4], among others. In 2002, the topic was addressed at the 55th World Health Assembly; in 2004, the World Alliance for Patient Safety was created [5]; and in 2005, the First Global Challenge for Patient Safety was created [6].
The perception of the existence of deficits in patient safety, the associated costs and the limitation of resources have made it essential to define improvement strategies in order to promote safety in association with healthcare delivery [7]. Important concepts have emerged, such as safety culture, which is structural indicator that promotes and facilitates initiatives to minimize risk and prevent adverse events.
According to the Global patient safety action plan 2021-2030: towards eliminating avoidable harm in health care, "The safety culture of an organization is the product of 2 of 20 individual and group values, attitudes, perceptions, competencies, and patterns of behavior that determine the characteristics of the organization's health and safety management" [8].
Safety climate is a more readily measurable component of safety culture. It evaluates the perceptions of safety status held by professionals in relation to their organization [9]. Questionnaires are practical tools that can be used to assess the safety climate, allowing organizations to act proactively and plan improvement strategies, evaluating the impact and effectiveness of implemented actions [7].
Research in primary health care (PHC) on the nature, causes or consequences of incidents is performed to a lesser extent than in hospital settings [10]. A meta-analysis published in 2019 revealed pooled prevalence of overall patient harm of 12%, but only two of the studies included in the review where based in PHC setting [11]. This may be due to an assumption that PHC is safer than hospital care and is in the early stages of development in some parts of the world and because records may not be easily accessible, making research in this area difficult [7].
A systematic review of PHC safety incidents and their association with harm identified a median of 2-3 incidents per 100 visits/medical records reviewed, 4% of which were associated with severe harm [10]. In PHC, the estimated proportion of patient safety incidents is generally lower than the estimated 10% of people who experience events in hospitals. However it is important to consider that in many parts of the world, the overall volume of people using PHC is substantially higher than those using hospital services. Therefore, the burden of potential harm related to patient safety incidents in PHC cannot be underestimated [10].
A recent study analyzed 1129 patient safety incidents reports in PHC and found that 25.0% did not reach the patient, 66.5% reached the patient without harm and 8.5% caused adverse events. By analyzing adverse events, it was possible to verify that 46.9% required observation, 34.4% caused temporary damage and required treatment, 13.5% required hospitalization and 5.2% caused severe permanent damage and/or a situation close to death. In this study, the main identified critical areas were communication (27.8%), management (25.1%), care delivery (23.5%) and medication (18.4%); few incidents were related to diagnosis (3.6%). Concerning specifically adverse events, the main identified critical areas were care delivery (33.3%), communication (20.8%), management (17.7%) and medication (15.6%); diagnosis was the least represented area (11.5%). The main contributing factors were related to the professional (55.8%), the organization (36.8%) and external factors (25.9%). For adverse events, the contributing factors were also related to the professional (47.9%), the organization (40.6%) and external factors (33.3%) [12].
Safety and quality in health care are linked; identification of critical points that require intervention, once flagged, allows for the development and implementation of improvement strategies [13].
The aim of this study was to characterize the safety climate in public PHC in the northern region of Portugal using the SAQ-Short Form 2006 PT and to assess associations between SAQ-Short Form 2006 PT scores (total and by domain) and demographic and professional characteristics.

Study Design
A cross-sectional study was conducted in all public PHC units in the northern region of Portugal.

Setting and Subjects
In mainland Portugal, public PHC is structured according to five regional health administrations (Alentejo, Algarve, Center, Lisboa e Vale do Tejo and North), subdivided into health center clusters (ACeS), including, among others, PHC units with varying types of organization and remuneration (family health units A (USF-A), family health units B (USF-B) and customized healthcare units (UCSP)) [14]. 3 of 20 The research population in the present study included all physicians, doctors in precareer training, nurses and technical assistants working in PHC units in the North Regional Health Administration in Portugal in 2022. According to Bi-CSP ® data (bicsp.min-saude.pt, accessed on 31 January 2022), this population includes a total of 7427 professionals (Appendix A): 2318 physicians (31.2%), 929 doctors in pre-career training (12.5%), 2386 nurses (32.1%) and 1794 technical assistants (24.2%).
Professionals who refused to participate or who did not met the eligibility criteria (firsttime participants in the study who worked as physicians, doctors in pre-career training, nurses and technical assistants in PHC units in the North Regional Health Administration in Portugal at their current workplace for a month or more) were excluded.

Data Collection
Data were collected from 7 June to 9 August 2022 through an online structured Google Forms ® questionnaire shared between 7 June and 5 August 2022 via institutional emails and by means of a snowball approach, i.e., a non-probabilistic sampling whereby participants help spread the word, recruiting additional respondents. The questionnaire was voluntary, self-completed and totally anonymous. It included an introductory message, informed consent to use data for the research purposes, questions about basic demographic and professional characteristics and the Safety Attitudes Questionnaire-Short Form 2006 validated for Portugal (SAQ-Short Form 2006 PT, Appendix B) [7]. A pilot test was conducted prior to the release of the questionnaire to participants. Completion of all items was defined as mandatory for response submission.

Variables and Instruments
The SAQ, developed by Sexton et al., is a widely used tools that has been adapted for use in intensive care units, operating rooms, and general inpatient and ambulatory clinics [7,9] (Appendix C). It has been extensively tested, is considered psychometrically valid and allows for the assessment of the perceptions and attitudes of health professionals related to safety [7]. The original version was developed in 2006 by Bryan Sexton, Eric Thomas and Bob Helmreich (University of Texas) based on the Intensive Care Unit Management Attitudes Questionnaire, which was derived from the Flight Management Attitudes Questionnaire, which is widely used in commercial aviation [9].
The SAQ-Short Form 2006 was translated, culturally adapted and validated for Portugal by Saraiva et al. [7] from the original SAQ version developed by Sexton et al. [9]. Permission to use this instrument in the present study was obtained from the copyright holder of the instrument by the one of the authors. This questionnaire assesses organizational, work environment-related and team-related factors [7].
The SAQ-Short Form 2006 PT consists of two parts [7]: The first part of the SAQ-Short Form 2006 PT consists of 41 items (Total SAQ Cronbach's alpha (α): 0.92) and six domains: team environment (perception of the quality of collaboration among professionals; items 1-6; α = 0.70), safety climate (perception of a solid and proactive organizational commitment to safety; items 7-13; α = 0.73), job satisfaction (pleasant or emotionally positive feeling resulting from the perception of the work experience; items 15-19; α = 0.86), stress recognition (recognition of how performance is influenced by stressful factors; items 20-23; α = 0.82), management perception (management approval regarding safety issues; unit and hospital; items 24-29; α = 0.88) and working conditions (perception of quality environment and logistics related to the workplace; items 30-32; α = 0.71). Items 14 and 33-36 are not part of the aforementioned domains, and items 2, 11 and 36 are scored inversely. In order to improve completion of the questionnaire by PHC professionals, three references were added to the SAQ-Short Form 2006 PT (Google Forms version), defining equivalence between terms: hospital = ACeS; unit = UCSP/USF; and clinical area = UCSP/USF. The answers to each item were provided on a Likert scale consisting of five levels: 1 = strongly disagree; 2 = partially disagree; 3 = neutral; 4 = partially agree; 5 = strongly agree; X = not applicable [7,9]. A calculation formula suggested by the authors of the original study [9], was applied, which involved averaging the responses (after inversion of reverse items 2, 11 and 36) and to calculation of a 100-point scale score (1 was subtracted from the mean, and the result was multiplied by 25). The final score ranges from 0-100 (0 = worst perception of the safety climate; 100 = best perception of the safety climate). "Not applicable" responses were considered missing [7,9]. A score of ≥75 indicates a strong safety environment, reflecting positive attitudes toward patient safety [13].
The second part of the questionnaire includes information on demographic and professional characteristics (gender, function and length of service) [7]. Age, workplace and type of primary care unit were added.

Data Analysis
Categorical variables were described by absolute and relative frequencies (n (%)). Normally distributed continuous variables were summarized using means and standard deviation (M (SD)). Non-normally distributed continuous variables were described using median and interquartile interval (Med [Q 1 ; Q 3 ]), where Q 1 is the first quartile, and Q 3 the third quartile. In ordinal variables, although it is recommended to present Med [Q 1 ; Q 3 ], we decided to also present M (SD) to facilitate interpretation and possible comparisons with the results of other studies. The normality of distributions was verified by visual observation of histograms and using the Kolmogorov-Smirnov test.
The internal consistency of the SAQ-Short Form 2006 PT scale and its six domains was assessed using Cronbach's alpha (α), with values above 0.70 considered acceptable. To study the association between two continuous variables, Pearson's coefficient (rP) was used when the two variables were normally distributed or Spearman's coefficient (rS) when at least one of the variables was not normally distributed or was ordinal. To compare means between 2 independent, normally distributed samples, a t-test for independent samples was used. To compare distributions between 2 independent, not normally distributed samples, the Mann-Whitney test was used. To compare means between 3 or more independent, normally distributed samples, one-way ANOVA test was used. To compare distributions between 3 or more independent, not normally distributed samples, the Kruskal-Wallis test was used. In both the one-way ANOVA test and the Kruskal-Wallis test, when significant differences were found, multiple comparisons were performed with Bonferroni adjustments.
The Statistical Package for the Social Sciences ® (SPSS, version 27.0 for Windows ® SPSS Inc., Chicago, IL, USA, 1989-2020) to analyze the results, and statistical significance was set at p ≤ 0.05.

Ethical Considerations
This study was authorized by the management boards of the institutions involved and approved by four ethics committees of the northern region of Portugal related to PHC (ethics committees of the North Regional Health Administration, the Alto Minho Local Health Unit, the Matosinhos Local Health Unit and the Northeast Local Health Unit). To avoid loss of anonymity, the names of PHC units were not collected. All participants signed an informed consent form. All recommended ethical principles were respected.

Descriptive Analysis of the Sample
A total of 676/7427 (9.1%) responses were collected. Among these, 2 (2/676, 0.3%) were excluded because the respondents did not consent to participate, and 25 (25/676, 3.7%) were excluded because the respondents did not meet the eligibility criteria (first time participants in the study who worked as physicians, doctors in pre-career training, nurses and technical assistants in PHC units in the North Regional Health Administration in Portugal at their current workplace for a month or more). The sample was stratified by function and workplace.
A total of 649/7427 (8.7%) responses were included in the analyses. The mean age of respondents was 42.6 years (SD = 9.9, range 18-67 years), and 77% (500/649) were women. The following distribution was found in terms of demographic and professional characteristics: (

Descriptive Analysis of SAQ-Short Form 2006 PT
A descriptive analysis of the SAQ-Short Form 2006 PT was performed. All 41 items of the SAQ-Short Form 2006 PT were analyzed. The median response rate for "not applicable" considering the totality of the items of the SAQ-Short Form 2006 PT was 3.2%. These responses were excluded from the analysis.
Four items obtained mean scores of less than 3 (Table 3) Figure 1 shows the distribution of responses for each of the 41 items of the SAQ-Short Form 2006 PT according to the Likert scale. These are constructed using absolute and relative frequencies (n (%)) without inversion of items 2, 11 and 36 and after exclusion of "not applicable" responses. The total percentage of "not applicable" responses was 7.0%.
Item 35 received the highest percentage of "indifferent" responses. Item 34 received the highest percentage of positive responses, i.e., "Agree" or "Strongly agree". Items 2 and 11 obtained the highest percentage of negative responses, i.e., "Disagree" or "Strongly disagree". Item 24b obtained the third-highest percentage of negative responses, i.e., "Disagree" or "Strongly disagree". Some of the items included in the management perception domain (items 24-29) were among those with highest percentage of negative answers, i.e., "Disagree" or "Strongly disagree".

Inferential Analysis
Each of the domains and the total SAQ-Short Form 2006 PT scores (Table 4) are described below. •

Team environment
The team environment domain is only associated with the characteristic function (p < 0.001) and type of primary care unit (p < 0.001). Regarding the variable function, multiple comparisons with Bonferroni adjustments show that differences were found between the categories of nurses and physicians (p < 0.001) and between nurses and doctors in pre-career training (p = 0.004), with the nurse category showing lower values in this domain, as well as between the categories of technical assistants and physicians (p = 0.007) and between technical assistants and doctors in pre-career training (p = 0.015), with the technical assistant category also showing

Inferential Analysis
Each of the domains and the total SAQ-Short Form 2006 PT scores ( and between UCSP and USF-B (p < 0.001), with the UCSP category presenting lower total scores.
Multiple comparisons with Bonferroni adjustments were performed. The differences found for each of the domains considering the workplace variable are shown in Figure 2 and presented in the following sections.

Principal Findings
This was the first study to evaluate the safety climate in PHC in Portugal. A score ≥75 indicates a strong safety environment [13], a cutoff value not accomplished in the

Principal Findings
This was the first study to evaluate the safety climate in PHC in Portugal. A score ≥75 indicates a strong safety environment [13], a cutoff value not accomplished in the present study, as the mean and median SAQ-Short Form 2006 PT total scores were 69.23 (SD = 15.73, range: 22.22-100.00) and 71.53 [59.03; 79.86], respectively.
The total SAQ-Short Form 2006 PT median scores were higher among female respondents, in workplace "13. ACeS Grande Porto I-Santo Tirso/Trofa" and in type of primary care unit USF-A and USF-B. In contrast, the lowest median values among male respondents, in "4. ACeS Alto Trás-os-Montes-Nordeste" and in UCSP. The workplace "13. ACeS Grande Porto I-Santo Tirso/Trofa" was considered, in recent, one of the best in terms of performance in North Regional Health Administration (Bi-CSP ® : bicsp.min-saude.pt, access in 14 September 2022), which might be related to safety perception. "4. ACeS Alto Trás-os-Montes-Nordeste" obtained a lower score in terms of performance in the North Regional Health Administration, although it was not the workplace with the lowest rate (Bi-CSP ® : bicsp.min-saude.pt, accessed on 14 September 2022). These results suggest an association between the safety climate and the level of performance, although such an association remains to be confirmed. USF-A and USF-B have organizational, functional and technical autonomy, which might explain the achievement of higher total SAQ-Short Form 2006 PT scores in comparison to the UCSP.
The scores obtained in the domains of team environment, safety climate, job satisfaction and stress recognition indicate a strong safety environment. In contrast, the scores obtained in the domains of management perception and working conditions (below the threshold of 75) indicate the need for improvement.
The differences obtained in SAQ-Short Form 2006 PT domain scores identify areas that require further intervention, which are discussed in the following sections. Stress recognition In the stress recognition domain, as the age of the respondent increases, the obtained SAQ-Short Form 2006 PT median score decreases, and as the length of service at the respondent's current workplace increases, so does the obtained score. The lowest scores in n this domain were associated with the categories of nurse and technical assistant. Healthcare professionals should be able to recognize excessive workload and fatigue in order to improve the safety climate and promote effectiveness. Keeping this concept in mind is especially important as the age of professionals increases, with shorter length of service at the current workplace and in the categories of nurse and technical assistant. Self-care might also be relevant to this domain and should be promoted. • Management perception The score obtained in the management perception domain was below the threshold of 75, indicating the need for improvement. The lowest median scores in this domain were obtained among male respndents and in the UCSP. The UCSP has less autonomy than the USF-A and USF-B, which might influence management perception. The implementation of measures that promote the transition from the UCSP to the USF might improve this result. The median score in the management perception domain was also calculated separately for ACeS and UCSP/USF, with lower values associated with ACeS management. In the USF, management is not hierarchical and develops in a cooperative atmosphere witch might have contributed to the better result obtained in the UCSP/USF. Healthcare professionals perceive that managers of ACeS and UCSP/USF need to improve in terms of compromise of patient safety, management performance, support of professional efforts, constructive personnel problem solving and ensuring appropriate human resources. The lack of appropriate human resources is particularly evident, with a negative score for this item (29). The need for improvement with respect to support of professional efforts and constructive personnel problem solving is particulary evident for ACeS management, with negative mean scores for these items (24b,e and 27b). This domain is in need of attention from managers and intervention. The category of UCSP obtained low median scores in several domains: team environment, safety climate, job satisfaction, management perception and working conditions. According to Bi-CSP ® data (bicsp.min-saude.pt, accessed on 31 January 2022), the workplace "4. ACeS Alto Trás-os-Montes-Nordeste" is the only one in the North Regional Health Administration totally constituted by UCSP (Appendix D) and was identified in the present study as having the lowest total SAQ-Short Form 2006 PT score. This does not reflect all variables inherent to each workplace, nor the quality of work developed.
Item 7 obtained the highest mean score, reflecting a positive perception of professionals with respect to patient care; item 24b obtained the lowest mean score, reflecting a perceived need for increased support from ACeS leadership.
Four items were obtained mean scores of less than 3: "36" is an inverse, but still with low scores, indicating a need for improved communication; "29", reflecting the perception of an insufficient ratio of professionals with respect to the number of patients; and "27b" and "24b", reflecting a perceived need for improvement in management of problems related to professionals and more support from ACeS leadership.
Item 35 received the highest percentage of "indifferent" responses. These results might be explained by difficulties in interpreting the question, as in most situations, pharmacists are not integrated in PHCs but work in community pharmacies. Item 34 received highest percentage of positive responses, i.e., "Agree" or "Strongly agree", indicating a positive perception of collaboration with staff physicians. Items 2 and 11 receieved the highest percentage of negative responses, i.e., "Disagree" or "Strongly disagree", a result to be inversed but still with potential for improvement. Item 24b received the third-highest percentage of negative responses, i.e., "Disagree" or "Strongly disagree", highlighting the perception of a need for more support from ACeS management, as previously mentioned.
Some of the items included in the management perception domain (items 24-29) are among those with a high percentage of negative answers, i.e., "Disagree" or "Strongly disagree", calling for leadership investment in communication, problem solving and professional support.

Comparison with Prior Work
Several papers have been published to date assessing safety climate. However, comparative analysis is limited because most such studies were conducted in hospitals or with different study populations, with some using a different assessment instrument other than the SAQ-Short Form 2006. Another limitation has to do with the differences between the validated SAQ-Short Form 2006 questionnaires used in different countries, as the items included in each domain are not always the same.
In 2006, Sexton et al. conducted a study in ambulatory clinics in the USA and obtained lower mean scores than those reported in the present study: team environment, 69.7 (vs. 77.41% in the present study); safety climate, 69.9 (vs. 71.91 in the present study); job satis-faction, 70.6 (vs. 75.61 in the present study); stress recognition, 66.7 (vs. 83.42 in the present study); and working conditions, 61.6 (vs. 66.55 in the present study); with the exception of perceptions of management, 55.3 (vs. 54.66 in the present study) [9].
In 2010, McGuire et al. conducted a study in a large practice with a primary care core in the USA staffed by clinicians in family medicine, pediatrics and internal medicine and obtained higher mean scores than those reported in the present study: team environment, 88.9 (vs. 77.41 in the present study); safety climate, 87. 8 (vs. 71.91 in the present study); job satisfaction, 86.2 (vs. 75.61 in the present study); stress recognition, 74. 8 (vs. 83.42 in the present study); perceptions of executive management, 72.6; perceptions of local management, 86.0 (vs. management perception 54.66 in the present study); and working conditions, 84.9 (vs. 66.55 in the present study) [15].
In 2019, Lousada et al. conducted a study in six primary care centers in Brazil and obtained lower mean scores than those reported in the present study: total SAQ-Short Form 2006 score, 62.6 (vs. 69.23 in the present study); team environment, 69.4 (vs. 77.41 in the present study); safety climate, 59.5 (vs. 71.91 in the present study); job satisfaction, 75.1 (vs. 75.61 in the present study); stress recognition, 65.8 (vs. 83.42 in the present study); management perception, 54.5 (vs. 54.66 in the present study); and working conditions, 51.2 (vs. 66.55 in the present study) [16].
Differences between studies are expected, as safety climate evaluates the perceptions of safety status held by professionals in relation to their organization [9].

Strengths and Limitations
This is the first study to evaluate the safety climate in PHC in Portugal. We intended for the study to be comprehensive, involving all public PHC units of the North Regional Health Administration, and inclusive, with the participation of physicians, doctors in precareer training, nurses and technical assistants. The obtained sample was representative of the studied population. The SAQ questionnaire is a widely used instrument with good psychometric qualities, and the consistency of the data gathered in this study supports its validity. By identifying multiple domains in safety climate, the SAQ highlights critical points and possible areas of intervention.
With respect to limitations, this study is a self-reported survey depending on the respondents' recall, meaning that results might be affected by reporting bias. The opportunity to participate in the study by filling in a form exclusively available online may represent a barrier to participation. The results may reflect, at least partially, effects of the COVID-19 pandemic. The cross-sectional nature of the study means the results do not measure any evolution that may have occurred. In this study, we also analyzed only a part of safety culture, focusing on professionals' perceptions.
Studies in this area are still scarce, indicating the need for attention from researchers.

Conclusions
This is the first study to assess the safety climate in PHC in Portugal. The median total SAQ-Short Form 2006 PT score was 71.53 [59.03; 79.86], which is below the threshold of ≥75, indicating safety deficits. The lowest median scores were obtained in the domains of management perception and working conditions. Further studies are needed to determine which factors might influence the safety climate in a positive or in a negative manner, as well as associations with performance and the occurrence of incidents.