Performance Evaluation of Oral Health Teams in Brazil: An Item Response Theory Approach

: Access to quality healthcare is an essential component of health policy. This cross-sectional study describes the actions performed by Brazilian Oral Health Teams (OHTs) analyzed in the National Program for Improving Access and Quality of Primary Care and the relationship of contextual aspects. A total of 22,993 OHTs that participated in the third cycle of the referred program were evaluated using a structured questionnaire. Thirteen procedures (items) related to spontaneous dental care and preventive, surgical, restorative, prosthetic, and oral cancer prevention or diagnosis were assessed. Item response theory was used to estimate the performance scores of OHTs, based on 13 items. The relationship between performance scores and contextual variables in Brazilian regions was analyzed by thematic maps and Spearman correlation ( p < 0.05). The highest difﬁculty parameters were for questions related to prosthetics (b = 0.879) and actions for oral cancer (b = 2.922). The 13 items were more appropriate to discriminate the teams with lower performance and relatively ineffective in differentiating those with better performance. A direct relationship with the Human Development Index (r s = 0.249; p = 0.004) and an indirect relationship with the Gini Index (r s = − 0.482; p < 0.001) were found. Contextual aspects presented a relationship with the performance of the Brazilian OHTs. The evaluated items showed some potential to discriminate the performance of OHTs since many of the evaluated dental procedures are performed by most of the teams.


Introduction
Brazil's Unified Health System (SUS, in Portuguese) is based on the primary healthcare (PHC) principles of health promotion, prevention, and comprehensive health care [1].In oral health, the expansion of care occurred with the inclusion of Oral Health Teams (OHTs) within primary care teams.The creation of the National Oral Health Policy (PNSB, in Portuguese) has improved the dental care of the population, with offers of prostheses in PHC and secondary care in specialty centers [2].
OHTs in Brazil offer surgical-restorative, preventive, and emergency clinical procedures, as well as actions to prevent oral cancer and rehabilitation through dental prostheses [3,4].After more than eighteen years of PNSB, there are still challenges for the consolidation of integrality and universalization of actions and services offered to the population [5].In this regard, the evaluation of primary care services can help identify and guide the solution of problems related to healthcare work processes [6].
In 2011, to expand access to both primary care and the quality of services offered to the general population, the Brazilian Ministry of Health (MofH) instituted the National Program for Improving Access and Quality of Primary Health Care (PMAQ-AB, in Portuguese).Until now, the PMAQ-AB has been Brazil's largest health service evaluation program.This program ended in 2018 after three evaluation cycles, with the first cycle taking place from 2011 to 2012, with 12,403 OHTs participating, the second cycle between 2013 and 2014, with 19,946 OHTs participating, and the third cycle between 2017 and 2018, involving 25,090 OHTs.[7].Results evaluating the performance of Brazilian OHTs in the first cycle of PMAQ-AB showed that teams with high performance develop more actions to identify oral cancer and provide more dental prostheses in PHC [8].Results of the second cycle also showed that the actions focused on oral cancer screening and the offer of dental prostheses were related to higher levels of difficulty and were conducted by OHTs with higher performance scores [9].
Brazil has an extensive territory and is marked by socioeconomic inequalities, which have an impact on the distribution of health services, affecting the population's health [1].Brazil has upgraded its health service coverage over the last three decades since the creation of SUS.However, despite these advances, it is still necessary to address lingering geographic inequalities within Brazil [10].
The aim of this study was to describe the actions performed by Brazilian OHTs within PHC and the relationship of contextual variables that lead to different levels of performance of OHTs that participated in the third cycle of evaluation of PMAQ-AB.The alternative hypothesis of this study was that there is an impact of contextual factors on the performance of Brazilian OHTs.

Materials and Methods
The Research Ethics Committee of Universidade Federal de Minas Gerais approved this study (CAAE 02396512.8.0000.5149).
Secondary data of 25,090 OHTs that adhere to the 3rd cycle of the "National Program for Improving Access and Quality of Primary Care" (PMAQ-AB) in Brazil were used.This cycle occurred between 2017 and 2018.A total of 2097 (8.4%) OHTs were excluded by PMAQ-AB evaluation criteria, as they did not follow the program's recommendations, like the presence of the dentist and dental equipment in the PHC unit and an adequate oral health surveillance system.Then, 22,993 OHTs were included.
The data were collected at the external evaluation phase.In this phase, evaluators visited the PHC units to verify documents to attest to the quality established by MofH for the organization of actions and practices considered essential in accordance with guidelines, principles, norms, and protocols.
An instrument developed by MofH, in partnership with teaching and research institutions from all over the country, was applied by previously calibrated teams.This instrument included issues related to the structure of dental facilities, working processes, management, and service organization.The interviews with the OHTs and the verification of PHC's documents generated the data obtained.The documents were verified at the moment of the interview, according to the quality standards established for the organization of practices and actions.Tablets were used to record the answers, using a program developed for PMAQ-AB.The partner institutions performed the validation of the data collected and sent them to the MofH's central database.
Questions from Module VI-Oral Health Professional Interview, section VI.7-Organization of team's agenda and actions offered and section VI.11-oral cancer care, were used in this study.A total of 21 questions, mostly dichotomous (yes/no), on primary dental healthcare procedures performed by OHTs were accessed.These questions included items with a focus on emergency care (response to spontaneous demand); surgical-restorative clinical procedures (deciduous tooth extraction, permanent tooth extraction, ulotomy/ulectomy, amalgam filling, composite filling, ionomer filling); preventive procedures (ionomer sealant application, fluoride application, supragingival and subgingival scaling, root planning, and coronal polishing); actions related to the manufacture of dental prostheses (impression, installation, and prostheses cementation); and actions related to the prevention or diagnosis of oral cancer (prevention and diagnosis of oral cancer, medical advice on tobacco use, medical advice on alcohol use and other drugs, medical advice on the prevention of exposure to solar radiation, active search in the community for potentially precancerous lesions and cases, systematic examination of oral mucosa, biopsies performed to diagnose oral cancer).The three questions about the manufacture of dental prostheses were compiled into one variable.The same was done for the seven questions about the prevention or diagnosis of oral cancer.Thirteen items with self-reported spontaneous dental care; preventive, surgical, restorative, and prosthetic-related procedures; and oral cancer prevention and diagnosis were then assessed.
For each procedure (item), such as spontaneous dental care, surgical-restorative clinical procedures, preventive procedures, and actions related to the manufacture of dental prostheses, the performance of OHTs was dichotomized for those OHTs that performed no procedures and other OHTs that performed some type of the procedure.For actions related to cancer prevention and diagnosis, three categories involved the OHTs that do not perform any action; OHTs that perform some of the seven actions; and OHTs that perform all of the seven actions.
The item response theory (IRT) model for graded responses [11] was used to estimate the performance scores of OHTs, based on 13 items.The IRT includes a set of statistical models that relate an individual's probability of response to an article and its latent characteristics, which cannot be directly measured, such as satisfaction, proficiency, and attitude [12].The latent construct for this study was the performance of the OHTs, estimated through the answers obtained on the execution or not of 13 selected items of dental procedures.
Descriptive analyses of dental procedures were performed.The first value from the decomposition of the polychoric correlation matrix was calculated to confirm the feasibility of IRT application, checking its domain.Cronbach's alpha was used to evaluate the internal consistency of the scale.
The IRT model enables a more in-depth analysis of the variables used to construct the measure, including the level of difficulty and capacity for discrimination [13].What determines the OHTs' performance is not the number of dental procedures performed by them but rather the weighing of the difficulty level and the capacity to discriminate between items.The discrimination parameter consists of the item's ability to distinguish between teams with different abilities.The higher the value of the discrimination parameter, the more the item is able to detect differences between teams.The difficulty parameter is the minimum skill a team needs to have a probability of 0.5 (50%) of performing the procedure.
The difficulty level and discrimination parameter of each procedure evaluated were calculated by using IRT, and, based on the results, each OHT received a score estimated by using Empirical Bayes method.The scores assigned to each team theoretically may vary from -∞ to +∞.The OHTs were classified according to their performance based on quartile cut-off points.In addition, the test information curve was obtained to identify the scoring range in which the instrument provides information.
Stata Software (Stata Corp. 2015.Stata Statistical Software: Version 14.University Station, TX, USA: Stata Corp LP) was used to organize the data.The IRTest package for Latent Variable Modeling and Item Response Analysis [14] of R software v. 4.0.2[15] was used to analyze the data.Descriptive analysis and correlation matrix to check the IRT hypothesis and methods of latent distribution estimation that can enhance the estimation accuracy and free the normality assumption on the latent distribution [14] were used to adjust the IRT model.
Brazil has 133 Intermediate Geographic Regions, 510 Immediate Geographic Regions, and 5570 municipalities, divided into five geographic macro-regions: North, Northeast, Southeast, South, and Midwest.The Intermediate Geographic Regions is an intermediate scale between the macro-regions and the Immediate Geographic Regions.The Intermediate Geographic Regions articulate the Immediate Geographic Regions by a differentiated higher hierarchy pole, public management flows, and the existence of more complex urban functions, organizing the territory [16].The relationship between performance scores and the median of contextual variables of the Human Development Index (HDI) and the Gini Index [17], in the Intermediate Geographic Regions [16] of Brazil, was analyzed by using thematic maps and the Spearman correlation (p < 0.05), using R software for analysis.

Results
The Cronbach alpha coefficient was 0.53, and the score related to the first component explained 45.32% of the analyzed items, which maintained the IRT unidimensionality hypothesis.
Table 1 shows the discrimination parameter and difficulty level of each evaluated procedure performed by OHTs.Difficulty parameters with negative values reveal which items were frequently performed by the OHTs, while positive values reveal items that are less frequently performed by the teams.The items with higher difficulty parameters were those related to prostheses (b = 0.879) and oral cancer monitoring actions (b = 2.922).On the other hand, the items with lower difficulty parameters were those related to response to spontaneous demand (b = −6.265)and fluoride application (b = −4.009).The performance score of 22,993 OHTs resulting from the IRT model adjustment showed an asymmetric distribution, with a mean of −0.005 (SD ± 0.727) and a median of 0.15.The highest score identified for a team was +2.538, which was classified as high-performing, and the lowest score obtained was −5.384, which was classified as low-performing.Skewness and kurtosis values were −0.0785 and 13.9985, respectively (Figure 1).The information curve of the test revealed that the 13 items are more appropriate to discriminate the teams with lower performance and are relatively ineffective in differentiating those with a better performance.Figure 2 shows the characteristic curves of the items analyzed rela dental care, along with preventive and surgical procedures performed exhibit similarities for spontaneous dental care, fluoride application, io pra and subgingival root planning, coronal polishing, and surgical pro cates that OHTs with negative values present a high probability of per cedures frequently.Figure 2 shows the characteristic curves of the items analyzed related to spontaneous dental care, along with preventive and surgical procedures performed by OHTs.Curves exhibit similarities for spontaneous dental care, fluoride application, ionomer sealant, supra and subgingival root planning, coronal polishing, and surgical procedures.This indicates that OHTs with negative values present a high probability of performing these procedures frequently.
Figure 3 shows the characteristic curves of the analyzed items related to restorative and prosthesis procedures, as well as the cancer monitoring performed by OHTs.The curves show similarities for the composite, amalgam, and ionomer fillings, with a difficulty parameter of close to −2, revealing that most OHTs present a high probability of performing these procedures.On the other hand, only OHTs with a value close to the maximum score (+2) present a high probability of performing any type of prosthesis procedures analyzed in this study.For oral cancer, OHTs with a higher probability of performing all actions of cancer prevention and diagnosis are those with a value close to the maximum score (+2).
The thematic maps showed a direct relationship of the estimated IRT scores with the HDI median and an indirect relationship with the Gini Index, confirmed by Pearson's correlation coefficient (r s = 0.294, p = 0.004, and r s = −0.482,p< 0.001, respectively).The map analysis shows that OHTs with the worst scores (<−0.37) are mostly located in the intermediate regions, corresponding to the North macro-region of the country (Figure 4A). Figure 4B,C, respectively, show the distribution of HDI and Gini Index medians, according to the division into intermediate regions [16].It can be observed that intermediate regions of Brazil with a lower HDI, the majority of which corresponds to the North and Northeast macro-regions, present OHTs with lower scores.Intermediate regions with a higher Gini Index and, therefore, a greater inequality correspond mainly to North and Northeast macro-regions and present OHTs with lower scores.Figure 2 shows the characteristic curves of the items analyzed related to spontaneous dental care, along with preventive and surgical procedures performed by OHTs.Curves exhibit similarities for spontaneous dental care, fluoride application, ionomer sealant, supra and subgingival root planning, coronal polishing, and surgical procedures.This indicates that OHTs with negative values present a high probability of performing these procedures frequently.OHTs not performing the procedure; curve 2 (dotted) represents the probability of OHTs performing the procedure.
Figure 3 shows the characteristic curves of the analyzed items related to restorative and prosthesis procedures, as well as the cancer monitoring performed by OHTs.The curves show similarities for the composite, amalgam, and ionomer fillings, with a difficulty parameter of close to −2, revealing that most OHTs present a high probability of performing these procedures.On the other hand, only OHTs with a value close to the maximum score (+2) present a high probability of performing any type of prosthesis procedures analyzed in this study.For oral cancer, OHTs with a higher probability of performing all actions of cancer prevention and diagnosis are those with a value close to the maximum score (+2).The thematic maps showed a direct relationship of the estimated IRT scores with the HDI median and an indirect relationship with the Gini Index, confirmed by Pearson's correlation coefficient (rs = 0.294, p = 0.004, and rs = −0.482,p< 0.001, respectively).The map analysis shows that OHTs with the worst scores (<−0.37) are mostly located in the intermediate regions, corresponding to the North macro-region of the country (Figure 4A). Figure 4B,C, respectively, show the distribution of HDI and Gini Index medians, according to the division into intermediate regions [16].It can be observed that intermediate regions of Brazil with a lower HDI, the majority of which corresponds to the North and

Discussion
Many of the dental procedures, such as spontaneous dental care, as well as preventive, restorative, and surgical procedures, presented a high probability of being performed by most of the OHTs.All procedures of cancer prevention and diagnosis, as well as those related to prostheses, presented a probability of being less commonly performed by the teams.The analyzed items were more appropriate in discriminating between the OHTs with a lower performance, which is relatively ineffective in differentiating between those with a better performance since many of the evaluated dental procedures are performed by most of the OHTs.A relationship of the estimated scores with contextual factors was observed, confirming the alternative hypothesis.
OHTs frequently conduct the basic procedures regardless of their performance score.These findings are similar to those obtained in two previous cycles of PMAQ-AB [3,4].The performance of clinical procedures is directly influenced by the infrastructure conditions, availability, and adequacy of materials and equipment.In this sense, OHTs are more likely to perform basic dental procedures because of the improvements in the structure of services and the expansion of access over the last eighteen years as a result of the PNSB [10,18,19].Positive changes in the indicators of access to oral health services, leading to an increase in users' ability to use them, were found when the first and second cycles of PMAQ-AB were compared.Among the PHC facilities that participated in both of the

Discussion
Many of the dental procedures, such as spontaneous dental care, as well as preventive, restorative, and surgical procedures, presented a high probability of being performed by most of the OHTs.All procedures of cancer prevention and diagnosis, as well as those related to prostheses, presented a probability of being less commonly performed by the teams.The analyzed items were more appropriate in discriminating between the OHTs with a lower performance, which is relatively ineffective in differentiating between those with a better performance since many of the evaluated dental procedures are performed by most of the OHTs.A relationship of the estimated scores with contextual factors was observed, confirming the alternative hypothesis.
OHTs frequently conduct the basic procedures regardless of their performance score.These findings are similar to those obtained in two previous cycles of PMAQ-AB [3,4].The performance of clinical procedures is directly influenced by the infrastructure conditions, availability, and adequacy of materials and equipment.In this sense, OHTs are more likely to perform basic dental procedures because of the improvements in the structure of services and the expansion of access over the last eighteen years as a result of the PNSB [10,18,19].Positive changes in the indicators of access to oral health services, leading to an increase in users' ability to use them, were found when the first and second cycles of PMAQ-AB were compared.Among the PHC facilities that participated in both of the PMAQ-AB, there was an increase in the proportion of units with at least one dentist that had a dental office and worked for at least two shifts per day and five days per week [19].Besides that, most Brazilian OHTs are well structured in relation to the availability of equipment, instruments, and supplies for dental practice, which may contribute to the understanding of the high frequency of performance of basic procedures by the teams [20].The comparison of the first and second cycles of PMAQ-AB showed a reduction in the number of OHTs that reported providing appointments for spontaneous demand [19].
However, the high frequency of procedures related to response to spontaneous demand, found in this study, can be explained because the Brazilian health system still presents accumulated needs in oral health, which leads to the aggravation of oral diseases with consequent painful symptomatology [21].By contrast, the frequent performance of curative and surgical procedures reveals that dental practices in PHC are still centered on the traditional healthcare model, based on direct restorations and extractions.Universal health coverage in an unequal country like Brazil is still a challenge [10], and some access attributes remain unsatisfactory, with organizational barriers persisting [19].Although these are important procedures, it is important to offer comprehensive dental care with specialized procedures [22].
The procedures related to prostheses were less likely to be performed by OHTs.The last national epidemiological survey [23] revealed a high prevalence of edentulism in the adult and elderly Brazilian population.The distribution of regional prosthetic laboratories in the country does not correspond to the epidemiological needs of the population, and the increase in the production of prostheses has been discrete [24].The results of the present study show that there are OHTs that create prosthesis impressions but do not install them and that do prosthesis impressions but do not cement them, which could result in an unfinished service.This is worrisome since, in Brazil, much of the need for prosthetic rehabilitation falls on the public health system [25].The OHTs that have human resources who are joined by public examinations, participate in permanent education, with a work process organized, and receive more support from municipal management are more likely to perform dental prostheses [26].A greater evaluation of dental prosthesis procedures should be carried out in Brazilian PHC services.Probably, improved work incentives and better organizational support may be needed for the majority of OHTs to offer dental prostheses to their patients [26].
The inclusion of OHTs in the PHC had a good impact on the campaign actions of oral cancer, follow-up, referral to specialized care, and the registration of suspected cases in Brazil [27].However, it is worrisome that OHTs are less likely to perform all actions for the prevention or diagnosis of the disease when compared to other procedures performed in PHC.The majority of OHTs perform some of the seven analyzed actions.This can lead to many cases without diagnosis or with a late diagnosis of the disease.This is very serious, since this type of cancer corresponds to about 2% of all cancer cases diagnosed worldwide [28], and its early diagnosis is important and decisive in determining an efficient treatment that can lead to a cure or increased survival of patients [29].It is important to ensure that PHC dentists are trained for early identification and referral for specialist care of patients with potentially malignant lesions, which can increase treatment success [9].Moreover, it is important to understand how network care is offered [27] to structure and/or reinforce financing and development policies for the teams, providing equal conditions throughout the country to expand the performance of procedures, especially those related to prosthetics and oral cancer monitoring.
The lower HDI and higher Gini Index median result in worse OHT scores, showing that these values are spatially distributed in a related way in the intermediate geographic regions, confirming the existence of socioeconomic differences in Brazil.The OHTs with the worst scores were located in the intermediate regions of Brazil, corresponding mainly to the North and Northeast regions.The expansion of SUS over the last three decades has allowed Brazil to improve the healthcare needs of its population, with the upgrading of healthcare service coverage.However, despite its successes, it is necessary to address lingering geographic inequalities [10].The North and Northeast regions present the greatest social and economic challenges among the Brazilian regions [1].These differences may result in better assistance of dental care services in the South and Southeast regions, which present better indices when compared to the rest of the country [1,30].The comparison between different Brazilian geographic regions related to the performance of primary dental care revealed that OHTs in the South and Southeast regions are better than the others, especially when compared to those in the North region, reinforcing the great social disparities [3,4].
Surgeries 2023, 4 576 Health inequalities in Brazil can be explained, in part, by differences in the access and infrastructure of services and in the determinants of health among these regions, which reflect the quality of care provided.
It is necessary to structure and improve public policies aimed at promoting comprehensive care in order to expand and qualify the work process of oral health services in PHC [31].However, it is important for expansion to promote a better regional distribution of OHTs in order to reduce inequities.Moreover, the insertion of missing procedures in PHC, which responds to the epidemiological needs of the population [32], can result in the improvement of the population's oral health [3].
This study used a secondary database of the third evaluation cycle of the PMAQ-AB.The participating OHTs were selected by the municipal manager, and these procedures, in turn, may have chosen those who have the best organization and structures, It can be considered a limitation of the study because weaker teams may not have participated in the study.Moreover, the PMAQ-AB is a pay-for-performance program that may motivate OHTs to provide positive answers to the questionnaire items [3].However, this study used the databases obtained by the largest Brazilian health services evaluation program ever instituted.The results found in the care can direct an improvement in financing policies and the development of OHTs to improve the provision of comprehensive healthcare services to the population.

Conclusions
IRT analyses revealed that items were more appropriate in discriminating between the lower-performance teams and were relatively ineffective in differentiating between those with better performance since many of the evaluated dental procedures are performed by most OHTs.Contextual aspects presented relationships with the performance of the Brazilian OHTs, reinforcing the great social disparities in the country.

Figure 2 .
Figure 2. Item curve characteristics-probability of OHTs performing actions related to preventive care and surgical procedures in Brazil (2017-2018).Curve 1 (black) represents the probability of

Figure 2 .
Figure 2. Item curve characteristics-probability of OHTs performing actions related to preventive care and surgical procedures in Brazil (2017-2018).Curve 1 (black) represents the probability of OHTs not performing the procedure; curve 2 (dotted) represents the probability of OHTs performing the procedure.

Figure 3 .
Figure 3. Item curve characteristics-probability of OHTs of performing actions related to restorative and prothesis procedures, along with oral cancer actions in Brazil (2017-2018).Curve 1 (black) represents the probability of OHTs not performing the procedure; curve 2 (dotted) represents the probability of OHTs performing the procedure.For oral cancer, curve 1 (black) represents the probability of OHTs not performing any action related to cancer prevention and diagnosis; curve 2 (dotted) represents the probability of OHTs performing any action related to cancer prevention and diagnosis; and curve 3 (light dotted) represents the probability of OHTs of performing all actions of cancer prevention and diagnosis.

Figure 3 .
Figure 3. Item curve characteristics-probability of OHTs of performing actions related to restorative and prothesis procedures, along with oral cancer actions in Brazil (2017-2018).Curve 1 (black) represents the probability of OHTs not performing the procedure; curve 2 (dotted) represents the probability of OHTs performing the procedure.For oral cancer, curve 1 (black) represents the probability of OHTs not performing any action related to cancer prevention and diagnosis; curve 2 (dotted) represents the probability of OHTs performing any action related to cancer prevention and diagnosis; and curve 3 (light dotted) represents the probability of OHTs of performing all actions of cancer prevention and diagnosis.

Figure 4 .
Figure 4. (A) Regional distribution of the scores of the OHTs evaluated in Brazil, (B) regional distribution of HDI, and (C) Gini Index for municipalities with evaluated oral health teams in Brazil (2015-2018).

Figure 4 .
Figure 4. (A) Regional distribution of the scores of the OHTs evaluated in Brazil, (B) regional distribution of HDI, and (C) Gini Index for municipalities with evaluated oral health teams in Brazil (2015-2018).