Validation of the American English Acute Cystitis Symptom Score

The diagnosis of acute uncomplicated cystitis (UC) is usually based on clinical symptoms. The study aims to develop and validate the American-English Acute Cystitis Symptom Score (ACSS), a self-reporting questionnaire for diagnosis and patient-reported outcome in women with acute uncomplicated cystitis (UC). After certified translation into American-English and cognitive assessment, the clinical validation of the ACSS was performed embedded in a US phase-II trial. 167 female patients with typical symptoms of UC were included in the study following US Food and Drug Administration (FDA) guidance. At Day 1 (diagnosis), the mean (SD) sum score of the six ACSS typical symptoms reached 10.60 (2.51). Of 100 patients followed-up last time on Day 5 or 6 (End-of-treatment, EoT), 91 patients showed clinical success according to the favored ACSS criteria (sum score of typical symptoms 0.98 (1.94)). There was no correlation between the severity of symptoms on Day 1 or between clinical success rate at EoT and level of bacteriuria on Day 1. The American-English ACSS showed high predictive ability and responsiveness and excellent levels of reliability and validity. It can now be recommended as the new master version in clinical and epidemiological studies, in clinical practice, or for self-diagnosis of women with symptoms of UC.


Introduction
Acute cystitis is the most frequent bacterial infection in women [1]. The diagnosis of acute uncomplicated cystitis (UC) can be made with high probability based on a focused history of lower urinary tract symptoms and the absence of vaginal discharge or irritation [2]. Various urinary symptoms have been used to assess the diagnosis and severity of UC in women [3][4][5][6][7], but only a few studies developed a questionnaire to also evaluate the severity and impact on activity impairment [5,6], which, however, were not designed for diagnostics of UC, but only for follow up.
The Acute Cystitis Symptom Score (ACSS) is a simple and self-reporting questionnaire for female patients with UC, allowing not only assessment of the presence but also the severity of typical and differential symptoms, quality of life, as well as considering additional health conditions and possible changes after therapy [8][9][10]. The ACSS has proven to be a valuable instrument for clinical studies and medical practice for initial diagnosis, as well as a patient-reported outcome (PRO) measure allowing the monitoring of the efficacy of therapy in women suffering from UC [8][9][10]. According to the US Food and Drug Administration (FDA) guidance [11] and European Medical Agency (EMA) draft guidelines [12], the clinical response is important not only for the primary composite efficacy endpoint, but also at each fixed time point assessment as a secondary endpoint, meaning that the ACSS could

Study Population of Clinical Validation
A total of 167 female patients aged 30 ) median (range) years with typical symptoms of AC at 11 sites were included in the clinical study, which followed the FDA guidance [11]. Demographic characteristics of the US and the international cohorts are provided in Table 1 with a mean (SD) age of 36.8 (15.3) and 34.6 (15.1) years, respectively. Cohorts were homogenous concerning age or additional conditions except for pregnancy because pregnant women were not included in the US study according to FDA guidance [11].

The ACSS in US Cohort at Day 1 (Diagnostics)
Of the 167 patients, 162 (97.0%) achieved at Day 1 a sum score of the "Typical" domain (ACSS) of 6 and higher, which shows an excellent agreement between the clinical diagnosis made by the treating physician according to FDA guidance [11] and the patient's symptoms scoring using the ACSS questionnaire [8,9].
Detailed questionnaire data for the US cohort on Day 1, including the severity of symptoms and impact on the quality of life (QoL) are provided in Table 2. About 80-90% of patients complained of the moderate-to-severe intensity of the following symptoms: urinary frequency, urinary urgency, dysuria, and sense of incomplete bladder emptying.

Comparison of the ACSS between the US and International Cohort at Day 1 (Diagnostics)
The comparison between the US (167) and the international cohort (237), in which the ACSS was used in the following languages [9]: Uzbek (313), Russian (87), Tajik (58), German (43), and Hungarian (16), did not show any statistically significant difference in the average number and sum scores of the "Typical" and "Quality of Life" domains, as well as in the sum score of the entire ACSS. The sum score of the "Differential" domain (such items as flank pain, vaginal and urethral discharge, and elevated body temperature) was lower for the US cohort compared to the international cohort and the difference was statistically significant (p = 0.003) (Table 3, Figure S1 and Figure S2). There was no significant association between the amount and/or severity of symptoms on Day 1 and the level of bacteriuria on Day 1 (Figure 2). Table 3. ACSS sum scores of typical symptoms, differential symptoms, and quality of life derived from the US and international group on Day 1.

US Group
International Group    Figure 3 represents the summary score of the typical domain of the ACSS on Day 1-6 of the US cohort. The results demonstrated that after high sum scores at Day 1 before treatment (mean 10.6), for the following days during treatment, the sum scores were reduced quite distinctly and reached almost a plateau on Days 5 and 6. Therefore we investigated more carefully the 100 patients who filled in the ACSS questionnaire on Day 1 and the last time either on Days 5 or 6.  Figure 3 represents the summary score of the typical domain of the ACSS on Day 1-6 of the US cohort. The results demonstrated that after high sum scores at Day 1 before treatment (mean 10.6), for the following days during treatment, the sum scores were reduced quite distinctly and reached almost a plateau on Days 5 and 6. Therefore we investigated more carefully the 100 patients who filled in the ACSS questionnaire on Day 1 and the last time either on Days 5 or 6. Table 4 demonstrates the sum scores of typical symptoms (total and concerning the amount of bacteriuria at Day 1), differential symptoms, and quality of life of the patients in the US cohort at Day 1 (Baseline) and Days 5/6 (EoT). The reduction from Day 1 to end-of-treatment at Days 5/6 was highly significant for all ACSS domains.

Patient-Reported Outcome Before, During and After Therapy
Correlation test for the sum scores of the "Typical" domain stratified according to the amount of bacteriuria at Day 1, has proven our hypothesis about the absence of significant relationships between the severity of UC or clinical outcome at EoT and the amount of bacteriuria at the time of the start of the therapy (Table 4). Table 5 shows in more detail the severity of the individual typical and differential symptoms, their impact on each of the quality of life categories, and the overall patient's assessment of symptomatic changes (ACSS "Dynamics") at EoT claimed by the same 100 patients of the US study cohort at Day 1 and 5/6 (EoT)  Table 4 demonstrates the sum scores of typical symptoms (total and concerning the amount of bacteriuria at Day 1), differential symptoms, and quality of life of the patients in the US cohort at Day 1 (Baseline) and Days 5/6 (EoT). The reduction from Day 1 to end-of-treatment at Days 5/6 was highly significant for all ACSS domains.  Table 4. ACSS sum scores of (i) typical symptoms in relation to amount of bacteriuria, (ii) differential symptoms, and (iii) quality of life derived from the same patients of the US group at Day 1 (Diagnostics) and Day 5/6 (End of Therapy). p-paired t-test.

ACSS
<0.001 Table 5. ACSS data derived from patients of the US group at Day 1 (diagnostics before the start of treatment) and Day 5/6 (end of treatment   Table 6 shows the number of cases when using certain breakpoints to determine success and non-success at Days 5/6 in the patients of the US (n = 100) and international study cohort at Day 5-9 (EoT) (n = 82) [10]. Of the five predefined thresholds, the threshold A (Sum score of typical domain ≤5 scores, no item >1 and "visible blood in urine" = 0) and D (Sum score of 4 FDA symptoms ≤4, no item >1 and "visible blood in urine" = 0) are favored and showed the same results with a success rate of 91% in the US and 80% in the international cohort. The higher success rate in the US cohort can probably be explained because all patients were treated with a suitable antibiotic, whereas the non-interventional treatment of the international cohort might have varied considerably. Table 6. The number of cases using certain ACSS thresholds for clinical success at Day 5/6 in the patients of the US (n = 100) and international group at Day 5-9 (end of treatment) (n = 82) [10]. Note: each case with any "visible blood in the urine (VBU)" at end of treatment was rated "non-success".

Validation of the American English ACSS Using Pre-and Posttreatment Results
The diagnostic values of the different items and the sum score of the "Typical" domain of the ACSS were tested, comparing results obtained by the ACSS before and after treatment. Figure 4 illustrates that not only the pure presence of typical symptoms, but rather severity (scores) increases the diagnostic accuracy as has been shown earlier [7]. Receiver operating characteristic (ROC) curves for each independent item and the sum score of the "Typical" domain demonstrated the best-balanced results for sensitivity (96%, 95% CI: 90-99%) and specificity (98%, 95% CI: 93-100%) for the sum score of 6 typical symptoms as compared to the individual symptoms. More detailed calculations, such as positive and negative predictive values, positive and negative likelihood ratio, and correlation with a positive outcome (diagnosis of UC) comparing different clinical diagnostic definitions are shown in Table 7. Again a sum score of 6 or more of the "Typical" domain (ACSS) showed the most favorable results as compared, e.g., for the FDA [11] and EMA [12] inclusion criteria for the clinical diagnosis of UC. At a sum score of the "Typical" domain (ACSS) of 6 the clinical diagnosis of UC can be made best balanced with a sensitivity of 94% and 87% and a specificity of 90% and 88% in two studies including 286 females (139 with UC and 147 without UC) and 517 females (285 with UC and 232 without UC), respectively [8,9]. Therefore, a threshold of a sum score of 6 and higher of the "Typical domain (ACSS) was used for the clinical diagnosis of UC. Table 7. Sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratio, and correlation with a positive outcome (PO for diagnosis of cystitis) using different criteria (mean and 95% CI). CI, confidence interval; PPV, positive predictive value, NPV, negative predictive value; +LR, positive likelihood ratio; −LR, negative likelihood ratio; AUC, area under the curve; PO, positive outcome (diagnosis is correct). Main symptoms and 3 EMA symptoms: urinary frequency, urinary urgency, dysuria; 4 FDA symptoms: urinary frequency, urinary urgency, dysuria, suprapubic pain.

Discussion
The ACSS, validated in several other languages (www.acss.world), was now also successfully translated and linguistically and clinically validated in American English.
Psychometric parameters and diagnostic values of the American English ACSS showed good-to-excellent results, which are comparable to the Russian as source and other versions as previously translated and validated in different languages [8][9][10][14][15][16][17][18].
Comparing the results obtained by the ACSS for diagnostics of acute UC and EoT showed again that not only the presence but also the severity (scoring) of the symptoms are important, which may be typical but not specific for acute UC because some of the symptoms can also be caused by other urological diseases. It also can be shown that for clinical diagnostics, the best balance between sensitivity and specificity can be obtained using a sum score of 6 or higher of all the six ACSS typical symptoms as has been shown earlier as well [8,9,15].
Using five different reasonable thresholds to differentiate between success and non-success resulted in the same outcome for the US study, but each threshold showed slightly different outcomes in the international group. The study shows again that the ACSS could also be used as a PRO measure instrument in prospective, randomized clinical studies comparing different treatment modalities.
By using the ACSS in this US clinical trial, it could be shown for the first time that the severity of symptoms and the clinical outcome do not correlate with the level of bacteriuria before treatment. There was no difference for both parameters (severity of symptoms and clinical outcome), whether the urine culture showed 10 5 CFU/mL (as requested by FDA and EMA for microbiological outcome assessment), 10 4 CFU/mL or <10 4 CFU/mL.
Since it has been shown in several studies that much lower amounts of bacteriuria are also clinically significant, clinical diagnostics and patient-reported outcome using validated questionnaires should become a priority in women with acute UC, especially when symptomatic treatment modalities are also involved [19][20][21]. Nevertheless, microbiological investigations should not be neglected, but much lower amounts of bacteriuria need to be considered by using appropriate laboratory methods.

Study Design
The clinical study was initiated, supported, and designed by Mission Pharmacal Company, San Antonio, TX 78239, USA, as a randomized, double-blind, placebo-controlled, multicenter Phase II trial of the efficacy and safety of MPC-SHRC for the relief of symptoms associated with uncomplicated UTI (ClinicalTrials.gov Identifier: NCT03129295; https://clinicaltrials.gov/ct2/history/NCT03129295? V_1=View) with the ACSS accepted as a study tool. The primary study protocol was approved by the Western Institutional Review Board, 270717. Further details on the study protocol are to be published elsewhere.

The ACSS as a Study Tool
The ACSS is a 2-part, self-reporting questionnaire (Figure 1), including the following questions:

Linguistic Validation of the American English Version of the ACSS
The linguistic validation process included two stages: (1) The certified translation into American English from original Russian [14,22,23] by Mapi Language Services, Lyon, France (Ref.No.  according to international guidelines with forward and backward translations and consideration of comments from subjects interviewed with American English as a first language [24][25][26], and (2) cognitive assessment by 10 US physicians, and by 49 US female subjects with and without a history of UC. The whole process was steered by a scientific committee (SC).

Clinical Validation
The clinical validation was performed as an embedded study of the above-mentioned Phase II trial following the study protocol, Good Clinical Practice, and Code of Federal Regulations Title 1, Parts 50, 56, and 312. The study consisted of two on-site visits: Visit 1 (Day 1) (baseline) and visit 2 (discharge). After the baseline examination, all subjects were randomized to receive either the investigational drug or placebo QID for 3 days. All randomized subjects received concurrent antibiotic treatment as prescribed by the investigator. Before any study-related activities, written informed consent was signed and personally dated by the subject.
The ACSS was first completed on Day 1 (diagnosis) before therapy and was then completed daily until 48 h after the end of treatment (EoT). In this side study, the blinding concerning the investigational drug or placebo was maintained. The scores of the ACSS questionnaire in the US study cohort were compared with those obtained in an international cohort of 237 female patients with acute UC from other studies [10], who have used the ACSS in their native languages (Uzbek, Russian, Tajik, German, and Hungarian), and who also fulfilled the FDA guidance [11] concerning requirements for clinical diagnosis of acute UC with at least two of the following four typical symptoms: urinary frequency, urinary urgency, dysuria, suprapubic pain, and in addition evidence of pyuria in urinalysis.
The amount of urine culture (colony forming units/mL) was tested concerning the presence and severity of the ACSS typical symptoms and the clinical outcome at EoT.

Data Acquisition and Processing
The data of the American English ACSS were retrieved from the US cohort. The ACSS data for the international cohort were retrieved from the e-USQOLAT database as described earlier [9,10]. Since US cohort consisted of only the patients, the data at Day 1 (Baseline) of the US cohort were used to define the data of positive outcome/"Patients", and the last data obtained at Days 5 or 6 (EoT) were used to define negative outcome/"Controls" to assess psychometric reliability and diagnostic value of the American English ACSS. The presence of symptoms (positive, negative), symptoms' severity (mild, moderate, severe), and the proposed diagnostic approaches (FDA, EMA, ACSS) were considered. Data processing included a procedure of dichotomization of variables for the assessment of diagnostic values. Relative variables were labelled as "0" for "negative / not match", and "1"-for "positive / match", as described in detail earlier [9,10].

Statistical Analysis
Variables were visually tested for the normality using Q-Q plots as well as numerically, using Shapiro-Wilk's test. Descriptive statistics were presented using mean, standard deviations (SD), 95% confidence intervals (CI), median and interquartile range (IQR). Psychometric reliability was measured by the values of internal consistency of the items in the domain and the entire questionnaire presented by Cronbach's alpha and strength of association between items and summary score of domains as well as the total score of the ACSS, presented by Pearson's r-coefficient of product-moment correlation. Diagnostic values of the domains and items of the ACSS and diagnostic modalities were assessed by calculation of sensitivity, specificity, positive and negative likelihood ratios, diagnostic odds ratio (DOR), and ROC-curve analysis.
Comparative analysis was performed via paired t-test (for numerical data), with the Welch correction in cases of inequality of variances, Wilcoxon signed ranks test (for ordinal and interval data), and McNemar's chi-square test (for categorical variables). Pearson's correlation coefficient was used to assess the strength of associations between variables and the outcome, and the interval nature of the outcome variable (e.g., grouped number of CFU of pathogens in urine culture). Statistical significance was set at 0.05.
R v.3.5.2 with in-built and additional packages was used for the statistical analysis and graphical representation of the results [27][28][29][30].

Conclusions
The American English ACSS showed high values of predictive ability and responsiveness, and excellent levels of reliability and validity for diagnostics of acute UC and as a PRO measure. Therefore the ACSS can now be recommended as a new master version for clinical or epidemiological studies, but also in clinical practice and for self-diagnosis for women with symptoms of acute UC with American English as their first language.
Supplementary Materials: The following are available online at http://www.mdpi.com/2079-6382/9/12/929/s1, Figure S1. Boxplots (IQR, range, mean ± SD) of the numbers of the ACSS typical symptoms in the US and in the international group. Figure S2. Boxplots (IQR, range, mean ± SD) of the summary scores of the ACSS typical symptoms in the US and in the international group. Figure S3: Boxplots (IQR, range, mean ± SD) of the severity scores of the six ACSS typical symptoms in the US group versus the three categories of bacteriuria at Day 1.
Author Contributions: All authors had participated in the development of the study plan and design, together with the representatives of Mission Pharmacal Company and Hurley Consulting Associates Ltd. (see Acknowledgements). J.F.A. cleaned and processed the study data and performed the statistical analysis. K.G.N. wrote and revised the first draft of the manuscript. A.P. and F.M.W. updated, complemented, and edited the manuscript, which was reviewed, discussed, finalized, and approved by all authors. All authors have read and agreed to the published version of the manuscript.