Letter to the Editor

To the Editor:

We read with great interest the report by William P. Scherer, DPM, MS, and Michael D. Scherer, BS, published in the November/December 2004 issue of the Journal (“A Comparison of Results from Two Mycology Laboratories for the Diagnosis of Onychomycosis: A Study of 85 Cases in a Geriatric Population”), describing a study in which results from two mycology laboratories were compared to determine their potential impact on the choice of therapy for onychomycosis. The study included 85 patients, aged 65 years and older, with clinical signs and symptoms of onychomycosis. Samples were taken from the hallux toenail and sent to two different mycology laboratories for fluorescent potassium hydroxide (KOH) preparation with microscopic examination and fungal culture. The authors stated that of the 85 cases studied, the two mycology laboratories reported similar KOH preparation results for only 58.8% of the patients and similar fungal culture results for genus and species identification for 37.6% of the patients. When the KOH preparation and fungal culture results were combined, the two mycology laboratories reported similar results for only 27.1% of the patients. Owing to the discrepancy between the findings of the two independent mycology laboratories, the authors concluded that their results leave physicians to question the reproducibility of fluorescent KOH preparation and fungal culture analysis for the diagnosis of onychomycosis in such patient populations and that this, in turn, may affect their choice of appropriate therapy.

The authors raise a very interesting point and are the first to address an important question: How consistent are mycologic results from different laboratories on which both prescribing physicians and managed-care organizations base their therapeutic decisions? The answer, according to Scherer and Scherer’s results, would be that they are not particularly consistent.

Background

We have asked a similar question but in a slightly different context. In large clinical trials, the central mycology laboratory results affect study enrollment, but, more important, they affect the integrity of data, particularly since mycologic cure is a key measure of therapeutic success. Despite heavy reliance on results obtained from mycology laboratories in such trials, there is really no way of knowing whether the results are in fact reproducible, other than relying on the laboratory’s own quality control.

During the course of a large, multicenter, open-label clinical trial, IRON-CLAD (Improving Results in ONychomycosis: Concomitant LAmisil and Debridement),[1] we became aware that the high incidence of negative KOH results and negative fungal culture results was hindering investigators’ ability to enroll patients in the study. Both a positive KOH result and a positive culture result were required for randomization into the IRON-CLAD trial. We wondered if the use of another laboratory would have given us a higher percentage of positive KOH and/or culture results, thus enhancing patient enrollment. Thus a sub-study was carried out to compare the results from the original mycology laboratory (Laboratory A = Fungus Testing Laboratory at the University of Texas Health Science Center at San Antonio) with results from another mycology reference laboratory (Laboratory B = The Center for Medical Mycology at the University Hospitals of Cleveland).

Methods

Four investigators in IRON-CLAD with demonstrated success in randomizing patients were selected for participation in this sub-study. We selected investigators with varying years of clinical trial experience who were podiatric physicians with busy academic or private practices outside the trial. They were instructed to continue with patient screening and nail sample acquisition as outlined in the protocol, but to begin splitting the nail samples into two equal aliquots to be sent, under identical transport conditions, to Laboratories A and B for direct examination and fungal culture. The two mycology laboratories were required to match KOH results (positive or negative) and fungal type (dermatophyte isolated = positive culture; no dermatophyte isolated = negative culture) for an individual patient sample.

Potassium hydroxide preparation methodologies were similar at the two laboratories, except that Laboratory A used chlorazole black E stain, while Laboratory B used Calcofluor white for direct examination of fungal hyphae. For cultures, Laboratory A used Sabouraud’s dextrose agar and Mycosel (Sabouraud’s dextrose agar + cyclohexamide) agar (Becton, Dickinson, Sparks, Maryland) to inhibit growth of nondermatophytic fungi. Laboratory B cultured patient specimens on Mycosel agar and potato dextrose agar + chloramphenicol and gentamicin to inhibit bacterial growth. In order to avoid bias in favor of the new laboratory (Laboratory B), investigators were informed that regardless of the outcome of the sub-study, only results from Laboratory A (the original laboratory specified in the protocol and used throughout the clinical trial) would be considered valid for patient enrollment. Besides their contractual obligations as clinical investigators in the IRON-CLAD trial, investigators received no additional monetary compensation or incentives for their participation in this sub-study. The new laboratory (Laboratory B) was compensated according to customary fees for nail KOH and culture testing.

Statistical Analysis

Agreements between the two laboratories were examined by reporting the kappa (κ) coefficient and its 95% confidence interval (CI) for KOH preparation results and for culture results.[2]-[5] Kappa is a chance-corrected measure of agreement between two observers (laboratories). Kappa equals 1 when there is perfect agreement and 0 when the agreement equals that expected by chance. Guidelines for describing the relative strength of agreement associated with kappa are available.[3]

Results

Forty-five male and female outpatients (56% men, 44% women), 19 to 77 years of age, were included in the sub-study. The mean age was 52 years, and the median age was 51 years. The two mycology laboratories were in “substantial” agreement for KOH results, as shown in Table 1 (κ = 0.69; 95% CI, 0.46–0.92), and “moderate” agreement for culture results for dermatophytes, as shown in Table 2 (κ = 0.41; 95% CI, 0.14–0.68).

With regard to enrollment in the clinical trial, both positive KOH and culture results were required for patient enrollment. When the KOH preparation and fungal culture results were combined, the two mycology laboratories were in “moderate” agreement, as shown in Table 3 (κ = 0.41; 95% CI, 0.19–0.63). The agreements’ qualities were based on guidelines in Kianifard.[3]

The choice of central mycology laboratory did not appear to affect enrollment, because the number of patients with positive KOH results and negative culture results in Laboratory A matched that of Laboratory B (12 patients); that is, both laboratories reported a similar rate of “screen failure,” albeit for some different patients. One patient who had negative KOH results and positive culture results from Laboratory B was excluded from Table 3 because there was no matching value at Laboratory A. We felt that the fungal elements need to be present, even if not detected microscopically, to result in a positive culture. In our opinion, it is rather unlikely for a patient to have a negative KOH result (no fungus) on microscopy but a positive culture result (live fungus), unless the presence of fungus on KOH preparation was initially missed.

Discussion

The agreements between results of the two laboratories, as measured by kappa, were greater in our sub-study than has been reported. Scherer and Scherer did not report kappa, but given the small percentage of similarities observed in their work, it is likely that there was low agreement, as they have observed.

Although laboratories may not differ drastically in their basic methodologies, the handling of specimens, quality-control procedures, and the level of technicians’ experience can all influence the results. Also, in our trial we instructed podiatric physicians to take samples from subungual debris at the most proximal end after debriding the distal edge of the mycotic nail and to include the distal nail clippings as well. In our experience, investigators’ sampling techniques also vary substantially. Among the four investigators in our sub-study, positive KOH results ranged from 55% to 85%, while positive culture results ranged from 25% to 60%. Taken together, this suggests that nail sampling by investigators plays a key role in determining the outcome of laboratory results. Another variable of interest is the quantity of sample that each laboratory received for a given patient. Neither laboratory was asked to weigh incoming samples, so it is possible that some investigators did not split the nail sample exactly equally before shipping. However, one would have expected this to favor neither laboratory.

Conclusion

This study supports a respectable reproducibility of fluorescent KOH preparation and fungal culture analysis between two large laboratories in an outpatient population participating in an onychomycosis trial. Furthermore, the data strongly suggest that it is unlikely that our choice of mycology laboratory negatively affected our investigators’ ability to enroll patients based on KOH or culture screening results.

Brian Arnold, MS

Farid Kianifard, PhD

Amir Tavakkol, PhD, Dip Bact

Novartis Pharmaceuticals Corp One Health Plaza East Hanover, NJ 07936

Table 1. KOH Results 

Table 1. KOH Results 

Table 2. Culture Results 

Table 2. Culture Results 

Table 3. Combined KOH and Culture Results 

Table 3. Combined KOH and Culture Results 

Authors’ Response

To the Editor:

We are pleased that authors besides ourselves are exploring the reproducibility and reliability of mycology results from different laboratories for the diagnosis of onychomycosis. We believe that this research topic has been long overlooked by the medical and podiatric professions and that eliminating the confusion of different mycology results should be a concern for the entire medical profession. Although we reached different conclusions regarding the reproducibility of results from two mycology laboratories for fluorescent KOH preparation and fungal culture analysis, several factors may explain our differences.

Our investigation, performed by a single podiatric physician in private practice, focused on a geriatric population of 85 patients with a mean age of 82.8 years, a median age of 81 years, and an age range of 65 to 99 years. We defined four categories of reported results to compare the two mycology laboratory results: fluorescent KOH preparation, fungal type, genus, and genus and species. We used a literal interpretation of the mycology reports as supplied by the two laboratories, whereby a positive culture was indicated by the presence of any type of growth, not just growth of a dermatophyte.

The study by Arnold et al involved four different investigators with a total of 45 patients with a mean age of 52 years, a median age of 51 years, and an age range of 19 to 77 years. They evaluated two categories of reported results to compare the results of the two mycology laboratories: KOH preparation and fungal type. Their study assumed that if a dermatophyte was isolated, it was considered to be a positive culture, and that if no dermatophyte was isolated, it was a negative culture.

Because the age demographics of the two studies were completely different and the laboratory evaluation criteria were not the same, it is reasonable to assume that two different conclusions could be reached. The patient population in the study by Arnold et al was significantly younger than in our study, which focused on the geriatric “Medicare-aged” population and aimed to compare the four reported categories as supplied by the two mycology laboratories. The laboratories we used reported 42.7% saprophytes, 24.2% dermatophytes, 14.7% yeast, and 18.5% no growth, whereas their study considered saprophytes and yeasts as negative cultures.

Although the study by Arnold et al used the kappa coefficient to examine agreements between the two laboratories, we decided not to use this method of statistical analysis because there is wide disagreement about the usefulness of kappa statistics to assess rater agreement. We were concerned about using a statistical measure that is a source of so much controversy and felt that kappa coefficient statistics would not be viewed as the unequivocal standard or default way to quantify agreement.

In our study, we found several examples of specimens with negative KOH and positive culture results. Laboratory 1 had 61 negative fluorescent KOH preparations. Evaluation of the fungal cultures for these 61 negative KOH preparations showed that 31 were positive for saprophytes, 10 were positive for yeast, 1 was positive for a dermatophyte, and 19 had no growth reported. Laboratory 2 had 36 negative fluorescent KOH preparations. Evaluation of the fungal cultures for these 36 negative KOH preparations showed that 11 were positive for saprophytes, 2 were positive for yeast, 5 were positive for dermatophytes, and 18 had no growth reported.

We respectfully disagree with the statement of Arnold et al that their study supports the reproducibility of fluorescent KOH preparation and fungal culture analysis between two large laboratories. According to their Table 3, “Combined KOH and Culture Results,” the two laboratories had a “perfect match” (+,+ or +,− or −,−) for only 27 out of 45 patients studied. To put this table into our perspective, the two laboratories had some type of discrepancy for 40% of the patients, even though they reported a “moderate” kappa agreement.

In our study, we had 85 patient samples collected by only one double-board-certified podiatrist, and we were unable to determine whether there was inadequate quality control at the mycology laboratories or whether one laboratory was correct and the other incorrect. When we compared the results for fluorescent KOH preparation and fungal culture analysis from the perspective of a clinical podiatric physician, we demonstrated that two independent mycology laboratories were unable to achieve adequate agreement in a majority of their results for a geriatric patient population with suspected onychomycosis in South Florida. It should be noted that we included the complete database printout for all of our data collected in our original submission to the Journal, but the table was not published owing to editorial policy. The entire database can be viewed online at http://www.onychomycosis.com.

We feel that further research is needed, preferably using a national patient population of 1,000 subjects with clinical signs and symptoms of onychomycosis, to definitively determine the reproducibility and reliability of mycology results from different laboratories. It should also be noted that the cost of obtaining fluorescent KOH preparation and fungal culture analysis from two different laboratories for a large-scale investigation is beyond the reach of private practitioners and that producing a study of this magnitude may require pharmaceutical support or federal grant funding.

William P. Scherer, DPM, MS

Michael D. Scherer, BS

PO Box 272207 Boca Raton, FL 33427