#
Image Features Based on Characteristic Curves and Local Binary Patterns for Automated HER2 Scoring^{ †}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. HER2 Assessment

#### 2.2. Dataset

#### 2.3. Processing Stages

## 3. Characteristic Curves

_{low}) denotes the percentage of staining with colour in the range given by the following inequalities:

_{1}≤ h < h

_{2}

s > s

_{low}

v

_{1}≤ v < v

_{2},

_{low}) plotted against s

_{low}gives the characteristic curve (or the percentage-saturation curve) of the image. In Equation (1), [h

_{1}, h

_{2}] denote fixed hue thresholds specifying allowable variations in the hue value, and similarly [v

_{1}, v

_{2}] denote value thresholds. Since we specify only the lower bound for saturation, progressively increasing s

_{low}, typically from 0.1 to 0.5, produces a non-increasing characteristic curve (Figure 3). This property of the characteristic curve is the direct result of p(s

_{low}) being proportional to the complement of a normalized cumulative histogram for saturation values.

_{low}is increased from 0.1 to 0.5. The resulting characteristic curve is also shown. The characteristics curves have the property that they are always monotonically decreasing smooth curves. They allow accurate polynomial approximations using cubic curves. The shape of the curve can be directly matched with the staining patterns given in the HER2 assessment guidelines (Table 1) for a straightforward interpretation of the derived score (Figure 4). For example, the characteristic curve always lies below the 10% threshold when the score is 0, and only a small initial segment of the curve lies above the 10% mark when the score is 1. If the score is 3+, the curve lies completely above the 30% mark, showing a strong and complete membrane staining. As seen in Figure 4, the curve passes through a much wider range of values of percentage staining when the score is 2+.

- If z
_{0}(=p(0.1)) <10%, then the whole curve lies below 10%, and the score is 0 - Else if z
_{n}_{−1}(=p(0.5)) >30%, then the whole curve lies above 30%, and the score is 3+ - Else if 10% ≤ z
_{0}(=p(0.1)) <40% and p(0.2) <15%, the score is 1+ - Else if p(0.4) <15%, then the score is 2+
- Else, the score is 3+

## 4. Local Binary Patterns

#### 4.1. LBP Computation

_{1}, h

_{2}] and saturation values with s > s

_{low}. The pixels passing the threshold test are converted to gray level by mapping h

_{1}to 0 and h

_{2}to 255. This gray-level image is used as the input for LBP computation. The LBP histogram of such images contain predominant features that represent the texture characteristics of the staining patterns. We denote the 256 values of the LBP histogram by L

_{i}, i = 0, …, 255.

#### 4.2. Rotation-Invariant Uniform LBP

_{i}, i = 0, …, 8. As an example,

_{4}= L

_{15}+ L

_{30}+ L

_{60}+ L

_{120}+ L

_{240}+ L

_{225}+ L

_{195}+ L

_{135}.

_{5}+ L

_{9}+ L

_{10}+ + L

_{11}+ L

_{13}…

#### 4.3. uLBP Feature Curves

_{i}in the rotation-invariant uLBP set can generate a feature curve as detailed below. When the input image’s saturation threshold s

_{low}is varied from 0.1 to 0.5 as discussed in Section 3, we get the corresponding variation in the LBP values L

_{i}. The LBP values are then combined into nine uLBP values U

_{i}as discussed in the previous section. Image regions outside the saturation threshold are assigned a pixel value 0. These “background” pixels of constant intensity will have an LBP value 255, and contribute to the uLBP bin U

_{8}. We discard the value of U

_{8}, as it mainly represents regions of constant intensity. The variation in the values of the remaining bins U

_{i}, i = 0, …, 7 shows a non-increasing trend very similar to that of the characteristic curve (Figure 7).

_{i}

^{′}= U

_{i}·100/(w·h)

_{i}, i = 0, …, 8 and also the non-uniform component Ū for images with HER2 scores 0, 1+, 2+, and 3+ are shown in Figure 7.

_{i}, i = 0, …, 7 show excellent discriminating power between the four HER2 classes, making them highly suitable for use as feature vectors in HER2 classification algorithms.

## 5. HER2 Classification and Scoring

_{i}

^{(j)}= p(s

_{i}), i = 1, …, n, j = 1, …, m are used as features. The class labels are denoted by y

_{j}∈ [0, 3], j = 1, …, m. We denote the feature matrix by X ∈ ℜ

^{m}

^{×(n+1)}, the output vector of labels by Y ∈ ℜ

^{m}

^{×1}, and the classifier parameter vector for each class by θ

_{k}∈ ℜ

^{(n+1)×1}, k = 1, …, 4. Here, class-1 corresponds to the set of training examples with HER2 score 1+, class-2 with HER2 score 2+, class-3 with HER2 score 3+, and class-4 with HER2 score 0. We then have the following equations for the hypothesis functions H, the cost function, and the gradient functions:

_{k})

^{m}

^{×1}, and g() denotes the sigmoid function. The cost function J(θ

_{k}) is then given by

_{k}) is defined as

_{i}on the characteristic curve or the LBP feature curve of a given sample are combined with the trained values of class parameters θ

_{k}for each class k = 1, …, 4, and the class that gives the maximum value for g(x

_{i}′θ

_{k}) is chosen. In the next section, we provide the result of classification experiments using the above methods.

## 6. Experimental Results and Analysis

_{low}∈ [0.1, 0.5]. The feature matrix X in Equation (5) therefore had the dimension 156 × 20. The gradient descent algorithm used 100 iterations to converge to the solution with a learning rate of 0.001 (Figure 8).

_{i}, i = 0, …, 7, each containing 20 sample points, were used in our analysis. We give below the classification results as a confusion matrix (Table 6).

## 7. Conclusions and Future Work

## Acknowledgments

## Conflicts of Interest

## References

- Hicks, D.G.; Schiffhauer, L. Standardized assessment of the Her2 status in breast cancer by immunohistochemistry. Lab. Med.
**2015**, 42, 459–467. [Google Scholar] [CrossRef] - Rakha, E.A.; Pinder, S.E.; Bartlett, J.M.; Ibrahim, M.; Starczynski, J.; Carder, P.J.; Provenzano, E.; Hanby, A.; Hales, S.; Lee, A.H.; et al. Updated UK recommendations for HER2 assessment in breast cancer. J. Clin. Pathol.
**2015**, 68, 93–99. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Gavrielides, M.A.; Gallas, B.D.; Lenz, P.; Badano, A.; Hewitt, S.M. Observer variability in the interpretation of HER2 immunohistochemical expression with unaided and computer aided digital microscopy. Arch. Pathol. Lab. Med.
**2011**, 135, 233–242. [Google Scholar] [CrossRef] [PubMed] - Akbar, S.; Jordan, L.B.; Purdie, C.A.; Thompson, A.M.; McKenna, S.J. Comparing computer-generated and pathologist-generated tumour segmentations for immunohistochemical scoring of breast tissue microarrays. Br. J. Cancer
**2015**, 113, 1075–1080. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Hamilton, P.W.; Bankhead, P.; Wang, Y.; Hutchinson, R.; Kieran, D.; McArt, D.G.; James, J.; Salto-Tellez, M. Digital pathology and image analysis in tissue biomarker research. Methods
**2014**, 70, 59–73. [Google Scholar] [CrossRef] [PubMed] - Farahani, N.; Parwani, A.V.; Pantanowitz, L. Whole slide imaging in pathology: Advantages, limitations and emerging perspectives. Pathol. Lab. Med. Int.
**2015**, 7, 23–33. [Google Scholar] [CrossRef] - Ghaznavi, F.; Evan, A.; Madabhushi, A.; Feldman, M. Digital imaging in pathology: Whole-slide imaging and beyond. Annu. Rev. Pathol. Mech. Dis.
**2013**, 8, 31–59. [Google Scholar] [CrossRef] [PubMed] - Razavi, S.; Hatipoglu, G.; Yalcin, H. Automatically diagnosing HER2 amplification status for breast cancer patients using large FISH images. In Proceedings of the 25th Signal Processing and Communications Applications Conference, Antalya, Turkey, 15–18 May 2017; pp. 1–4. [Google Scholar]
- Department of Computer Science, University of Warwick: Her2 Scoring Contest. Available online: http://www2.warwick.ac.uk/fac/sci/dcs/research/combi/research/bic/her2contest/ (accessed on 15 November 2016).
- Department of Computer Science, University of Warwick: Her2 Contest Results. Available online: http://www2.warwick.ac.uk/fac/sci/dcs/research/combi/research/bic/her2contest/outcome (accessed on 15 November 2016).
- Qaiser, T.; Mukherjee, A.; Reddy Pb, C.; Munugoti, S.D.; Tallam, V.; Pitkäaho, T.; Lehtimäki, T.; Naughton, T.; Berseth, M.; Pedraza, A.; et al. Her2 Challenge Contest: A detailed assessment of Her2 scoring algorithms and man vs machine in whole slide images of breast cancer tissues. Histopathology
**2018**, 72, 227–238. [Google Scholar] [CrossRef] [PubMed] - Mukundan, R. A Robust Algorithm for Automated Her2 Scoring in Breast Cancer Histology Slides Using Characteristic Curves. In Medical Image Understanding and Analysis; Communications in Computer and Information Science; Valdés Hernández, M., González-Castro, V., Eds.; Springer: Cham, Switzerland, 2017; Volume 723, pp. 386–397. [Google Scholar]
- Pietikainen, M.; Zhao, G.; Hadid, A.; Ahonen, T. Computer Vision Using Local Binary Patterns; Springer: London, UK, 2011; ISBN 978-0-85729-748-8. [Google Scholar]
- Goode, A.; Gilbert, B.; Harkes, J.; Jukie, D.; Satyanarayanan, M. OpenSlide: A vendor-neutral software foundation for digital pathology. J. Pathol. Inform.
**2013**, 4. [Google Scholar] [CrossRef] - Livanos, G.; Zervakis, M.; Giakos, G.C. Automated analysis of immunohistochemical images based on curve evolution approaches. In Proceedings of the IEEE Conference of Imaging Systems and Techniques, Beijing, China, 22–23 October 2013; pp. 112–115. [Google Scholar]
- Sørensen, L.; Shaker, S.B.; de Bruijne, M. Quantitative analysis of pulmonary emphysema using local binary patterns. IEEE Trans. Med. Imaging
**2010**, 29, 559–569. [Google Scholar] [CrossRef] [PubMed] - Morales, S.; Engan, K.; Naranjo, V.; Colomer, A. Detection of diabetic retinopathy and age-related macular degeneration from fundus images through local binary patterns and random forests. In Proceedings of the IEEE International Conference on Image Processing, Quebec City, QC, Canada, 27–30 September 2015; pp. 4838–4842. [Google Scholar]
- Sarwinda, D.; Bustamam, A. Detection of Alzheimer’s disease using advanced local binary pattern from hippocampus and whole brain of MR images. In Proceedings of the International Joint Conference on Neural Networks, Vancouver, BC, Canada, 24–29 July 2016; pp. 5051–5056. [Google Scholar]
- Tiwari, A.K.; Pachori, R.B.; Kanhangad, V.; Panigrahi, B.K. Automated diagnosis of epilepsy using key-point based local binary pattern of EEG signals. IEEE J. Biomed. Health Inform.
**2017**, 21, 888–896. [Google Scholar] [CrossRef] [PubMed] - Urdal, J.; Engan, K.; Kvikstad, V.; Janssen, E.A.M. Prognostic prediction of histopathological images by local binary patterns and RUSBoost. In Proceedings of the 25th European Signal Processing Conference, Kos, Greece, 2 September 2017; pp. 2349–2353. [Google Scholar]
- Sigirci, I.O.; Albayrak, A.; Bilgin, G. Detection of mitotic cells using completed local binary pattern in histopathological images. In Proceedings of the 23rd Signal Processing and Communications Applications Conference, Malatya, Turkey, 16–19 May 2015; pp. 1078–1081. [Google Scholar]
- Zhang, H.; Chen, Z.; Chi, Z.; Fu, H. Hierarchical local binary pattern for branch retinal vein occlusion recognition with fluorescein angiography images. Electron. Lett.
**2014**, 50, 1902–1904. [Google Scholar] [CrossRef] - Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell.
**2002**, 24, 971–987. [Google Scholar] [CrossRef] - Watt, J.; Borhani, R.; Katsaggelos, A.K. Machine Learning Refined: Foundations, Algorithms and Applications, 1st ed.; Cambridge Uniersity Press: Cambridge, UK, 2016; ISBN 978-1107123526. [Google Scholar]
- Keay, T.; Conway, C.M.; O’Flaherty, N.; Hewitt, S.M.; Shea, K.; Gavrielides, M.A. Reproducibility in the automated quantitative assessment of HER2/neu for breast cancer. J. Pathol. Inform.
**2013**, 4. [Google Scholar] [CrossRef]

**Figure 1.**Whole Slide Image (WSI) tiles showing different levels of staining and their corresponding HER2 scores. Reproduced from [12] with permission.

**Figure 2.**Processing stages in the extraction of characteristic curves and local binary pattern (LBP) features. ROI: region of interest.

**Figure 3.**Intermediate stages in the generation of a characteristic curve. Reproduced from [12] with permission.

**Figure 4.**Variations in the shapes of the characteristic curves with different levels of staining. Reproduced from [12] with permission.

**Figure 7.**The values of the first four uniform local binary pattern (uLBP) bins corresponding to four images with different HER2 scores. The x-axis denotes the variation of the saturation threshold s

_{low}from 0.1 to 0.5.

**Figure 8.**Convergence of the cost functions of the four-class logistic regression algorithm. Reproduced from [12] with permission.

**Figure 9.**An example showing two tile positions with varying image characteristics within the same WSI. Reproduced from [12] with permission.

HER2 Score | Assessment | Staining Pattern |
---|---|---|

0 | Negative | No staining is observed, or membrane staining is observed in less than 10% of tumor cells |

1+ | Negative | A faint/barely perceptible membrane staining is detected in greater than 10% of tumor cells. The cells exhibit incomplete membrane staining. |

2+ | Weakly Positive | A weak to moderate membrane staining is observed in greater than 10% of tumor cells. |

3+ | Positive | A strong complete membrane staining is observed in greater than 10% of tumor cells. |

**Table 2.**Number of WSIs provided for training and testing the classification algorithm. Reproduced from [12] with permission.

Training Set | Test Set | ||
---|---|---|---|

Ground Truth HER2 Score | Number of WSIs | Contest-1 No. of WSIs | Contest-2 No. of WSIs |

0 | 13 | 28 | 6 |

1+ | 13 | ||

2+ | 13 | ||

3+ | 13 | ||

Total | 52 |

Number of 1’s | Byte Values | |||||||
---|---|---|---|---|---|---|---|---|

0 | 0 | |||||||

1 | 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 |

2 | 3 | 6 | 12 | 24 | 48 | 96 | 192 | 129 |

3 | 7 | 14 | 28 | 56 | 112 | 224 | 193 | 131 |

4 | 15 | 30 | 60 | 120 | 240 | 225 | 195 | 135 |

5 | 31 | 62 | 124 | 248 | 241 | 227 | 199 | 143 |

6 | 63 | 126 | 252 | 249 | 243 | 231 | 207 | 159 |

7 | 127 | 254 | 253 | 251 | 247 | 239 | 223 | 191 |

8 | 255 |

**Table 4.**Confusion matrix for the multi-class logistic regression algorithm. Reproduced from [12] with permission.

HER2 Score | Predicted | Accuracy = 88.46% | |||||
---|---|---|---|---|---|---|---|

0 | 1+ | 2+ | 3+ | Precision | Recall | ||

Actual | 0 | 37 | 2 | 0 | 0 | 0.86 | 0.95 |

1+ | 6 | 29 | 4 | 0 | 0.83 | 0.74 | |

2+ | 0 | 4 | 34 | 1 | 0.87 | 0.87 | |

3+ | 0 | 0 | 1 | 38 | 0.97 | 0.97 |

**Table 5.**Confusion matrix for the multi-class logistic regression algorithm with the reduced feature set. Reproduced from [12] with permission.

HER2 Score | Predicted | Accuracy = 83.3% | |||||
---|---|---|---|---|---|---|---|

0 | 1+ | 2+ | 3+ | Precision | Recall | ||

Actual | 0 | 37 | 2 | 0 | 0 | 0.80 | 0.95 |

1+ | 8 | 24 | 7 | 0 | 0.75 | 0.61 | |

2+ | 1 | 6 | 31 | 1 | 0.79 | 0.79 | |

3+ | 0 | 0 | 1 | 38 | 0.97 | 0.97 |

**Table 6.**Confusion matrix for the multi-class logistic regression algorithm with uLBP feature vectors.

HER2 Score | Predicted | Accuracy = 90.38% | |||||
---|---|---|---|---|---|---|---|

0 | 1+ | 2+ | 3+ | Precision | Recall | ||

Actual | 0 | 38 | 1 | 0 | 0 | 0.86 | 0.97 |

1+ | 5 | 31 | 3 | 0 | 0.86 | 0.79 | |

2+ | 1 | 4 | 33 | 1 | 0.92 | 0.85 | |

3+ | 0 | 0 | 0 | 39 | 0.98 | 1.00 |

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mukundan, R.
Image Features Based on Characteristic Curves and Local Binary Patterns for Automated HER2 Scoring. *J. Imaging* **2018**, *4*, 35.
https://doi.org/10.3390/jimaging4020035

**AMA Style**

Mukundan R.
Image Features Based on Characteristic Curves and Local Binary Patterns for Automated HER2 Scoring. *Journal of Imaging*. 2018; 4(2):35.
https://doi.org/10.3390/jimaging4020035

**Chicago/Turabian Style**

Mukundan, Ramakrishnan.
2018. "Image Features Based on Characteristic Curves and Local Binary Patterns for Automated HER2 Scoring" *Journal of Imaging* 4, no. 2: 35.
https://doi.org/10.3390/jimaging4020035