#
Mereotopological Correction of Segmentation Errors in Histological Imaging^{ †}

^{1}

^{2}

^{3}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Discrete Mereotopology (DM)

#### 3.1. Conceptual Neighbourhood Graphs, Continuity and Change, and Composition Tables

#### 3.2. Implementation

## 4. Examples

#### 4.1. Example 1: Segmenting Cells in Culture Using Set-Theoretic and Discrete Topological Operators

- Start with $\mathsf{PO}({n}^{\prime},{c}_{1})$.
- By 15${}_{\mathsf{PO}}$ this gives $\mathsf{PO}\left|\mathsf{NTPPi}\right|\mathsf{TPPi}({\mathsf{cl}}_{D}({n}^{\prime}),{c}_{1})$.
- By 1${}_{\mathsf{PO}\left|\mathsf{NTPPI}\right|\mathsf{TPPI}}$ this gives $\mathsf{EQ}\left|\mathsf{NTPP}\right|\mathsf{TPP}({\mathsf{cl}}_{D}({n}^{\prime}),\mathsf{sum}({\mathsf{cl}}_{D}({n}^{\prime}),{c}_{1}))$.
- Next, from $\mathsf{EQ}({n}^{\prime},{n}^{\prime})$ and 16${}_{\mathsf{EQ}}$ we have $\mathsf{EQ}|\mathsf{NTPP}({n}^{\prime},{\mathsf{cl}}_{D}({n}^{\prime}))$.
- The RCC8D weak composition $\mathsf{EQ}|\mathsf{NTPP}\circ \mathsf{EQ}|\mathsf{NTPP}|\mathsf{TPP}$ is $\mathsf{EQ}\left|\mathsf{NTPP}\right|\mathsf{TPP}$.
- Hence from 3 and 4 using 5 we have $\mathsf{EQ}\left|\mathsf{NTPP}\right|\mathsf{TPP}({n}^{\prime},\mathsf{sum}({\mathsf{cl}}_{D}({n}^{\prime}),{c}_{1}))$.

#### 4.2. Example 2: Segmenting Cells in Culture: Adding Other Morphological Operators

## 5. Discussion

## 6. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Abbreviations

H&E | haematoxylin and eosin stain |

QSR | qualitative spatial reasoning |

DM | discrete mereotopology |

MM | mathematical morphology |

RCC | region connection calculus |

RCC5/8D | the five/eight-element relation sets used in DM |

$\mathsf{sum}$ | union operator |

$\mathsf{prod}$ | intersection operator |

$\mathsf{diff}$ | difference operator |

$\mathsf{compl}$ | complementation operator |

$\mathsf{xor}$ | exclusive-or (or symmetrical difference) operator |

${\mathsf{int}}_{D}$ | discrete interior operator |

${\mathsf{cl}}_{D}$ | discrete closure operator |

${\mathsf{ext}}_{D}$ | discrete exterior operator |

${\mathsf{bndry}}_{D}$ | discrete boundary operator |

$\mathsf{P}$ | part |

$\mathsf{O}$ | overlap |

$\mathsf{C}$ | contact or connection |

JEPD | jointly exhaustive pairwise disjoint |

$\mathsf{DC}$ | disconnection |

$\mathsf{EC}$ | external connection |

$\mathsf{PO}$ | partial overlap |

$\mathsf{TPP}$ | tangential proper-part |

$\mathsf{NTPP}$ | non-tangential proper part |

$\mathsf{EQ}$ | equality |

$\mathsf{DR}$ | disjoint |

$\mathsf{PP}$ | proper-part |

$\mathsf{R}|{\mathsf{R}}^{\prime}$ | disjunction of R and R’ |

$\mathsf{Ri}$ | inverse relation of R |

$\mathsf{SC}$ | self-connected |

${\mathsf{nbhd}}_{\langle \alpha ,\beta \rangle}(R)$ | conceptual neighbourhood of binary relation R |

RCC8D-CT | the RCC8D composition table |

IJ | ImageJ image processing software |

${\mathsf{cl}}_{D}^{-}$ | disjoint discrete closure |

## Appendix A

#### Appendix A.1.

## References

- Randell, D.A.; Galton, A.; Fouad, S.; Mehanna, H.; Landini, G. Model-based correction of segmentation errors in digitised histological images. In Communications in Computer and Information Science, Proceedings of the 21st Conference on Medical Image Understanding and Analysis, Edinburgh, UK, 11–13 July 2017; Springer: Berlin, Germany, 2017; Volume 723, pp. 718–730. [Google Scholar]
- Galton, A.P. The Mereotopology of Discrete Space. In Proceedings of the International Conference on Spatial Information Theory: Cognitive and Computational Foundations of Geographic Information Science (COSIT ’99), Stade, Germany, 25–29 August 1999; Springer: London, UK, 1999; pp. 251–266. [Google Scholar]
- Galton, A.P. Discrete Mereotopology. In Mereology and the Sciences: Parts and Wholes in the Contemporary Scientific Context; Springer: Berlin, Germany, 2014; pp. 293–321. [Google Scholar] [Green Version]
- Bloch, I. Spatial Reasoning under Imprecision Using Fuzzy Set Theory, Formal Logics and Mathematical Morphology. Int. J. Approx. Reason.
**2006**, 41, 77–95. [Google Scholar] [CrossRef] - Galton, A.; Landini, G.; Randell, D.; Fouad, S. Ontological levels in histological imaging. In Formal Ontology in Information Systems; Ferrario, R., Kuhn, W., Eds.; IOS Press: Annecy, France, 2016; pp. 271–284. [Google Scholar]
- Landini, G.; Randell, D.; Galton, A. Discrete Mereotopology in Histological Imaging. In Proceedings of the 17th Conference on Medical Image Understanding and Analysis, Birmingham, UK, 17–19 July 2013; pp. 101–106. [Google Scholar]
- Landini, G.; Randell, D.A.; Breckon, T.; Han, J.W. Morphologic characterization of cell neighborhoods in neoplastic and preneoplastic epithelium. Anal. Quant. Cytol. Histol.
**2010**, 32, 30–38. [Google Scholar] [PubMed] - Randell, D.; Landini, G. Discrete Mereotopology in Automated Histological Image Analysis. In Proceedings of the Second ImageJ User and Developer Conference, Luxembourg, 7–8 November 2008; pp. 151–156. [Google Scholar]
- Randell, D.; Landini, G.; Galton, A.P. Discrete Mereotopology in Automated Histological Image Analysis. IEEE Trans. Pattern Anal. Mach. Intell.
**2013**, 35, 568–581. [Google Scholar] [CrossRef] [PubMed] - Randell, D.A.; Cui, Z.; Cohn, A.G. A Spatial Logic Based on Regions and Connection. In Proceedings of the 3rd International Conference on Knowledge Representation and Reasoning, Los Altos, CA, USA, 26–29 October 1992; pp. 165–176. [Google Scholar]
- Li, S.; Ying, M. Generalised Region Connection Calculus. Artif. Intell.
**2004**, 160, 1–34. [Google Scholar] [CrossRef] - Cui, Z.; Cohn, A.G.; Randell, D.A. Qualitative Simulation Based on a Logic of Space and Time. In Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI’92), San Jose, CA, USA, 12–16 July 1992; pp. 679–684. [Google Scholar]
- Sioutis, M.; Condotta, J.-F.; Salhi, Y.; Mazure, B.; Randell, D.A. Ordering Spatio-Temporal Sequences to Meet Transition Constraints: Complexity and Framework. In Proceedings of the Artificial Intelligence Applications and Innovations (AIAI-2015), Beijing, China, 7–9 September 2005; pp. 130–150. [Google Scholar]
- Chen, J.; Cohn, A.G.; Liu, D.; Wang, S.; Ouyang, J.; Yu, Q. A Survey of Qualitative Spatial Representations. Knowl. Eng. Rev.
**2015**, 30, 106–136. [Google Scholar] [CrossRef] - Gurcan, M.N.; Boucheron, L.E.; Can, A.; Madabhushi, A.; Rajpoot, N.M.; Yener, B. Histopathological Image Analysis: A Review. IEEE Rev. Biomed. Eng.
**2009**, 2, 147–171. [Google Scholar] [CrossRef] [PubMed] - Sirinukunwattana, K.; Pluim, J.P.W.; Chen, H.; Qi, X.; Heng, P.-A.; Guo, Y.B.; Wang, L.Y.; Matuszewski, B.J.; Bruni, E.; Sanchez, U.; et al. Gland Segmentation in Colon Histology Images: The GlaS Challenge Contest. Med. Image Anal.
**2016**, 35, 489–502. [Google Scholar] [CrossRef] [PubMed] - Li, S.; Ying, M. Region Connection Calculus: Its Models and Composition Table. Artif. Intell.
**2003**, 145, 121–146. [Google Scholar] [CrossRef] - Weidenbach, C.; Dimova, D.; Fietzke, A.; Kumar, R.; Suda, M.; Wischnewski, P. SPASS Version 3.5. In Proceedings of the 22nd International Conference Automated Deduction, Montreal, QC, Canada, 2–7 August 2009; pp. 140–145. [Google Scholar]
- Ruifrok, A.C.; Johnston, D.A. Quantification of Histochemical Staining by Color Deconvolution. Anal. Quant. Cytol. Histol.
**2001**, 23, 291–299. [Google Scholar] [PubMed] - Landini, G.; Randell, D.A.; Fouad, S.; Galton, A. Automatic thresholding from the gradients of region boundaries. J. Microsc.
**2017**, 265, 185–195. [Google Scholar] [CrossRef] [PubMed] - Zuiderveld, K. Contrast Limited Adaptive Histogram Equalization, Graphics Gems IV; Academic Press Professional, Inc.: San Diego, CA, USA, 1994; pp. 474–485. [Google Scholar]
- Prewitt, J.M.S.; Mendelsohn, M.L. The analysis of cell images. Ann. N. Y. Acad. Sci.
**1966**, 128, 1035–1053. [Google Scholar] [CrossRef] [PubMed] - Soille, P.V. Morphological Image Analysis: Principles and Applications, 2nd ed.; Springer: Berlin, Germany, 2003. [Google Scholar]

**Figure 1.**An example of a H&E (haematoxylin and eosin)-stained H400 cell (

**a**) and an idealised representation of this in (

**b**) where the nucleus (violet) forms an expected proper-part of its host cell. In comparison, (

**c**) depicts an anomalous case where the nucleus overlaps but projects beyond the cytoplasm boundary forming a partial-overlap relation, with two morphologically-based corrections of this in (

**d**,

**e**) and where the original profiles of the nucleus and cytoplasm prior to the correction are outlined in black.

**Figure 2.**Directed graphs encoding set theoretic operators. In each case, the regions and the resulting operation on them are non-null. $\mathsf{DC}$: disconnection; $\mathsf{EC}$: external connection; $\mathsf{PO}$: partial overlap; $\mathsf{TPP}$: tangential proper part; $\mathsf{NTPP}$: non-tangential proper part; $\mathsf{TPPi}$: inverse of $\mathsf{TPP}$; $\mathsf{NTPPi}$: inverse of $\mathsf{NTPP}$; $\mathsf{EQ}$: equality.

**Figure 3.**Directed graphs encoding discrete topological operators. In each case the regions and the resulting operation on them are non-null.

**Figure 4.**An example of segmenting and resegmenting cells from a H400 haematoxylin and eosin (H&E)- stained culture using set-theoretic and discrete topological operators. The top row shows (

**a**) original H&E-stained image (field width 65 micrometres); (

**b**) haematoxylin channel; (

**c**) eosin channel; (

**d**) binary segmented nuclei; (

**e**) binary segmented cytoplasm. In the second row, (

**f**) is a colour composite merge of (d,e) where magenta, green, and white correspond to cytoplasm, nucleus, and cytoplasm + nucleus overlap, respectively; From (

**g**) to (

**j**) are shown possible resegmentations of the top cell in (f) that correct the $\mathsf{PO}$ relationship of the nucleus and cytoplasm into $\mathsf{PP}$ by means of successive erosions of the nucleus (

**g**); corresponding dilations of the cytoplasm (

**h**); extending the footprint of the cytoplasm so the nucleus is contained as a part (

**i**); and reducing the nucleus in an intersection operation with the cytoplasm in (

**j**); The third row shows (

**k**) as an example of a nucleus overlapping two cytoplasm regions and possible resegmentations by splitting the nuclear region into two separated parts in an intersection operation (

**l**); removing the non-overlapping region between the two cells with a union operation (

**m**); adding the nucleus to one cell and removing the overlap from the other (

**n**); and adding the nuclear boundary to one cell and removing the closure of the result from the other (

**o**). See text for details of the procedures involved.

**Figure 5.**(

**a**) Original cropped image of cells in H400 H&E-stained culture; (

**b**) nuclei segmented using a regional gradient and circularity contraint method; (

**c**) cytoplasm mask using the histogram-based minimum method; (

**d**) binary watershed on (c) and colour composite merge with (b); nuclei = green, magenta = cytoplasm, white = nuclei/cytoplasm overlap showing $\mathsf{PO}$ relations formed where the watershed boundaries cut the nuclei; (

**e**) repartitioning the cytoplasm by extracting the watershed-induced PO cases, and merging the associated adjacent regions of cytoplasm in (

**f**); (

**g**) resegmentation of PO cases using (f,b): colour composite merge: nuclei = green, magenta = cytoplasm; (

**h**) detail of the watershed partitioning generating partially overlapping nuclei and cytoplasm, and the result of the cytoplasm merge and resegmentation using a binary dilation without merging operation. Remaining PO cases (see single green pixel at 9 o’clock of upper-right cell) is then reassigned to the cytoplasm using the sum operator—see Section 4.2 for explanation.

**Table 1.**Resegmentation details for Figure 4. Here CT refers to the RCC8D composition table.

Figure | Initial Relation | Relation after Resegmentation | Graph Operation |
---|---|---|---|

(g) | $\mathsf{PO}(n,c)$ | $\mathsf{TPP}({\mathsf{int}}_{D}^{5}(n),c)$ | ${13}_{\mathsf{PO}}$ |

(h) | $\mathsf{PO}(n,c)$ | $\mathsf{TPP}(n,{\mathsf{cl}}_{D}^{6}(c))$ | ${16}_{\mathsf{PO}}$ |

(i) | $\mathsf{PO}(n,c)$ | $\mathsf{TPP}(n,\mathsf{sum}(n,c))$ | ${1}_{\mathsf{PO}}$ |

(j) | $\mathsf{PO}(n,c)$ | $\mathsf{TPP}(\mathsf{prod}(n,c),c)$ | ${4}_{\mathsf{PO}}^{\prime}$ |

(l) $\left(\right)$ | $\begin{array}{c}\mathsf{PO}({n}^{\prime},{c}_{1})\hfill \\ \mathsf{PO}({n}^{\prime},{c}_{2})\hfill \end{array}$ | $\left(\right)$ | ${4}_{\mathsf{PO}}^{\prime}$ |

(m) | $\mathsf{PO}({n}^{\prime},{c}_{1})$ | $\mathsf{TPP}({n}^{\prime},\mathsf{sum}({n}^{\prime},{c}_{1}))$ | ${1}_{\mathsf{PO}}$ |

(n) $\left(\right)$ | $\begin{array}{c}\mathsf{PO}({n}^{\prime},{c}_{1})\hfill \\ \mathsf{PO}({n}^{\prime},{c}_{2})\hfill \end{array}$ | $\begin{array}{c}\mathsf{TPP}({n}^{\prime},\mathsf{sum}({n}^{\prime},{c}_{1}))\hfill \\ \mathsf{DC}({n}^{\prime},\mathsf{diff}({c}_{2},{\mathsf{cl}}_{D}({n}^{\prime})))\hfill \end{array}$ | $\begin{array}{c}{1}_{\mathsf{PO}}\\ {15}_{\mathsf{PO}},{6}_{\mathsf{PO}},{16}_{\mathsf{EQ}},\mathrm{CT}\end{array}$ |

(o) $\left(\right)$ | $\begin{array}{c}\mathsf{PO}({n}^{\prime},{c}_{1})\\ \mathsf{PO}({n}^{\prime},{c}_{2})\end{array}$ | $\begin{array}{c}\mathsf{NTPP}({n}^{\prime},\mathsf{sum}({\mathsf{cl}}_{D}({n}^{\prime}),{c}_{1}))\hfill \\ \mathsf{DC}({n}^{\prime},\mathsf{diff}({c}_{2},{\mathsf{cl}}_{D}^{2}({n}^{\prime}))\hfill \end{array}$ | $\begin{array}{c}{15}_{\mathsf{PO}},{1}_{\mathsf{PO}},{16}_{\mathsf{EQ}},\mathrm{CT}\\ {15}_{\mathsf{PO}},{6}_{\mathsf{PO}},{16}_{\mathsf{EQ}},{16}_{\mathsf{NTPP}},\mathrm{CT}\end{array}$ |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Randell, D.A.; Galton, A.; Fouad, S.; Mehanna, H.; Landini, G.
Mereotopological Correction of Segmentation Errors in Histological Imaging. *J. Imaging* **2017**, *3*, 63.
https://doi.org/10.3390/jimaging3040063

**AMA Style**

Randell DA, Galton A, Fouad S, Mehanna H, Landini G.
Mereotopological Correction of Segmentation Errors in Histological Imaging. *Journal of Imaging*. 2017; 3(4):63.
https://doi.org/10.3390/jimaging3040063

**Chicago/Turabian Style**

Randell, David A., Antony Galton, Shereen Fouad, Hisham Mehanna, and Gabriel Landini.
2017. "Mereotopological Correction of Segmentation Errors in Histological Imaging" *Journal of Imaging* 3, no. 4: 63.
https://doi.org/10.3390/jimaging3040063