Skin Capacitive Image Stitching and Occlusion Measurements

Abstract

The aim of this study is to develop new analysis techniques for skin capacitive image stitching and occlusion measurements. Through image stitching, small skin capacitive images can be stitched into large skin capacitive images and, therefore, provide more skin image information. Through occlusion, e.g., keeping the measurement device on skin for a period of time, the skin health status can be studied through time-dependent response curves. Results show that time-dependent skin capacitive imaging curves can tell us the information about transdermal water loss (TEWL) as well as skin surface profiles. By using the structural similarity index measure (SSIM), the TEWL map can be constructed, which shows the water loss map on the skin surface. We first present the theoretical background and then the experimental results.

1. Introduction

Skin capacitive imaging is a novel technology based on capacitive fingerprint sensor, which was originally developed for biometric application. Skin capacitive imaging has since found applications for skin water content measurements, skin solvent penetrations, skin texture, and skin microrelief analysis. Léveque et al. were the first to report it as a new tool for investigating the skin surface in vivo [1]. Batisse et al. used it to study skin surface [2]. Bevilacqua et al. studied the characterization of a capacitive imaging system for skin surface analysis [3]. Gherardi et al. used a capacitive image analysis system to characterize the skin surface [4]. Nam et al. designed and implemented a capacitive fingerprint sensor circuit in CMOS technology [5]. Singh et al. investigated skin capacitive imaging for in vivo skin hydration and microrelief measurements [6]. Singh et al. used skin capacitance imaging for skin surface profiles and skin dynamic water concentration measurements [7]. Martinelli et al. used an array of capacitive fingerprint sensors for chemical detections [8]. Bevilacqua et al. conducted age-related skin analysis by capacitance images [9]. Ou et al. developed a new in vivo skin capacitive imaging analysis technique by using a gray-level co-occurrence matrix (GLCM) [10]. Ou et al. developed a new skin image retrieval technique by using the Gabor wavelet texture feature [11]. Pan et al. developed image analysis techniques for skin capacitive images [12]. Zhang et al. used skin capacitive imaging for skin characterizations and solvent penetration measurements [13]. Bontozoglou et al. developed applications of skin capacitive imaging for human skin texture and hair analysis [14]. Xiao et al. studied the effect of suntan lotion on skin by using both transdermal water loss (TEWL) and skin capacitive imaging [15]. Zhang developed a skin capacitive imaging analysis tool by using Deep Learning GoogLeNet [16]. Hossain et al. incorporated deep learning into capacitive images for smartphone user authentication [17]. Navaraj et al. developed fingerprint-enhanced capacitive-piezoelectric flexible sensing skin to discriminate static and dynamic tactile stimuli for robotic arms [18]. Rowe et al. [19] and Sharma et al. [20] worked on multispectral fingerprint biometric systems, which have also recently become quite popular as they provide high security and recognition. Morgeneier provided a detailed review of the latest innovations in fingerprint capture [21]. Jeon et al. developed a highly sensitive self-capacitive fingerprint sensor [22].

In this paper, we present our latest research work on skin capacitive imaging by developing an image stitching technique and studying the skin capacitive imaging occlusion effects. We first present the theoretical background and then the experimental results.

2. Materials and Methods

2.1. Apparatus

The two measurement technologies used in this study are the Epsilon permittivity imaging system and the AquaFlux TEWL instrument (Biox Systems Ltd., London, UK), as shown in Figure 1. Epsilon is a skin capacitive imaging technology based on a Fujitsu fingerprint sensor, which has 256 × 300 pixels with 50 µm spatial resolution and 8-bit grayscale capacitance resolution per pixel [6,7,10,11,12,13,14,15,16]. With the Epsilon, skin capacitive images can be measured as a snapshot as well as an occlusive contact over a period of time. Unlike other similar skin measurement instruments, Epsilon is fully calibrated, which means it can measure the absolute dielectric constant of the sample. In physics, dielectric constant is the ratio of a material’s permittivity to the permittivity of free space. The dielectric constant of a material is a measure of its ability to store electrical energy.

AquaFlux is a closed condenser-chamber measurement technology for skin transdermal water loss (TEWL) measurements [23,24,25]. Its patented cylindrical measurement chamber provides a controlled measurement environment, which enhances the repeatability and accuracy of the measurements. With the AquaFlux, TEWL can be accurately measured. AquaFlux is also fully calibrated, which means the absolute water loss can be measured; the measurement unit is gram of water per unit square meter of area over a period of an hour.

2.2. Skin Capacitive Image Stitching

In skin capacitive imaging measurements, Epsilon can generate skin capacitive images with 17.5 mm × 15 mm size and 50 µm spatial resolution. Although this size of the skin image is fine for most of skin studies, it is inadequate if you want to observe larger skin areas, such as in the case of skin damages, skin diseases, or transdermal drug delivery. To solve this problem, a new skin capacitive image stitching technique has been developed in which skin capacitive image measurements were taken in a row, as shown in Figure 2. In order to have good stitching results, at least 30% overlap between two adjacent images is needed, according to previous experiences. A region of interest (ROI) was first selected on the right-hand side of image 1, and a search for the same region in image 2 was then performed. By locating the same region in images 1 and 2, images 1 and 2 can then be joined together; this is called stitching. This process is then repeated for the rest of the images.

Figure 3 shows the pseudocode of the skin capacitive image stitching technique.

The key of skin capacitive image stitching is the ROI searching algorithm. A ROI searching algorithm has been developed based on the technique called template matching [26,27]. If T(x,y) is used to represent the ROI of the first image, where x and y are the horizontal and vertical positions in the image, respectively, and I(x,y) is used to represent the second image, or the target image, then the following equation can be used to calculate the matching result image R(x,y) by using the normalized correlation coefficient (CCoeff_Normed) method:

$$\mathit{R}\left(\mathit{x},\mathit{y}\right)=\frac{{\sum }_{{\mathit{x}}^{\prime },{\mathit{y}}^{\prime }}\left(\mathit{T}\left({\mathit{x}}^{\prime },{\mathit{y}}^{\prime }\right)\times \mathit{I}\left(\mathit{x}+{\mathit{x}}^{\prime }, \mathit{y}+{\mathit{y}}^{\prime }\right)\right)}{\sqrt{{\sum }_{{\mathit{x}}^{\prime },{\mathit{y}}^{\prime }}{\left(\mathit{T}\left({\mathit{x}}^{\prime },{\mathit{y}}^{\prime }\right)\right)}^2\times {\sum }_{{\mathit{x}}^{\prime },{\mathit{y}}^{\prime }}{\left(\mathit{I}\left(\mathit{x}+{\mathit{x}}^{\prime }, \mathit{y}+{\mathit{y}}^{\prime }\right)\right)}^2}}$$

(1)

The location (x,y) in the second image where R(x,y) has the maximum value is the position of the best match.

2.3. Skin Capacitive Image Occlusion Measurements

Skin capacitive image occlusion is performed by keeping the Epsilon device on the skin for a period of time, i.e., occlusion, and recording the skin images over this period. In this study, the skin was occluded by using Epsilon for a period of 60 s, and the skin capacitive images were recorded once per second. The occlusion will block the skin transdermal water loss (TEWL) and, hence, increase the Epsilon reading over a period of time. The increase in the Epsilon reading should be proportional to skin TEWL. To prove that, TEWL measurements were also performed on the same skin site by using AquaFlux instrument. This is carried out by placing the AquaFlux TEWL instrument on the same skin site; a TEWL measurement typically takes about one minute.

A new method has also been developed to compare the texture changes of occluded skin capacitive images at different times based on an algorithm called the structural similarity index measure (SSIM) [28], which is typically used for measuring the similarity between two images.

SSIM compares two images (x and y) based on the computation of three factors: luminance (l), contrast (c), and structure (s):

$$\mathit{S}\mathit{S}\mathit{I}\mathit{M}\left(\mathit{x},\mathit{y}\right)={\left[\mathit{l}\left(\mathit{x},\mathit{y}\right)\right]}^{\mathit{\alpha }}+{\left[\mathit{c}\left(\mathit{x},\mathit{y}\right)\right]}^{\mathit{\beta }}+{\left[\mathit{s}\left(\mathit{x},\mathit{y}\right)\right]}^{\mathit{\gamma }}$$

(2)

where

$$\mathit{l}\left(\mathit{x},\mathit{y}\right)=\frac{2{\mathit{\mu }}_{\mathit{x}}{\mathit{\mu }}_{\mathit{y}}+{\mathit{C}}_1}{{\mathit{\mu }}_{\mathit{x}}{}^2+{\mathit{\mu }}_{\mathit{y}}{}^2+{\mathit{C}}_1}$$

(3)

$$\mathit{c}\left(\mathit{x},\mathit{y}\right)=\frac{2{\mathit{\sigma }}_{\mathit{x}}{\mathit{\sigma }}_{\mathit{y}}+{\mathit{C}}_2}{{\mathit{\sigma }}_{\mathit{x}}{}^2+{\mathit{\sigma }}_{\mathit{y}}{}^2+{\mathit{C}}_2}$$

(4)

$$\mathit{s}\left(\mathit{x},\mathit{y}\right)=\frac{{\mathit{\sigma }}_{\mathit{x}\mathit{y}}+{\mathit{C}}_3}{{\mathit{\sigma }}_{\mathit{x}}{\mathit{\sigma }}_{\mathit{y}}+{\mathit{C}}_3}$$

(5)

where μ_x, μ_y, σ_x, σ_y, and σ_xy are the local means, standard deviations, and cross-covariance for images x and y, respectively. The constants α, β, γ, and C₁, C₂, C₃, are empirical parameters that can be typically chosen as α = β = γ = 1 and C₃ = C₂/2.

The SSIM values range between 0 and 1, where 0 means does not match, and 1 means a perfect match between the two images.

In this experiment, three skin sites were chosen: skin site 1, upper volar forearm; skin site 2, lower volar forearm; and skin site 3, palm, as shown in Figure 4. For each skin site, both Epsilon occlusion and AquaFlux TEWL measurements were performed. The measurements were repeated on both left and right arms.

3. Results

All the measurements were performed on a healthy volunteer (female, 45–55, Caucasian) under normal ambient laboratory conditions of 20–21 °C and 40–50% RH. The volunteer was instructed to avoid excess water intake, and the measurements were performed in the morning. The volar forearm skin sites used were initially wiped clean with ETOH/H2O (95/5) solution. The volunteer was then acclimatized in the laboratory for 20 min prior to the experiments.

3.1. Skin Capacitive Image Stitching

Figure 5 shows the skin capacitive image stitching results. Figure 5A–D shows the four skin capacitive images consecutively captured along the volar forearm. The images were taken with at least 30% overlap. Moreover, the figure also shows the best-matched ROI at each image. Figure 5E shows the final stitched image. The stitching algorithm is implemented in the program Python. The original skin capacitive images have a size of 17.5 mm × 15 mm each, and the stitching final image has a size of about 40 mm × 10 mm. The result shows that the ROI search algorithm can effectively search and locate the best-matched ROI in each image. The stitched image can provide a better picture of the skin in a larger area. In this example, a residue of a skin moisturizing cream was left on the skin. The stitched image provides better understanding of the distribution of the cream on the skin surface.

3.2. Skin Capacitive Image Occlusion Measurements

Figure 6 shows the skin capacitive images of the upper volar forearm (skin site 1) at different times, 1–60 s. Figure 7 and Figure 8 show the skin capacitive images of the lower volar forearm (skin site 2) and palm (skin site 3). Again, the skin capacitive images have a size of 17.5 mm × 15 mm each. As you can see, the skin images are getting brighter and brighter, indicating the increase in skin hydration due to the occlusion.

By plotting the average dielectric constants of the above images against time, skin capacitive image occlusion curves can be obtained. Figure 9 shows the typical occlusion curves of three different skin sites, e.g., the upper volar forearm (skin site 1), lower volar forearm (skin site 2), and palm (skin site 3), over a period of 60 s. Again, the results confirm the increase of dielectric constants over time, but different skin sites have different shapes of curves. The volar forearm skin sites, skin sites 1 and 2, have a more or less linear increase, while skin site 3 has a curved increase; it has an accelerated increase in the beginning and then a linear increase later. This could be due to the skin texture differences between the skin sites, where, in the volar forearm, the skin is more uniform and has less gaps (microrelief lines), and, in the palm, the skin is less uniform and has more gaps.

Figure 10 shows the correlations between TEWL and the increase of dielectric constants. This is performed by plotting the TEWL values against the corresponding dielectric constant increases of the same skin sites. The increase of dielectric constants is calculated by using the image at 60 s minus the image at 1 s. TEWL values were measured by using AquaFlux. The results show a strong correlation between TEWL and the increase of dielectric constants. The R² (or the coefficient of determination) [29] shows a good fit of the data and the linear regression model, with a value of around 0.9.

However, at the palm skin site, with higher TEWL values, the data are more scattered. To understand this better, the Epsilon images at 1 s and 60 s were compared. The differences (Diff) and structural similarity index measure (SSIM) of the two images were also calculated. The results are shown in Figure 11. The results show that for the upper volar forearm (skin site 1) and lower volar forearm (skin site 2), both the Diff and SSIM results show very little skin texture information, which means the increase in skin hydration due to occlusion is more uniform across the skin surface. On the other hand, for the palm (skin site 3), both the Diff and SSIM results show strong skin surface texture information, which means the water comes off the skin surface differently at different locations, depending on the skin texture. This skin texture-dependent water loss was named as the “TEWL map”. The TEWL map can be very useful for studying skin damages, skin diseases, as well as skin transdermal drug delivery. The results also show that the SSIM results carry more skin texture information than the Diff results.

4. Discussion

Skin capacitive image stitching is similar and yet different from traditional panoramic photography [30]. On one hand, skin capacitive image stitching is a lot simpler, as skin capacitive images do not need to tilt, or there is no need to worry about lighting when stitching them; they also do not have parallax errors [31] as photos often do. On the other hand, skin capacitive image stitching is more difficult than photo stitching, as skin capacitive images are a lot more similar with each other than real-life photos. Traditional photo stitching is performed through detecting and locating key interest points at different photos; key interest points are also called features [32,33]. By identifying the position of the same key interest points at two photos, the two photos can then be joined together. Unfortunately, this method does not work for skin capacitive images due to their higher similarity. The new skin capacitive image stitching technique developed in this study is much more efficient in handling images with higher similarity.

Our previous studies showed that skin occlusion measurements by using the Epsilon instrument are a very effective method to study skin health conditions [34,35]. With occlusion curves, the skin can be identified whether it is healthy or damaged; the different types of skin damages can be identified as well. In this study, it was also discovered that skin occlusion measurements also carry skin TEWL information. The increase of dielectric constants is highly correlated with TEWL measurement results, e.g., the higher the TEWL values, the higher the increase, and the lower the TEWL values, the lower the increase. However, this correlation does depend on skin uniformity or texture. To study the skin uniformity, the structural similarity index measure (SSIM) has been used to construct the skin TEWL map, which shows the water loss map on the skin surface.

5. Conclusions

We presented our latest research work on the development of new analysis techniques for skin capacitive image stitching and occlusion measurements. Skin capacitive image stitching is performed through an ROI searching algorithm based on template matching. With skin capacitive image stitching, images of a larger skin area can be obtained, which will be very useful for many skin studies. Skin capacitive image occlusion is performed by keeping the Epsilon device on the skin for a period of time, i.e., occlusion, and recording the skin images over this period of time. The results showed that the increase of dielectric constants is highly correlated with skin TEWL values, e.g., the higher skin water loss will cause the higher increase of dielectric constants. At the skin sites where the skin is more uniform, the results are more repeatable, whereas at the skin sites where the skin is less uniform, the results are more scattered. By using the structural similarity index measure, a skin “TEWL map” can be generated. The TEWL map can be very useful for studying the heterogeneity of the skin.