Vision-Based System for Automated Estimation of the Frontal Area of Swimmers: Towards the Determination of the Instant Active Drag: A Pilot Study

Swimmers take great advantage by reducing the drag forces either in passive or active conditions. The purpose of this work is to determine the frontal area of swimmers by means of an automated vision system. The proposed algorithm is automated and also allows to determine lateral pose of the swimmer for training purposes. In this way, a step towards the determination of the instantaneous active drag is reached that could be obtained by correlating the effective frontal area of the swimmer to the velocity. This article shows a novel algorithm for estimating the frontal and lateral area in comparison with other models. The computing time allows to obtain a reasonable online representation of the results. The development of an automated method to obtain the frontal surface area during swimming increases the knowledge of the temporal fluctuation of the frontal surface area in swimming. It would allow the best monitoring of a swimmer in their swimming training sessions. Further works will present the complete device, which allows to track the swimmer while acquiring the images and a more realistic model of conventional active drag ones.


Preliminaries
Swimming performance is affected by both the propulsive and resistive forces [1]. Regarding the hydrodynamic drag, this force can be defined as an external force that acts in the swimmer's body parallel but in the opposite direction of his movement [2,3]. According to Newton's second law of motion, the sum of these forces determines the acceleration of the swimmer (Equation (1)). m a = F P + F R (1) where m is the swimmer's mass, a is the acceleration of the swimmer, and F P and F R are the total propulsive and resistive forces, respectively. According to Equation (1), when the magnitude of the propulsive and the resistive force are equal, a swimmer would maintain a constant speed. When, however, the amount of one force exceeds that of the other, the swimmer will accelerate or decelerate. Thus, swimmers should have an ability to both produce large propulsive force and to reduce resistive forces for the purpose of improving performance. Quantifying these forces when swimming is therefore very beneficial in improving our understanding about swimming technique and improving swimming performance. In this sense, it has been found that speed in swimming sprint events is positively correlated to maximal force and mean force production [4,5]. Nevertheless, an

Related Works
The most recent work of Takagi et al. [16] reviewed the literature on front crawl, focusing on propulsive and resistive forces at different swimming velocities. The relationships between energetic, biomechanical, and fluid dynamics indices in competitive swimming were established factors that determine the mean of the instantaneous magnitudes of hand velocity over some time. Experimental attempts to quantify the time-series resistive force have been scarce. The only study in which the time-series resistive force was assessed was conducted by Morais et al. [17]. The time-series resistive force was quantified by obtaining the time-series frontal surface area and forward swimming velocity and applying these to the steady-state equation below (Equation (2)): where F Ri is the resistive force at time i, C R is the resistive force coefficient derived from VPM, ρ is the density of the water, and A i and v i are the frontal surface area and the forward velocity of the swimmer at time i. A limitation of this method is the application of the steady-state equation to a non-steady-state (fluctuating velocity) condition, which suggests that the absolute drag value derived from this method might not be accurate. Nevertheless, it is likely that the fluctuation of the frontal surface area affects proportionally the resistive force variation, as a larger reference area would produce a larger hydrodynamic pressure difference between the front and back sides of the body. Thus, quantifying changes in the frontal surface area during swimming should provide an insight into the resistive force fluctuation. The frontal surface area during swimming has also been assessed in another study in which intra-cycle variation of the frontal surface area was investigated in the four swimming strokes and in streamlined kicking [18]. In both studies, however, the frontal surface area was obtained at a very low sampling frequency. Morais et al. [17] obtained the surface area at only five key events in one stroke cycle and estimated the time-series one-cycle frontal surface area data using a spline function, and Gatta et al. [18] calculated the surface area with 12.5 Hz, which was 2-4 times lower than the recommended sampling frequency range for swimming motion analysis (25-50 Hz) [19].
The reason for the low sampling frequency used in the two studies was probably due to the analysis process involved. Researchers in both studies manually outlined the body in video images to obtain the frontal surface area, which was probably very time consuming. Considering that manual processing of data obtained at a high sampling frequency is both challenging and time consuming, establishing an automatic process to obtain the frontal surface area would substantially improve our ability to collect and analyse more complex and accurate data. At the moment, this is an open problem that we tried to sort out.
The purpose of the present study was to develop an automated method to obtain the frontal surface area during swimming. Such a method would be of great benefit for both swimming researchers and practitioners when exploring the relationships between technique and resistive forces in swimming, as it would lead us better understand the temporal fluctuation of the frontal surface area that is directly linked to the time-series resistive force in swimming.

Experimental Protocol
A set of experimental tests were conducted with one regional male swimmer (age: 20 y; body mass 68 kg; height: 173 cm; training hours: 9 h per week). The swimmer was specialized in individual medley events and had 5 years of competitive training experience. The participant visited the swimming pool in a non-fatigued state (non-intense exercise in the 48 h and no strength training in the 72 h prior to testing). The study was performed in accordance with the Declaration of Helsinki (October 2008, Seoul, Korea), and the experimental protocols were approved by the ethical committee of the local University (Approval Number UNNE-2020-010).
Before the test started, an investigator took pictures of the whole body in different positions for subsequent calculation of the dry model (see Figure 1). Following this process, the swimmer performed a 400-m standardized warm-up consisting of 200-m easy front crawl, 100 m of short sprint sets (12.5 fast and 12.5 easy), and two sets of 25-m kick and 25-m swim. All measurements were conducted on the same day in an indoor 25-m swimming pool with a water temperature of 27 degrees Celsius.

Hardware Description and Experimental Setting
Two underwater cameras (frontal and lateral) filmed 5 different 60-s tethered swimming trials at sub-maximal intensity with a 90-s rest between each trial. The cameras were positioned at 60-cm depth taking into account the swimming pool characteristics. The swimmer was connected to SwimOne device by means of a hardness through a steel cable [20]. For the work presented here, the drum of SwimOne was locked, and the swimmer therefore performed the swimming trials without moving forward. Figure 2a illustrates the testing setting, and Figure 2b details the camera position regarding to the swimmer, with and as the angle and distance from the reference point of the swimmer to the lateral camera and and as the angle and distance to the frontal one. The placement of the cameras was established trying to minimize and as much as possible taking into account the pool limitation. The final values that determine the cameras poses are = 5.6°, = 11.2°, = 3140 mm, and = 6352 mm. Swimmer was placed at 3 and 6 m from the lateral and front cameras, respectively (taking as reference the swimmer hips), in order to reduce as much as possible and . In this way, we could obtain an optimal balance between the validity of the measurements obtained and the quality of the swimmer image. Both cameras were the same model and allowed to record videos of 1080 × 1920 resolution with a framerate of 30 fps (normal mode). The cameras were calibrated using a series of poles of fixed length. They were positioned at specifically known positions (sagittal and frontal plane) throughout the area in which swimmers performed each trial (Figure 3).

Hardware Description and Experimental Setting
Two underwater cameras (frontal and lateral) filmed 5 different 60-s tethered swimming trials at sub-maximal intensity with a 90-s rest between each trial. The cameras were positioned at 60-cm depth taking into account the swimming pool characteristics. The swimmer was connected to SwimOne device by means of a hardness through a steel cable [20]. For the work presented here, the drum of SwimOne was locked, and the swimmer therefore performed the swimming trials without moving forward. Figure 2a illustrates the testing setting, and Figure 2b details the camera position regarding to the swimmer, with θ L and ρ L as the angle and distance from the reference point of the swimmer to the lateral camera and θ F and ρ F as the angle and distance to the frontal one. The placement of the cameras was established trying to minimize θ L and θ F as much as possible taking into account the pool limitation. The final values that determine the cameras poses are θ F = 5.6 • , θ L = 11.2 • , ρ L = 3140 mm, and ρ F = 6352 mm. Swimmer was placed at 3 and 6 m from the lateral and front cameras, respectively (taking as reference the swimmer hips), in order to reduce as much as possible θ F and θ L . In this way, we could obtain an optimal balance between the validity of the measurements obtained and the quality of the swimmer image.

Hardware Description and Experimental Setting
Two underwater cameras (frontal and lateral) filmed 5 different 60-s tethered swimming trials at sub-maximal intensity with a 90-s rest between each trial. The cameras were positioned at 60-cm depth taking into account the swimming pool characteristics. The swimmer was connected to SwimOne device by means of a hardness through a steel cable [20]. For the work presented here, the drum of SwimOne was locked, and the swimmer therefore performed the swimming trials without moving forward. Figure 2a illustrates the testing setting, and Figure 2b details the camera position regarding to the swimmer, with and as the angle and distance from the reference point of the swimmer to the lateral camera and and as the angle and distance to the frontal one. The placement of the cameras was established trying to minimize and as much as possible taking into account the pool limitation. The final values that determine the cameras poses are = 5.6°, = 11.2°, = 3140 mm, and = 6352 mm. Swimmer was placed at 3 and 6 m from the lateral and front cameras, respectively (taking as reference the swimmer hips), in order to reduce as much as possible and . In this way, we could obtain an optimal balance between the validity of the measurements obtained and the quality of the swimmer image. Both cameras were the same model and allowed to record videos of 1080 × 1920 resolution with a framerate of 30 fps (normal mode). The cameras were calibrated using a series of poles of fixed length. They were positioned at specifically known positions (sagittal and frontal plane) throughout the area in which swimmers performed each trial ( Figure 3). For illustrative purpose, Figure 4 represents two videoframes of the lateral and frontal cameras.  For illustrative purpose, Figure 4 represents two videoframes of the lateral and frontal cameras. For illustrative purpose, Figure 4 represents two videoframes of the lateral and frontal cameras.  Figure 5 summarizes the steps in the algorithm for automatic detection of the swimmer's frontal area, which was developed using Computer Vision Toolbox of Matlab TM . Figure 5a represents the Data Flow Diagram of the Algorithm, and Figure 5b shows the equivalent pseudocode.

Frontal area Detection Algorithm
The algorithm starts with extracting each videoframe to be processed. These frames are captured in RGB colormap. Then, the region of interest (ROI) is selected with the purpose to decrease the computing time by analysing only the region where the swimmer is placed at the acquired image [21]. Figure 6 shows the ROI of a frontal frame for illustrative purpose.  Figure 5 summarizes the steps in the algorithm for automatic detection of the swimmer's frontal area, which was developed using Computer Vision Toolbox of Matlab TM . Figure 5a represents the Data Flow Diagram of the Algorithm, and Figure 5b shows the equivalent pseudocode.

Frontal Area Detection Algorithm
The algorithm starts with extracting each videoframe to be processed. These frames are captured in RGB colormap. Then, the region of interest (ROI) is selected with the purpose to decrease the computing time by analysing only the region where the swimmer is placed at the acquired image [21]. Figure 6 shows the ROI of a frontal frame for illustrative purpose.  The following steps are to detect the swimwear, the harness, and the cap of the swimmer and to replace it by a region with similar colour to the skin of the swimmer.
In this study, swimwear was dark red, and the cap was black, together with the harness of SwimOne device (see Figure 1), which were all replaced with other colours to make a clear contrast with the water colour using colour thresholding technique [22].
The resulting mask was configured in CIELAB colour space, and before this step, RGB frame was converted to CIELAB colour space, where represents the lightness of the pixel, represents the value of the colour line between green and red, and channel represents the value of the colour line between blue and yellow [23,24]. Appendix A summarizes the equations to obtain channels L, a, and b from RGB ones. The mask for removing swimwear in the frontal camera is defined by the following: 25.466 < < 98.620 −46.336 < < 8.860 −31.446 < < 60.616 The following steps are to detect the swimwear, the harness, and the cap of the swimmer and to replace it by a region with similar colour to the skin of the swimmer.
In this study, swimwear was dark red, and the cap was black, together with the harness of SwimOne device (see Figure 1), which were all replaced with other colours to make a clear contrast with the water colour using colour thresholding technique [22].
The resulting mask was configured in CIELAB colour space, and before this step, RGB frame was converted to CIELAB colour space, where L represents the lightness of the pixel, a represents the value of the colour line between green and red, and b channel represents the value of the colour line between blue and yellow [23,24]. Appendix A summarizes the equations to obtain channels L, a, and b from RGB ones. The mask for removing swimwear in the frontal camera is defined by the following: 25.466 < L < 98.620 −46.336 < a < 8.860 −31.446 < b < 60.616 (3) with L, a, and b representing the lightness, red-green, and blue-yellow channels, respectively.
This mask was applied to identify the pixels illustrating the swimwear, cap, and harness, which were substituted in the RGB image by (RGB) = (27,131,135), which corresponded to a pixel colour of the chest of the swimmer. Figure 7 illustrates the application of the mask to a frame with the ROI of the original image. with , , and representing the lightness, red-green, and blue-yellow channels, respectively.
This mask was applied to identify the pixels illustrating the swimwear, cap, and harness, which were substituted in the RGB image by ( ) = (27,131,135), which corresponded to a pixel colour of the chest of the swimmer. Figure 7 illustrates the application of the mask to a frame with the ROI of the original image. The following steps consist of removing water area and other elements, such as lanes. These steps were implemented by other sequential masks. The first one was applied on a normalised CIELAB colourmap: 5 < * < 160 (4) The following steps consist of removing water area and other elements, such as lanes. These steps were implemented by other sequential masks. The first one was applied on a normalised CIELAB colourmap: with L * as the normalization of L channel into (0, 255) range. Other elements were removed by a filter area down to the water line, which is characterized by i = 97, with i as the row coordinate of the frame ROI. The result of applying both masks is illustrated in Figure 8. The filter shown in Equation (4) together with the area filter (i ≥ 97) also remove the possible reflexion of the image of the swimmer up to the water line. The following steps consist of removing water area and other elements, such as lanes. These steps were implemented by other sequential masks. The first one was applied on a normalised CIELAB colourmap: 5 < * < 160 (4) with * as the normalization of channel into (0, 255) range.
Other elements were removed by a filter area down to the water line, which is characterized by = 97, with as the row coordinate of the frame ROI. The result of applying both masks is illustrated in Figure 8. The filter shown in Equation (4) together with the area filter ( ≥ 97) also remove the possible reflexion of the image of the swimmer up to the water line. The ROI of the video frame was then converted to grayscale and subsequently binarised to obtain the black and white image. The binarisation of the image was performed using the algorithm described elsewhere [25]. The subsequent filter area consists of removing all the blobs with an area less than a minimum allowed value (set experimentally to 3000 pixels) and to fill the resulting blob to avoid internal holes. This process is illustrated in Figure 9. The ROI of the video frame was then converted to grayscale and subsequently binarised to obtain the black and white image. The binarisation of the image was performed using the algorithm described elsewhere [25]. The subsequent filter area consists of removing all the blobs with an area less than a minimum allowed value (set experimentally to 3000 pixels) and to fill the resulting blob to avoid internal holes. This process is illustrated in Figure 9. Finally, regionprops function was applied to the black and image to obtain the number of pixels of the remaining area [26]. Figure 10 represents the results for different frames. For illustrative purposes, the final mask was coloured in red over the original frame.  Finally, regionprops function was applied to the black and image to obtain the number of pixels of the remaining area [26]. Figure 10 represents the results for different frames. For illustrative purposes, the final mask was coloured in red over the original frame. Finally, regionprops function was applied to the black and image to obtain the number of pixels of the remaining area [26]. Figure 10 represents the results for different frames. For illustrative purposes, the final mask was coloured in red over the original frame. Using a laptop with a Intel(R) Core(TM) i7-8850h with 16GB RAM and NVIDIA Quadro P2000, the computing time of the proposed algorithm is illustrated in Figure 11 for one of the experiments.  Using a laptop with a Intel(R) Core(TM) i7-8850h with 16 GB RAM and NVIDIA Quadro P2000, the computing time of the proposed algorithm is illustrated in Figure 11 for one of the experiments. Using a laptop with a Intel(R) Core(TM) i7-8850h with 16GB RAM and NVIDIA Quadro P2000, the computing time of the proposed algorithm is illustrated in Figure 11 for one of the experiments.

Extension to Lateral Area Detection
The algorithm described in Figure 5 can be also applied for detecting the lateral area of the swimmer by customizing the masks described in the frontal area determination. Figure 12 shows the final result for lateral area estimation. Note that both frontal and lateral area were only determined in pixels, and no conversion to the real area was performed [27]. Equation (2)

Extension to Lateral Area Detection
The algorithm described in Figure 5 can be also applied for detecting the lateral area of the swimmer by customizing the masks described in the frontal area determination. Figure 12 shows the final result for lateral area estimation. coordinates and image coordinates will be a constant (f.e. one pixel ≈ 6.23 mm ), and therefore, no influence over the goodness of the relationship between effective area and the linear velocity of the swimmer is expected. Nevertheless, further work will include the camera calibration procedure in order to obtain frontal and lateral areas in m 2 .  Note that both frontal and lateral area were only determined in pixels, and no conversion to the real area was performed [27]. Equation (2) is a model of the resistive force where A i should be expressed in m 2 but the influence of the correlation between word coordinates and image coordinates will be a constant (f.e. one pixel ≈ 6.23 mm 2 ), and therefore, no influence over the goodness of the relationship between effective area and the linear velocity of the swimmer is expected. Nevertheless, further work will include the camera calibration procedure in order to obtain frontal and lateral areas in m 2 .     Figures 13 and 14 should be correlated to the velocity of the swimmer to validate or modify model (2).

Frequency Domain Characterization
For illustrative purpose, Figure 16 represents the frequency spectrum of frontal and lateral area for Test I. Results shown in Figures 13 and 14 should be correlated to the velocity of the swimmer to validate or modify model (2). Figure 15 presents the representation of the Frontal Area vs. Lateral Area in both Test (Test I, Figure 15a, Test II, Figure 15b) together with their linear regression. Note that the coefficients of determination are very small (R 2 = 0.1122 and R 2 = 0.09951), and therefore, no correlation between both areas is observed.

Frequency Domain Characterization
For illustrative purpose, Figure 16 represents the frequency spectrum of frontal and lateral area for Test I.

Frequency Domain Characterization
For illustrative purpose, Figure 16 represents the frequency spectrum of frontal and lateral area for Test I.

Frequency Domain Characterization
For illustrative purpose, Figure 16 represents the frequency spectrum of frontal and lateral area for Test I.  The frequency and amplitude values could be used to be correlated to the swimmer speed when the device shown in Figure 17 will be available. The frequency and amplitude values could be used to be correlated to the swimmer speed when the device shown in Figure 17 will be available.

Discussion
This study aimed to determine the frontal area of swimmers employing an automated vision system. The resistive forces that influence the swimmer in the water include form, wave drag (as a result of accelerating the water away from the body), and frictional drag, which are influenced by the swimmer's velocity, boundary layer, shape, size, and the frontal surface area [28], while controversial issues exist due to the difficulty of accurately measuring the wetted surface area of the swimmer's body [29]. We propose a vision-based System for Automated Estimation of the frontal surface area of swimmers, and these instant measurements could allow adapting the technical characteristics for swimmers into the session to improve the swimming performance reducing the passive drag. Our results showed the different time evolution of the frontal area, and the difference with the study of Morais et al. [17] is how the image in our study processes automatically, while these authors manually digitized the calculation of the frontal surface area. Morais et al. compared the passive drag calculation between a single frontal surface area land-based measure and frontal surface area measures obtained at key events during the stroke cycle of front crawl swimming; however, the frontal surface area was obtained at a very low sampling frequency. In addition, the work of Gatta et al. [18] established frontal area values through the swimming stroke cycle for all strokes, trying to estimate active drag (in all strokes) at different swimming speeds, calculating the surface area 2-4 times lower than the recommended sampling frequency range for swimming motion analysis.
In comparison with the aforementioned studies, the novelty of this paper is in establishing an automatic process to obtain the frontal surface area, improving our ability to collect and analyse more complex and accurate data. The main limitations of this research may be summarized as follows: (a) we only analysed one swimmer for collecting and (b) we analysed in one stroke (crawl). Further studies are warranted to develop a new device for installation in a swimming pool (Figure 17).

Conclusions
The development of an automated method to obtain the frontal surface area during swimming increases the knowledge of the temporal fluctuation of the frontal surface area in swimming. It would allow the best monitoring of a swimmer in their swimming training sessions.
A novel algorithm for estimating the frontal and lateral area is presented. The computing time allows to obtain a reasonable online representation of the results.

Discussion
This study aimed to determine the frontal area of swimmers employing an automated vision system. The resistive forces that influence the swimmer in the water include form, wave drag (as a result of accelerating the water away from the body), and frictional drag, which are influenced by the swimmer's velocity, boundary layer, shape, size, and the frontal surface area [28], while controversial issues exist due to the difficulty of accurately measuring the wetted surface area of the swimmer's body [29]. We propose a vision-based System for Automated Estimation of the frontal surface area of swimmers, and these instant measurements could allow adapting the technical characteristics for swimmers into the session to improve the swimming performance reducing the passive drag. Our results showed the different time evolution of the frontal area, and the difference with the study of Morais et al. [17] is how the image in our study processes automatically, while these authors manually digitized the calculation of the frontal surface area. Morais et al. compared the passive drag calculation between a single frontal surface area land-based measure and frontal surface area measures obtained at key events during the stroke cycle of front crawl swimming; however, the frontal surface area was obtained at a very low sampling frequency. In addition, the work of Gatta et al. [18] established frontal area values through the swimming stroke cycle for all strokes, trying to estimate active drag (in all strokes) at different swimming speeds, calculating the surface area 2-4 times lower than the recommended sampling frequency range for swimming motion analysis.
In comparison with the aforementioned studies, the novelty of this paper is in establishing an automatic process to obtain the frontal surface area, improving our ability to collect and analyse more complex and accurate data. The main limitations of this research may be summarized as follows: (a) we only analysed one swimmer for collecting and (b) we analysed in one stroke (crawl). Further studies are warranted to develop a new device for installation in a swimming pool (Figure 17).

Conclusions
The development of an automated method to obtain the frontal surface area during swimming increases the knowledge of the temporal fluctuation of the frontal surface area in swimming. It would allow the best monitoring of a swimmer in their swimming training sessions.
A novel algorithm for estimating the frontal and lateral area is presented. The computing time allows to obtain a reasonable online representation of the results.
The final objective of this research is to experimentally correlate the linear velocity of the swimmer to the frontal area estimated in this work. For this purpose, a novel device has been designed to follow the swimmer during the activity in order to determine the frontal area and the velocity in a synchronous way. Figure 17 shows the conceptual design of the device.
Our purpose is to experimentally validate model (2)  L, a, and b channel can be therefore determined as: where f (t) = and X n , Y n , and Z n are the tristimulus values of the white reference.