A Dataset for Comparing Mirrored and Non-Mirrored Male Bust Images for Facial Recognition

: Facial recognition, as well as other types of human recognition, have found uses in identiﬁcation, security, and learning about behavior, among other uses. Because of the high cost of data collection for training purposes, logistical challenges and other impediments, mirroring images has frequently been used to increase the size of data sets. However, while these larger data sets have shown to be beneﬁcial, their comparative level of beneﬁt to the data collection of similar data has not been assessed. This paper presented a data set collected and prepared for this and related research purposes. The data set included both non-occluded and occluded data for mirroring assessment. Dataset


Summary
Human and facial recognition [1] have a wide variety of prospective uses.The applications of facial recognition include retail store customer identification [2,3], applicant screening [4], and access control [5].Facial and human recognition are also used by law enforcement entities [6].
In addition to simply recognizing an individual, facial imagery can be further analyzed to learn or infer about the subject.Age [7] and gender [7,8] can, in many cases, be easily determined from imagery.It can suggest individuals' interest levels [9] and emotional state [9].
A variety of techniques have been proposed for identification and classification.Neural networks [10] are commonly used for this purpose.Facial and human data also suffers from numerous problems relating to subject state and collection.Thus, a variety of techniques for dealing with a wide number of issues, including lighting conditions, distortion [11,12], and occlusion [11][12][13][14], as well as having only limited training data [15] and subjects' facial expressions [16], have been developed.
One particular use of facial recognition is to identify an individual for security [17][18][19][20] purposes.Recognition can be used to grant access [21,22] or validate identity [18,23].When used for security applications, the system must be robust to attacks that may try to confuse or deny recognition capabilities.
Mirroring has been used extensively as part of recognition and, in particular, facial and human recognition studies.Several studies [24][25][26] have used mirroring to expand a training dataset, including removal of the need to photograph subjects from certain perspectives that correspond to the mirror of other ones [27].Mirroring has also been used to aid recognition by replacing data that is either missing or occluded [28] and to expand a shape model [29] used as part of the recognition process.Studies have

•
Facilitates the testing of the impact of the use of mirrored imagery for testing facial recognition algorithms and for comparing this impact between algorithms.

•
Includes mirrored and non-mirrored data for different subject positions, lighting angles, lighting brightness, and lighting temperature conditions.

•
Includes mirrored and non-mirrored non-occluded data and data with hat and glasses occlusions.

•
The data was collected in a controlled environment with a consistent background.

Data Description
The data set was comprised of 735 images in JPEG format, including 525 manually collected images and 210 images that were mirrored images of the images from the third and fourth subject orientation positions, as shown in Figure 1.
Despite the recurrent use of mirroring for facial and other recognition applications, only limited studies of its effectiveness have been performed.Hu et al. [36] evaluate the use of both flipaugmented and non-augmented data sets, showing a limited benefit to augmentation.Miclut et al. [37] compare non-augmented data sets' effectiveness with mirrored and combined mirrored and rotated augmented data sets.No study was located where the use of augmented data was compared to the use of collected data from the corresponding positions.
This data set provided data that can be used to validate the efficacy of the use of mirroring for training or presenting images for recognition to facial recognition systems.It was designed to facilitate this assessment under a variety of conditions that these systems may face.
To this end, it includes imagery of subjects from multiple perspectives, with multiple lighting conditions and multiple lighting brightness and temperature levels.Most importantly, it includes data for five positions that have been manually collected and two positions that have been generated based on mirroring data from two other manually collected positions.A total of 245 images were included per subject, including 175 collected images and 70 which were generated through mirroring.In addition to these non-occluded images, the dataset includes images where the subject's face is partially occluded by a hat or a pair of glasses.Thus, the data set includes 735 images for each subject, of which 525 are manually collected and 210 are generated through mirroring.
This dataset: • Facilitates the testing of the impact of the use of mirrored imagery for testing facial recognition algorithms and for comparing this impact between algorithms.

•
Includes mirrored and non-mirrored data for different subject positions, lighting angles, lighting brightness, and lighting temperature conditions.

•
Includes mirrored and non-mirrored non-occluded data and data with hat and glasses occlusions.

•
The data was collected in a controlled environment with a consistent background.

Data Description
The data set was comprised of 735 images in JPEG format, including 525 manually collected images and 210 images that were mirrored images of the images from the third and fourth subject orientation positions, as shown in Figure 1.The images were of males between 18 and 26 years old.Many had similar skin tones and facial and upper chest characteristics, to facilitate testing of the systems' ability to correctly identify similar subjects.The JPEG images could be opened and processed by numerous application programs and they contained the original metadata from the camera that was used to collect them.The resolution of the images was 5184 × 3456 pixels.The images were of males between 18 and 26 years old.Many had similar skin tones and facial and upper chest characteristics, to facilitate testing of the systems' ability to correctly identify similar subjects.The JPEG images could be opened and processed by numerous application programs and they contained the original metadata from the camera that was used to collect them.The resolution of the images was 5184 × 3456 pixels.

Data Organization
The organization convention for the data set is as follows.The top-level directories are named with a number that represents the subject, for example, "3".In each of these directories are three subfolders called, "vanilla", "glasses", and "hat"."Vanilla" contains all of the subject's images without them wearing any occlusion, "glasses" contains all of their images with them wearing glasses, and "hat" contains all of their images with them wearing a hat.In each of these directories, are a set of five "position" subdirectories, each of which includes the subject's orientation positions.These are labeled as pos_0 to pos_4.In addition, the two mirrored positions are labeled as m_pos_3 and m_pos_4.In each position directory, there are lighting angle subdirectories which correspond to each of the light's positions.These are labeled as angle_1 to angle_7.Finally, in each lighting subdirectory, there are five images of the subject labeled in the format of IMG_####.JPG in the following order: warm, cold, low, medium, and high.In the mirrored directories, the naming format is ###_IMG_####.JPG; however, the ordering still corresponds to warm, cold, low, medium, and high.The labels of the individual images (warm, cold, low, medium, and high) correspond to the lighting settings, which are discussed in detail in the following section.

Comparison to Other Data Sets
A variety of other facial recognition datasets exist.Learned-Miller et al. [39] created a "Labeled Faces in the Wild" data set that was harvested from websites.This data set included data for 5749 people; however, it only included approximately two images per individual (13,000 images in total).Guo et al. [40] also harvested images from the web for their dataset, "MS-Celeb-1M."This data set is comprised of 10,000,000 images of 100,000 people.Both of these data sets were collected automatically and could potentially have duplicates that were not detected by the creators' algorithms, incorrect association of images to a single individual, or other characteristics associated with the curating algorithms.Another data set, entitled "IARPA Janus Benchmark A" [41] has images of 500 people, but only an average of 11.4 images per individual.The "Pgu-Face" dataset [42], similarly, had 896 images covering 224 subjects.The "Extended Cohn-Kanade Dataset (CK+)" dataset [43] had slightly more images per individual, covering 210 individuals with only 23 images per subject.The "FRGC 1.0.4"dataset [44] includes 152 subjects, but only an average of five images per individual.The "Face Recognition Technology (FERET)" data set [45] includes 14,126 images covering 1199 subjects (for about 12 images per subject).Finally, the "ORL Database of Faces" [46] contains only 400 images covering 40 individuals with only 10 images per subject.
The dataset presented herein covers only 11 subjects (which is less than other datasets); however, it includes 735 images per subject, which is far more than any of the other data sets.In terms of the total number of images, the dataset described herein has 8085 images, making it the fourth largest of the datasets.Two of these datasets were automatically generated ("Labeled Faces in the Wild" and "MS-Celeb-1M"), and it is thus the second largest manually collected dataset, from those surveyed (the NIST FERET data set is larger, being comprised of 14,126 images).

Methods
The image data set was collected using a Cannon EOS Rebel T2i camera with a 60 mm Ultrasonic EFS lens and set in aperture priority mode with an ISO of 800 and an FSTOP of eight.AI focus, peripheral illumination correction, red eye reduction, auto white balance, and spot metering were enabled, and auto exposure bracketing was set at 01.The flash was disabled.The camera was oriented in landscape mode.
Two Neewer LED500LRC lights were used to illuminate the background and one Yongnuo YN600L light was used to illuminate the subject.A projector screen was used as the backdrop.The camera, subject, and lighting locations are shown in Figure 2. The non-numbered 'X' locations are the positions of the background lights.The camera and subject positions are identified by an image of a camera and human, respectively.The numbered 'X' locations correspond to the seven subject illumination lighting positions (which create the seven lighting angles).The exact coordinates of the subject, camera, and lights are provided in Table 1.
Data 2019, 4, 26 4 of 10 illumination lighting positions (which create the seven lighting angles).The exact coordinates of the subject, camera, and lights are provided in Table 1.The subjects were photographed in five different positions (as shown in Figure 3).Additionally, the data was processed to mirror the position 0 and 1 images (the two leftmost images in Figure 3), to replace the data collected at positions 3 and 4. Figure 4 depicts the mirrored images shown in positions 3 and 4 (the two rightmost images), along with the collected data in positions 0, 1, and 2.   The subjects were photographed in five different positions (as shown in Figure 3).Additionally, the data was processed to mirror the position 0 and 1 images (the two leftmost images in Figure 3), to replace the data collected at positions 3 and 4. Figure 4 depicts the mirrored images shown in positions 3 and 4 (the two rightmost images), along with the collected data in positions 0, 1, and 2.   The subjects were photographed in five different positions (as shown in Figure 3).Additionally, the data was processed to mirror the position 0 and 1 images (the two leftmost images in Figure 3), to replace the data collected at positions 3 and 4. Figure 4 depicts the mirrored images shown in positions 3 and 4 (the two rightmost images), along with the collected data in positions 0, 1, and 2.   For each of the five positions (including both the mirrored and non-mirrored ones), photos were taken with seven different lighting angles.The seven angles of light are depicted in Figure 2, and images demonstrating the effect of these lighting angles are presented in Figure 5.For each position and lighting angle, images were collected at five different lighting settings.These lighting settings are listed in Table 2 and are visually depicted in Figure 6.Table 2. Lighting configurations and their titles [38].

Configuration Title Settings
Warm 60% brightness on warm (3200 k) Cold 60% brightness on cold (5500 k) Low 10% brightness on warm (3200 k) and 10% brightness on cold (5500 k) Medium 40% brightness on warm (3200 k) and 40% on brightness on cold (5500 k) High 70% brightness on warm (3200 k) and 70% brightness on cold (5500 k) The images that are used in mirrored positions 3 and 4 were created through a pixel-by-pixel replacement from the left of the image to the right and vice versa.Figure 7 depicts this mirroring for the forward-facing position.Note that both images look similar, but are distinguishable by comparing small features, such as looking at the patterns of hair on the left and right of the head.For each of the five positions (including both the mirrored and non-mirrored ones), photos were taken with seven different lighting angles.The seven angles of light are depicted in Figure 2, and images demonstrating the effect of these lighting angles are presented in Figure 5.For each position and lighting angle, images were collected at five different lighting settings.These lighting settings are listed in Table 2 and are visually depicted in Figure 6.For each of the five positions (including both the mirrored and non-mirrored ones), photos were taken with seven different lighting angles.The seven angles of light are depicted in Figure 2, and images demonstrating the effect of these lighting angles are presented in Figure 5.For each position and lighting angle, images were collected at five different lighting settings.These lighting settings are listed in Table 2 and are visually depicted in Figure 6.Table 2. Lighting configurations and their titles [38].

Configuration Title Settings
Warm 60% brightness on warm (3200 k) Cold 60% brightness on cold (5500 k) Low 10% brightness on warm (3200 k) and 10% brightness on cold (5500 k) Medium 40% brightness on warm (3200 k) and 40% on brightness on cold (5500 k) High 70% brightness on warm (3200 k) and 70% brightness on cold (5500 k) The images that are used in mirrored positions 3 and 4 were created through a pixel-by-pixel replacement from the left of the image to the right and vice versa.Figure 7 depicts this mirroring for the forward-facing position.Note that both images look similar, but are distinguishable by comparing small features, such as looking at the patterns of hair on the left and right of the head.Table 2. Lighting configurations and their titles [38].

Configuration Title Settings
Warm 60% brightness on warm (3200 k) Cold 60% brightness on cold (5500 k) Low 10% brightness on warm (3200 k) and 10% brightness on cold (5500 k) Medium 40% brightness on warm (3200 k) and 40% on brightness on cold (5500 k) High 70% brightness on warm (3200 k) and 70% brightness on cold (5500 k)  For each of the five positions (including both the mirrored and non-mirrored ones), photos were taken with seven different lighting angles.The seven angles of light are depicted in Figure 2, and images demonstrating the effect of these lighting angles are presented in Figure 5.For each position and lighting angle, images were collected at five different lighting settings.These lighting settings are listed in Table 2 and are visually depicted in Figure 6.Table 2. Lighting configurations and their titles [38].

Configuration Title Settings
Warm 60% brightness on warm (3200 k) Cold 60% brightness on cold (5500 k) Low 10% brightness on warm (3200 k) and 10% brightness on cold (5500 k) Medium 40% brightness on warm (3200 k) and 40% on brightness on cold (5500 k) High 70% brightness on warm (3200 k) and 70% brightness on cold (5500 k) The images that are used in mirrored positions 3 and 4 were created through a pixel-by-pixel replacement from the left of the image to the right and vice versa.Figure 7 depicts this mirroring for the forward-facing position.Note that both images look similar, but are distinguishable by comparing small features, such as looking at the patterns of hair on the left and right of the head.The images that are used in mirrored positions 3 and 4 were created through a pixel-by-pixel replacement from the left of the image to the right and vice versa.Figure 7 depicts this mirroring for the forward-facing position.Note that both images look similar, but are distinguishable by comparing small features, such as looking at the patterns of hair on the left and right of the head.For each of the five positions (including both the mirrored and non-mirrored ones), photos were taken with seven different lighting angles.The seven angles of light are depicted in Figure 2, and images demonstrating the effect of these lighting angles are presented in Figure 5.For each position and lighting angle, images were collected at five different lighting settings.These lighting settings are listed in Table 2 and are visually depicted in Figure 6.Table 2. Lighting configurations and their titles [38].

Configuration Title Settings
Warm 60% brightness on warm (3200 k) Cold 60% brightness on cold (5500 k) Low 10% brightness on warm (3200 k) and 10% brightness on cold (5500 k) Medium 40% brightness on warm (3200 k) and 40% on brightness on cold (5500 k) High 70% brightness on warm (3200 k) and 70% brightness on cold (5500 k) The images that are used in mirrored positions 3 and 4 were created through a pixel-by-pixel replacement from the left of the image to the right and vice versa.Figure 7 depicts this mirroring for the forward-facing position.Note that both images look similar, but are distinguishable by comparing small features, such as looking at the patterns of hair on the left and right of the head.However, the mirrored images could not just be directly swapped.The lighting angles had to be reversed to most closely replicate or approximate the collected images, both in terms of subject position and in terms of the incident angle of the lighting.For example, to generate the mirrored image for subject position 4, lighting angle 7, the image from subject position 0, lighting angle 1 was used and flipped horizontally.

Mirrored Captured
Even with this, the mirrored images for the subject positions 3 and 4, in this dataset, do not have exactly the same lighting angles of incidence, as the lighting configuration of the experimental setup was not perfectly symmetrical.Table 3 lists the angle of incidence for all the lighting and position combinations (including both mirrored and non-mirrored configurations).Figures 8 and 9 depict this visually for positions 3 and 4, respectively.However, the mirrored images could not just be directly swapped.The lighting angles had to be reversed to most closely replicate or approximate the collected images, both in terms of subject position and in terms of the incident angle of the lighting.For example, to generate the mirrored image for subject position 4, lighting angle 7, the image from subject position 0, lighting angle 1 was used and flipped horizontally.
Even with this, the mirrored images for the subject positions 3 and 4, in this dataset, do not have exactly the same lighting angles of incidence, the lighting configuration of the experimental setup was not perfectly symmetrical.Table 3 lists the angle of incidence for all the lighting and position combinations (including both mirrored and non-mirrored configurations).Figures 8 and 9  Mirroring images collected from subject positions 0 and 1, to substitute in place of images collected from the remaining subject positions could be used to offset the image asymmetry created by the lighting placement asymmetry.However, the combination of the mirrored and non-mirrored data produces a greater set of lighting incidence angles than either full collection or half-collection and mirroring alone.Note also that some lighting angles are the same or very similar, while other mirroring and lighting angle changes, both together and independently, by looking at the impact relative to positions with more and less similar angles of incidence between the collected and mirrored data.Subjects were asked to keep a neutral face while being photographed, and to face forward relative to their body orientation.The individual rotated to each position and the Yongnuo YN600L light was moved between positions and had its settings altered, while the subject remained stationary.Images were reviewed after each set to ensure that the subject did not move, blink, or show expression and to verify that all required images were present and of suitable quality.Subjects' attire (shirt) was not controlled and there were minor fluctuations in the subjects' position and facial state.All other aspects of data collection were controlled.Mirroring images collected from subject positions 0 and 1, to substitute in place of images collected from the remaining subject positions could be used to offset the image asymmetry created by the lighting placement asymmetry.However, the combination of the mirrored and non-mirrored data produces a greater set of lighting incidence angles than either full collection or half-collection and mirroring alone.Note also that some lighting angles are the same or very similar, while other lighting angles are further removed.Because of this, the data set can be used to assess the impact of mirroring and lighting angle changes, both together and independently, by looking at the impact relative to positions with more and less similar angles of incidence between the collected and mirrored data.
Subjects were asked to keep a neutral face while being photographed, and to face forward relative to their body orientation.The individual rotated to each position and the Yongnuo YN600L light was moved between positions and had its settings altered, while the subject remained stationary.Images were reviewed after each set to ensure that the subject did not move, blink, or show expression and to verify that all required images were present and of suitable quality.Subjects' attire (shirt) was not controlled and there were minor fluctuations in the subjects' position and facial state.All other aspects of data collection were controlled.

Figure 1 .
Figure 1.Orientation of subjects as described in Reference [38].

Figure 1 .
Figure 1.Orientation of subjects as described in Reference [38].

Figure 3 .
Figure 3. Subjects were photographed in five different positions.

Figure 3 .
Figure 3. Subjects were photographed in five different positions.Figure 3. Subjects were photographed in five different positions.

Figure 3 .
Figure 3. Subjects were photographed in five different positions.Figure 3. Subjects were photographed in five different positions.

Figure 4 .
Figure 4. Photos of three different positions (left three images) with two mirrored images (right two images).

Figure 5 .
Figure 5. Photos were taken with seven different lighting angles at each position.

Figure 6 .
Figure 6.Five different lighting settings were used for each subject position and lighting angle.From left to right, shown are: warm, cold, low, medium, and high.

Figure 4 .
Figure 4. Photos of three different positions (left three images) with two mirrored images (right two images).

Figure 4 .
Figure 4. Photos of three different positions (left three images) with two mirrored images (right two images).

Figure 5 .
Figure 5. Photos were taken with seven different lighting angles at each position.

Figure 6 .
Figure 6.Five different lighting settings were used for each subject position and lighting angle.From left to right, shown are: warm, cold, low, medium, and high.

Figure 5 .
Figure 5. Photos were taken with seven different lighting angles at each position.

Figure 4 .
Figure 4. Photos of three different positions (left three images) with two mirrored images (right two images).

Figure 5 .
Figure 5. Photos were taken with seven different lighting angles at each position.

Figure 6 .
Figure 6.Five different lighting settings were used for each subject position and lighting angle.From left to right, shown are: warm, cold, low, medium, and high.

Figure 6 .
Figure 6.Five different lighting settings were used for each subject position and lighting angle.From left to right, shown are: warm, cold, low, medium, and high.

Figure 4 .
Figure 4. Photos of three different positions (left three images) with two mirrored images (right two images).

Figure 5 .
Figure 5. Photos were taken with seven different lighting angles at each position.

Figure 6 .
Figure 6.Five different lighting settings were used for each subject position and lighting angle.From left to right, shown are: warm, cold, low, medium, and high.

Figure 7 .
Figure 7. Cropped mirrored and captured images from subject position 2 (center).

Figure 7 .
Figure 7. Cropped mirrored and captured images from subject position 2 (center).

Figure 8 .
Figure 8.Comparison of the mirrored and collected images for subject position 3 at lighting angles 2, 4, and 7.

Figure 8 .
Figure 8.Comparison of the mirrored and collected images for subject position 3 at lighting angles 2, 4, and 7.

Figure 9 .
Figure 9.Comparison of mirrored and collected images for subject position 4 at lighting angles 1, 4, and 6.

Figure 9 .
Figure 9.Comparison of mirrored and collected images for subject position 4 at lighting angles 1, 4, and 6.
Data 2019, 4, 26 4 of 10 illumination lighting positions (which create the seven lighting angles).The exact coordinates of the subject, camera, and lights are provided in Table 1.

Table 3 .
Lighting angle (in degrees) for subject positions and lighting angles, where 3 and 4 refer to the mirrored images.

Table 3 .
Lighting angle (in degrees) for subject positions and lighting angles, where 3′ and 4′ refer to the mirrored images.