Mechatronic Design and Experimental Research of an Automated Photogrammetry-Based Human Body Scanner
Abstract
:1. Introduction
- the scanned person remains stationary and the device performs a series of measurements using a number of sensors permanently mounted in the space around the person,
- the scanned person is turned on a rotating platform and body measurements are taken from one side with the use of several sensors,
- the scanned person stands on a stationary platform, and the measurement is carried out using several sensors mounted on one or more arms rotating around the platform.
- limiting the number of sensors used for scanning, in relation to solutions in which they are mounted in fixed poses in relation to the scanned object, which should allow for a lower cost of the system production,
- scanning without rotating the person thanks to the scanner solution with a fixed platform and a mast rotating around it,
- ensuring the automation of the scanning process in relation to handheld scanners, which require the involvement of an additional person.
- How may the scanning accuracy be assessed?
- How to compare a 3D model with a higher-accuracy reference model?
- What is the reproducibility of the results in laboratory tests using a measuring dummy?
- What is the correlation between quality rates and the accuracy of body reconstruction at the stage of laboratory tests and in real conditions with the participation of humans?
- What are the differences and issues with scanning dummies and humans?
- What is the correlation between the results of quantitative and qualitative research?
- What are the results for different strategies in terms of camera positioning relative to the mast and the number of cameras used on the mast?
- What results are obtained depending on the height of the scanned person?
- What are the limitations of the photogrammetry in relation to scanning humans?
2. State of the Art Review
3. HUBO Scanning System
3.1. Design Assumptions
3.2. Mechanical Design of the Scanner
3.3. Mechatronic Architecture of the Scanner
3.4. Software Architecture of the HUBO Scanning System
3.5. Scanner Operation and User’s Mobile Application
3.6. Innovative Features of the HUBO Scanning System
- automation of the object-scanning process, enabling continuous operation of the scanner while reducing the necessary service by staff, or even eliminating it;
- greater scanning comfort for the user due to the fact that the platform does not rotate;
- enabling the configuration of the working space of the device for the assumed dimensions of the object;
- quick assembly and disassembly of the device for the purpose of its transport;
- a more energy-saving solution compared to a similar system with a rotating platform;
- lighting up the scanned object;
- providing access only to authorized users;
- ensuring the configuration of device parameters by the user in terms of scanning time and the resulting accuracy of the model;
- providing the ability of voice control of the scanner and receive voice notifications about its work;
- remote configuration and diagnostics of the scanner;
- processing in cloud computing to accelerate measurement data processing and obtain a 3D model faster.
4. Laboratory Research Using Dummies
4.1. Methodology of the Research
- A.
- selection of optimal geometrical parameters of the scanner;
- B.
- selection of the optimal distribution of cameras or other sensors and lamps;
- C.
- selection of optimal scanning parameters;
- D.
- selection of optimal settings for sensors and lamps;
- E.
- selection of optimal data-processing algorithms and their parameters.
4.2. Quantitative Research Using Reference 3D Model
- the measurement error is the distance of the point (for ) from the surface of the real object ;
- the measurement error for the reference 3D model at point is ;
- the measurement errors for the tested 3D model obtained from the HUBO scanner at the nearby points , and are , and , respectively;
- in a typical case , and , where .
- parameters of the general equation of the plane , that is, , defined by the points , and :
- estimated measurement error determined as a distance of point from the plane , that is from point to point :
4.3. Qualitative Research Using Realistic Dummies
- (a)
- the vertical distribution of camera viewpoints (on the z axis) which covered the variants: a1—even distribution of these points along the object height, a2—distribution of the viewpoints adjusted to the specificity of the figure;
- (b)
- the pitch angles of cameras, which were set so that for different variants there was a different number of viewpoints common for two cameras (one common viewpoint for variant b1 and three common viewpoints for variant b2); the distribution of viewpoints adjusted to the specificity of the figure was used for all variants in this group;
- (c)
- varied number of camera positions around the scanned object n, for the best variants of camera poses relative to the mast (variants from groups a and b),
- (d)
- varied number of camera positions along the mast k, where the starting points were the best strategies analyzed in the variants from groups a and b.
5. Experimental Studies Involving Humans
6. Conclusions and Directions of Further Works
- The developed and verified solution allowed the identified problems occurring in other solutions to be overcome. In particular, it enables the reduction of the number of sensors necessary for scanning in relation to systems with permanently installed sensors, which results in a much lower solution cost. The disadvantage of many solutions of a similar class related to rotating a person was also eliminated. In addition, it allows one to automate the scanning process, eliminating the need to engage an additional person to operate it.
- The developed scanner solution meets all design assumptions, including the maximum height of the scanned person, scanning time, and the accuracy of reconstruction. The user’s mobile application allows the user to perform self-scanning. The configuration of the scanner can adjust to the height of the scanned person.
- The selection of appropriate scanning and data-processing parameters is a multi-criteria optimization problem. In this work, the focus was on the issue of selecting the optimal parameters of the scanning system as part of research using realistic dummies and with the participation of humans, taking into account their different heights. The proposed methodology for selecting the distribution and parameters of sensors can also be used for other scanning systems of a similar class.
- The article presents a method of evaluating the accuracy of reconstruction using a high-accuracy reference 3D model and based on selected measurement points. Since the accuracy assessment is independent of the scanner design and scanning technique, this approach can be successfully applied to any scanning system.
- Within the laboratory research using dummies and the experimental studies involving humans, a number of research problems that are within the scope of this work were also analyzed. As a result of quantitative and qualitative research, answers to key research questions were obtained and limitations of the photogrammetry technique in the application to scanning people were identified. The analyzed research issues are so general that research related to them can be conducted for various scanning systems. Moreover, most of them can be performed for the photogrammetry and other scanning techniques as well.
- The RMSE parameter was used to quantitatively assess the accuracy of the reconstruction in experimental studies, which can also be used in investigations of other scanners based on the photogrammetry technique.
- As a result of the research, it was noticed, among other things, that the photogrammetry technique does not cope well with objects that have few features, where many similar features are next to each other, or where there are shiny surfaces. Moreover, there are inaccuracies in the reconstruction in the case of blurred or overexposed photographs, which make it difficult to correctly find the features of the object.
- However, if reconstruction errors are encountered, one can try to combine the results for different processing parameters, also applying this treatment in an automatic way.
- As a result of experimental research, satisfactory results of reconstruction of human figure were obtained, in the cases of both scanning dummies and real people.
- Scanning people is more problematic than scanning dummies due to the occurrence of involuntary body movements that might decrease the accuracy of the reconstruction.
- Considering the achieved functionalities, the HUBO scanning system can already be used in selected industries, e.g., modeling.
- The use of other types of sensors, such as depth cameras and related data-processing methods, which will enable processing the point cloud right away.
- The development of data-processing technique alternatives to photogrammetry, e.g., NeRF, which will enhance the accuracy of reconstruction and overcome the limitations of the photogrammetry technique related to object features, including shiny surfaces.
- The application of artificial intelligence methods to improve the accuracy of reconstruction, including the completion of missing fragments and smoothing of surfaces.
- Development of methods for determining anthropometric parameters of the human figure for use in various industries, e.g., clothing, fitness and medical.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- 3D Scanner Catalog—Over 500 3D Scanners to Choose from. 2022. Available online: https://www.aniwaa.com/catalog/3d-scanners (accessed on 14 February 2023).
- Lansard, M. The 6 Best 3D Body Scanners in 2022. 2022. Available online: https://www.aniwaa.com/buyers-guide/3d-scanners/best-3d-body-scanners/ (accessed on 16 April 2023).
- Lee, H.; Xu, Y. Influence of motivational orientations on consumers’ adoption of virtual fitting rooms (VFRs): Moderating effects of fashion leadership and technology visibility. Int. J. Fash. Des. Technol. Educ. 2022, 15, 297–307. [Google Scholar] [CrossRef]
- Schindler, K. Mathematical foundations of photogrammetry. In Handbook of Geomathematics; Springer: Berlin/Heidelberg, Germany, 2015; pp. 3087–3103. [Google Scholar]
- Ernst, M.; Christophersen, A.; Sanduloff, S. Online Shopping Featuring My Customized Avatar—Generating Customized Avatars for a Sustainable Shopping Experience in E-Commerce. In Proceedings of the 12th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, 3DBODY.TECH 2021, Lugano, Switzerland, 19–20 October 2021. [Google Scholar]
- Barreto, M.A.; Perez-Gonzalez, J.; Herr, H.M.; Huegel, J.C. ARACAM: A RGB-D Multi-View Photogrammetry System for Lower Limb 3D Reconstruction Applications. Sensors 2022, 22, 2443. [Google Scholar] [CrossRef]
- Grazioso, S.; Caporaso, T.; Selvaggio, M.; Panariello, D.; Ruggiero, R.; Di Gironimo, G. Using photogrammetric 3D body reconstruction for the design of patient–tailored assistive devices. In Proceedings of the 2019 II Workshop on Metrology for Industry 4.0 and IoT (MetroInd4. 0&IoT), Naples, Italy, 4–6 June 2019; pp. 240–242. [Google Scholar]
- Hussien, D.A.; Abed, F.M.; Hasan, A.A. Stereo Photogrammetry vs Computed Tomography for 3D Medical Measurements. Karbala Int. J. Mod. Sci. 2019, 5, 2. [Google Scholar] [CrossRef] [Green Version]
- Dilian, O.; Kimmel, R.; Tezmah-Shahar, R.; Agmon, M. Can We Quantify Aging-Associated Postural Changes Using Photogrammetry? A Systematic Review. Sensors 2022, 22, 6640. [Google Scholar] [CrossRef]
- Roy, S.; Grunwald, A.T.D.; Alves-Pinto, A.; Maier, R.; Cremers, D.; Pfeiffer, D.; Lampe, R. A Noninvasive 3D Body Scanner and Software Tool towards Analysis of Scoliosis. Biomed Res. Int. 2019, 2019, 4715720. [Google Scholar] [CrossRef] [Green Version]
- Petriceks, A.H.; Peterson, A.S.; Angeles, M.; Brown, W.P.; Srivastava, S. Photogrammetry of Human Specimens: An Innovation in Anatomy Education. J. Med. Educ. Curric. Dev. 2018, 5, 1–10. [Google Scholar] [CrossRef] [Green Version]
- Burk, Z.; Johnson, C. Method For Production of 3D interactive Models Using Photogrammetry For Use in Human Anatomy Education. HAPS Educ. 2019, 23, 457–463. [Google Scholar] [CrossRef]
- Naked—The World’s First Home Body Scanner. Available online: https://nakedlabs.com (accessed on 14 November 2022).
- How Does Styku Predict Body Fat? Available online: https://www.styku.com/help/search/how-does-styku-calculate-body-fat (accessed on 16 April 2023).
- Silver, B.; Wilson, P.B. Reliability and Minimal Detectable Change of the Styku 3D Body Scanner. Meas. Phys. Educ. Exerc. Sci. 2020, 24, 165–171. [Google Scholar] [CrossRef]
- Bartol, K.; Bojanić, D.; Petković, T.; Pribanić, T. A review of body measurement using 3D scanning. IEEE Access 2021, 9, 67281–67301. [Google Scholar] [CrossRef]
- Dixit, I.; Kennedy, S.; Piemontesi, J.; Kennedy, B.; Krebs, C. Which Tool Is Best: 3D Scanning or Photogrammetry—It Depends on the Task. Adv. Exp. Med. Biol. 2019, 1120, 107–119. [Google Scholar] [CrossRef]
- Stark, E.; Haffner, O.; Kučera, E. Low-Cost Method for 3D Body Measurement Based on Photogrammetry Using Smartphone. Electronics 2022, 11, 1048. [Google Scholar] [CrossRef]
- Do, P.N.B.; Nguyen, Q.C. A Review of Stereo-Photogrammetry Method for 3-D Reconstruction. In Proceedings of the 2019 19th International Symposium on Communications and Information Technologies (ISCIT), Ho Chi Minh City, Vietnam, 25–27 September 2019; pp. 138–143. [Google Scholar]
- Jiménez-Jiménez, S.I.; Ojeda-Bustamante, W.; Marcial-Pablo, M.; Enciso, J. Digital Terrain Models Generated with Low-Cost UAV Photogrammetry: Methodology and Accuracy. ISPRS Int. J. Geo-Inf. 2021, 10, 285. [Google Scholar] [CrossRef]
- Setiyadi, S.; Mukhtar, H.; Cahyadi, W.A. A Comparative Study of Affordable Photogrammetry Software for Reconstructing 3D Model of a Human Foot. In Proceedings of the 2021 IEEE 7th International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), Bandung, Indonesia, 23–25 August 2021; pp. 286–291. [Google Scholar] [CrossRef]
- Boe, T.; Carter, C. Human Photogrammetry: Foundational Techniques for Creative Practitioners. Int. J. Comput. Graph. Animat. 2020, 10, 1–20. [Google Scholar] [CrossRef]
- Zeraatkar, M.; Khalili, K. A Fast and Low-Cost Human Body 3D Scanner Using 100 Cameras. J. Imaging 2020, 6, 21. [Google Scholar] [CrossRef] [Green Version]
- Kottner, S.; Schulz, M.M.; Berger, F.; Thali, M.; Gascho, D. Beyond the visible spectrum—applying 3D multispectral full-body imaging to the VirtoScan system. Forensic Sci. Med. Pathol. 2021, 17, 565–576. [Google Scholar] [CrossRef]
- myeggO|The 3D Digital Identity Generator. Available online: https://myeggo.com/generator (accessed on 27 December 2022).
- Freeman Gebler, O.; Goudswaard, M.; Hicks, B.; Jones, D.; Nassehi, A.; Snider, C.; Yon, J. A Comparison of Structured Light Scanning and Photogrammetry for the Digitisation of Physical Prototypes. Proc. Des. Soc. 2021, 1, 11–20. [Google Scholar] [CrossRef]
- Zheng, X.; Ma, R.; Gao, R.; Hao, Q. Phase-SLAM: Mobile Structured Light Illumination for Full Body 3D Scanning. In Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–1 October 2021; pp. 1617–1624. [Google Scholar] [CrossRef]
- Nhung, L.T.H.; Tu, N.N.; Trinh, N.N.; Hieu, P.M.; Linh, N.T.D.; Le, N.T. Development of 3D Breast Measurement System Using Structured Light for Bra Design. In Intelligent Systems and Networks: Selected Articles from ICISN 2021; Springer: Singapore, 2021; Volume 243, pp. 373–385. [Google Scholar] [CrossRef]
- Alshawabkeh, Y.; Baik, A.; Miky, Y. Integration of Laser Scanner and Photogrammetry for Heritage BIM Enhancement. ISPRS Int. J. Geo-Inf. 2021, 10, 316. [Google Scholar] [CrossRef]
- Kudela, P.; Palčák, M.; Zábovská, K.; Bučko, B. Integration of photogrammetry within laser scanning approach. In Proceedings of the 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 6 November 2022; pp. 1691–1694. [Google Scholar]
- Karami, A.; Menna, F.; Remondino, F. Combining Photogrammetry and Photometric Stereo to Achieve Precise and Complete 3D Reconstruction. Sensors 2022, 22, 8172. [Google Scholar] [CrossRef]
- Villa, C.; Flies, M.J.; Jacobsen, C. Forensic 3D documentation of bodies—combining CT scanning photogrammetry. J. Forensic Radiol. Imaging 2018, 12, e2–e7. [Google Scholar] [CrossRef]
- Koval, A. Quantitative Comparison of Manual vs. 3D Scanner Human Body Measurements. In Proceedings of the 11th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, 3DBODY.TECH 2020, Online, 17–18 November 2020. [Google Scholar]
- Rumbo-Rodriguez, L.; Sanchez-SanSegundo, M.; Ferrer-Cascales, R.; Garcia-D’Urso, N.; Hurtado-Sanchez, J.A.; Zaragoza-Marti, A. Comparison of Body Scanner and Manual Anthropometric Measurements of Body Shape: A Systematic Review. Int. J. Environ. Res. Public Health 2021, 18, 6213. [Google Scholar] [CrossRef]
- Strunevich, E.; Detering-Koll, U.; Quattelbaum, B. Investigation of usability and measurement accuracy of 3D body scanning mobile applications. Commun. Dev. Assem. Text. Prod. 2020, 1, 130–140. [Google Scholar] [CrossRef]
- McDonald, C.; Oviedo, L.; Ballester, A. Working Group Progress For IEEE P3141—Standard for 3D Body Processing, 2019–2020. In Proceedings of the 11th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, 3DBODY.TECH 2020, Online, 17–18 November 2020. [Google Scholar]
- AliceVision|Photogrammetric Computer Vision Framework. Available online: https://alicevision.org (accessed on 14 November 2022).
- Trojnacki, M.; Dąbek, P.; Jaroszek, P. Analysis of the Influence of the Geometrical Parameters of the Body Scanner on the Accuracy of Reconstruction of the Human Figure Using the Photogrammetry Technique. Sensors 2022, 22, 9181. [Google Scholar] [CrossRef]
- Evaluation of Measurement Data—Guide to the Expression of Uncertainty in Measurement; International Organization for Standardization: Geneva, Switzerland, 2008; Volume 50, p. 134.
Object | Male Dummy | Child Dummy | ||||
---|---|---|---|---|---|---|
Variant | b1m | d2m | e1m | b1c | d2c | e1c |
k [-] | 8 | 10 | 8 | 6 | 7 | 6 |
n [-] | 32 | 32 | 32 | 32 | 32 | 32 |
p [-] | 1 | 1 | 2 | 1 | 1 | 2 |
c [-] | 256 | 320 | 512 | 192 | 224 | 384 |
RMSE [px] | 0.796 | 0.867 | 0.944 | 0.860 | 0.847 | 0.980 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Trojnacki, M.; Dąbek, P.; Jaroszek, P. Mechatronic Design and Experimental Research of an Automated Photogrammetry-Based Human Body Scanner. Sensors 2023, 23, 5840. https://doi.org/10.3390/s23135840
Trojnacki M, Dąbek P, Jaroszek P. Mechatronic Design and Experimental Research of an Automated Photogrammetry-Based Human Body Scanner. Sensors. 2023; 23(13):5840. https://doi.org/10.3390/s23135840
Chicago/Turabian StyleTrojnacki, Maciej, Przemysław Dąbek, and Piotr Jaroszek. 2023. "Mechatronic Design and Experimental Research of an Automated Photogrammetry-Based Human Body Scanner" Sensors 23, no. 13: 5840. https://doi.org/10.3390/s23135840
APA StyleTrojnacki, M., Dąbek, P., & Jaroszek, P. (2023). Mechatronic Design and Experimental Research of an Automated Photogrammetry-Based Human Body Scanner. Sensors, 23(13), 5840. https://doi.org/10.3390/s23135840