To ensure the future competitiveness of manufacturing companies, it is necessary to face constantly changing customer requirements and market turbulence. Consequently, a shortening of the product development cycle is constantly pursued in order to adapt to the dynamic market efficiently and quickly [1
]. Manufacturers today tend to adopt product differentiation strategies and more customer-centric approaches to remain competitive. Hence, mass customization and product diversification are one of the most commonly implemented business models. As part of the digital transformation and Industry 4.0, these objectives are being realized. Additive manufacturing technologies offer high potential for individualization at low cost in a short period of time [2
]. Cost-effective and high-quality mass customization requires technological resources such as reverse engineering and 3D scanning. Currently, smartphones and tablets can be utilized as 3D scanners as well [3
]. This lowers the entry barrier for both private users and industrial users to digitize objects. While the customer only needs to be provided with the hardware, the manufacturer can realize the product design by means of reverse engineering (RE). This means that the basic scanning can be performed by customers themselves so that the manufacturer can offer them a product tailored to their needs. An example for this is the business model of the company HEXR, which produces custom-fit bike helmets with additive manufacturing. In order to scan the shape of the customers’ heads, a smartphone application and a fitting cap are needed. The application uses the standard smartphone camera and the random texture of the fitting cap to model the shape of the head [4
]. However, with LiDAR and TrueDepth technologies included in the newest devices by Apple, even more enhanced possibilities to digitize real objects are offered.
] defines the processes of designing, manufacturing, assembling and maintaining products and systems as engineering and divides it into two different types. While forward engineering implies the process of the approach from a highly abstracted idea to the physical implementation of a system, RE refers to already existing objects. Thereby, an existing part, subassembly or product without drawings and documentations is duplicated. Furthermore, RE is a process to obtain a 3-dimensional computer model from an existing object through measurements without consideration of functionalities. This process is also known as Computer Aided Reverse Engineering (CARE), which involves the steps of data collection, mesh construction and surface fitting [6
]. For data collection, different hardware can be used, whereas mesh construction and surface fitting focus mainly on software solutions. The hardware is divided into noncontact and contact-based methods, which can be used for data acquisition [7
When the highest scan accuracy is required, coordinate measuring machines (CMM) can be used. These are based on the principle of contact by measuring the surface to be tested with a probe. Due to the high precision manufacturing processes and built-in reference standards, CMMs provide the highest accuracy in inspection processes [6
]. In addition, the results are independent of the reflection rate of a surface. Limited operating space is a point of emphasis because the object needs to be accessed and touched by the probe. The amount of contact points determines the accuracy. Therefore, the process can get time-consuming by inspecting complex structures as well as unknown shapes [6
An alternative to this procedure is the noncontact method, in which no physical contact to the surface is needed. Noncontact methods are subdivided into different scan technologies including photogrammetry, structured light and Time of Flight (ToF). Those techniques are used by a variety of scanning systems with different capabilities and limitations [7
]. Based on the field of application, it is important to select a suitable scanner that meets the requirements. Besides a relatively short processing time, noncontact 3D scanners cost much less compared to a CMM [8
]. In addition, a high point cloud density of some 3D scanners can be used to measure complex and deep features with high accuracy. Thereby, the point density is defined by the quantity of emitted points and the obtained measured values. As a result, a higher point density provides more measured values for the same scan area [10
]. Portable scanners are also becoming increasingly interesting for the manufacturing industry.
The scanning method Structured Light is based on the principle of triangulation, while incident laser lines are projected onto the object to be scanned. A common industrial 3D scanner is the Artec Space Spider, which uses structured light as scan technology. The Artec Space Spider uses a blue LED with a narrow wavelength, which allows better filtering of interference from ambient light [11
]. From the narrowband of laser stripes with known angular width, many points of the stripe are reflected by the surface. These points are detected by a charge-coupled device (CCD) sensor and transformed to 3D coordinates via the triangulation principle [8
]. In total, the Artec Space Spider uses three cameras at various angles and depths. In the center of the device, an RGB camera is installed surrounded by LED flash light to capture textures without the need for special light setup. Thereby the Artec Space Spider is advertised with a pinpoint accuracy of 0.05 mm and 3D resolution of 0.1 mm [11
Recent developments of commercial devices such as smartphones and tablets have shown that in addition to photogrammetry, scanning is also feasible with LiDAR (light detection and ranging) and TrueDepth. LiDAR includes ToF measurements, which determine the time it takes for an object, particle or wave to travel a distance. Therefore, LiDAR emits a pulse or modulated light signal and measures the time difference in the returning wavefront [12
]. TrueDepth uses vertical-cavity surface-emitting laser (VCSEL) technology and consists of a traditional camera, an infrared camera, a proximity sensor and a dot projector as well as a flood illuminator. The system is named and patented by Apple. The dot projector emits more than 30,000 points of infrared light, which are reflected on the surfaces. Consequently, the infrared camera picks up these light dots again and the pattern is analyzed by software to create a depth map. Using this depth map, a mathematical model is generated by machine learning algorithms [9
In iOS, the operating system of Apple’s smartphones and tablets, TrueDepth is mainly used for 3D face authentication and recognition, while LiDAR enables new features for Augmented Reality by accelerating plane detection. To scan objects, it is therefore necessary to install an additional application. Heges is such an iOS application that exploits TrueDepth and LiDAR. The authors of [16
] evaluated different smartphone applications, showing that Heges has one of the finest 3D resolutions (0.5 mm). The application can be used to scan objects and export scans as STL and polygon (PLY) files, while colors can also be captured. In contrast to other applications, Heges is also not limited to scan faces only. In addition, a screen sharing function facilitates scanning, and a one-time payment allows unlimited export of these scan files [16
]. Apple itself does not specify the accuracy of the respective technologies or hardware.
While the hardware and the software of the device determine the scan accuracy internally, there are also external factors influencing scan quality. A literature research has shown that factors are reflectance, shape and color of the object as well as surface texture and ambient lightning. In addition, the distance between object and scanner, scanning strategy and scanning movements influences scan quality [7
]. During post-processing, the meshing and surface fitting have also shown influences [17
]. Depending on the scan accuracy, different fields of applications can be exploited.
To determine the potential use of LiDAR and TrueDepth included in the recent iPad Pro and iPhone 12 Pro lineup as a 3D scanner, this study was conducted. For the evaluation, an iPad Pro (2020) [3
] was used and was compared to the industrial Artec Space Spider Handheld 3D Scanner [11
]. Therefore, different Lego bricks were scanned and compared via the software GOM Inspect. The aim of this study was to determine the scan accuracy of LiDAR and TrueDepth as 3D scanning technique.
In this study, an iPad Pro (2020) was used as a 3D scanner to evaluate the accuracy of the built-in LiDAR and TrueDepth technologies in comparison to an industrial 3D scanning system. Therefore, Heges was chosen as the application offering one of the finest scan resolutions (0.5 mm). As scan objects, Lego bricks of different shapes were used. By scanning the same objects simultaneously with the industrial 3D scanner Artec Space Spider, the results were then analyzed and compared in GOM Inspect.
Based on a literature research, there are different kinds of measuring methods, presenting processes to determine the scanning accuracy [7
]. The VDI 2634, for instance, defines a process for the verification of optical 3D measuring systems. However, this method is for 3D scanners using triangulation, whereas LiDAR is based on ToF and TrueDepth on VCSEL technology. Since there is no official method to determine the scanning accuracy, this study evaluates the scanning accuracy due to shape tolerances.
The LiDAR technology proved to be impractical for scanning small objects such as Lego bricks. For this reason, no results could be obtained to determine the accuracy. LiDAR is based on the ToF technique, which makes it possible to measure the time differential in the returning wavefront independently of scene textures. Therefore, full frame real-time depth estimates are possible [12
]. In addition, the data are contaminated with random systematic measurement errors. A method for improving the depth map is superresolution. Therefore, a high-resolution RGB camera is combined with a depth camera. An algorithm is used to combine those two images [12
]. So far (iOS 14), Apple does not provide the raw LiDAR depth data for additional applications on an iOS device. Instead, the LiDAR depth data is merged with the accompanying color (RGB) data using AI to create a depth map. Furthermore, the mesh of the depth map consists of coarse meshed triangles. In effect, LiDAR might be used to assist Augmented Reality or to scan large objects, for example, rooms, but it is not applicable to scan small objects such as Lego bricks. The depth image detected with TrueDepth shows a much finer mesh. Thereby, the dots of the infrared pattern are more closely arranged with each other than emitted dots with LiDAR are. As a result, TrueDepth is much more suitable for scanning small objects.
4.1. Impact of Color
By evaluating the flatness of different-colored Lego bricks, it can be demonstrated that color has an influence on the surface quality of the generated CAD models, using TrueDepth and the Artec Space Spider. In total, the objects scanned via TrueDepth show less accurate surfaces than objects scanned by the industrial scanner. However, no measurement exceeds the set accuracy of 0.5 mm in the application. Thus, the results are satisfying. Nevertheless, it is important to include the influencing factor color when high-precision scans are required. Black-colored surfaces, for example, were not able to be scanned. Black objects absorb visible light, but in the near infrared from about 1 μm it reflects the radiation. The TrueDepth camera uses infrared sensors that capture light in the wavelength range of 800 nm to 1300 nm to achieve pattern-based depth mapping. Therefore, the black object absorbs and reflects a high proportion of infrared emitted by TrueDepth and the object cannot be scanned. The same applies to the Artec Space Spider, whose blue structured light pattern is absorbed. In addition, the red brick showed the highest deviation of 0.044 mm scanned by TrueDepth, while the Artec Space Spider was not even able to track the object at all. Due to the blue light LED illumination of the Artec Space Spider, red is not visible for the scanner because the light gets highly absorbed. At the same time. the highest standard deviation of 0.104 mm was achieved by scanning the red brick with the TrueDepth technology.
This case shows how much influence disturbances can have. Disturbance variables could be environmental, which have not been eliminated yet, or the post-process could have a large influence. Thus, further experiments could show the influence of meshing and surface fitting. When scanning dark surfaces, antiglare spray or ambient light might improve the scan quality [7
]. Since the blue and white objects obtained the highest accuracy with both scan technologies, those colors were chosen to determine the scanning accuracy.
4.2. Scanning Accuracy
The scanning accuracy determines the potential use of a 3D scanner and thus fields of applications. In order to make a generally valid statement about the scan accuracy of arbitrarily formed objects, shape tolerances were evaluated.
Although the objects scanned by Artec Space Spider showed small deviations for straightness, flatness and cylindricity, these shape tolerances have a great influence on the scanning accuracy with TrueDepth. When considering straightness and flatness, four of six measurements had deviations of more than 0.5 mm. In fact, the set parameter of 0.5 mm accuracy was therefore not fulfilled. However, the high standard deviation of 0.161 mm (straightness) and 0.105 mm (flatness) showed the potential of more accurate CAD models. Thereby, it was observed that both the scan strategy and the scan movements can lead to much more accurate results. For example, it was found that two rotations of the scanned object actually improved the scanning accuracy. In comparison to the Artec Space Spider one can assume that hardware and software are further developed by producing more repeatable results. In fact, this can be observed by evaluating cylindricity. When using TrueDepth, the live stream of capturing the Lego brick geometry showed that the object was lost and was captured again several times during one scan. This led to greater deviations of the generated mesh. Analogously, the same effect can be observed when testing roundness. Thereby, the Artec Space Spider showed greater deviations as well.
Even though the CAD models have certain shape tolerances, the accuracy of the size, form, and orientation were not verified. Therefore, the target-actual comparison has to be analyzed. Especially in this analysis, the highest deviations were found. Considering the position and profile of a surface, the ratio of deviations between the Artec Space Spider and TrueDepth showed similar behavior. The scans taken with TrueDepth were almost twice as high as those of the Artec Space Spider. It is worth mentioning that the most accurate CAD model generated by Heges had a more accurate surface than the CAD model of Artec Space Spider. Notable were deviations of 4.51 mm (Artec Space Spider) and 4.92 mm (TrueDepth) in average by inspecting the profile of a line. However, the highest deviations were formed by small deviating areas. Also, in those cases the standard deviations were very high leading to the conclusion that more accurate scans can be achieved.
Manufacturers today tend to adopt product differentiation strategies and more customer-centric approaches to ensure future competitiveness. Mass customization and product diversification are key parts of these trends that require new hardware and software solutions. In this context, RE provides a method to individualize a product in a cost- and time-efficient way. In this case, the customer must be provided with hardware for 3D scanning. The LiDAR and TrueDepth technologies included in the newest devices by Apple offer such a deployment. However, the scanning accuracy determines the potential application and possible use cases. In this work, we have highlighted the potential of a smartphone and tablet as a cost-efficient 3D scanner. Additionally, we have pointed out influencing factors during 3D scanning. Based on the color influence as well as shape and position tolerances, we evaluated the scanning accuracy of an iPad Pro with the LiDAR and TrueDepth camera and compared it with an industrial 3D scanner, the Artec Space Spider. In principle, the industrial scanner should still be preferred if it is available. The advantage of smart devices, however, is that they are widely distributed among potential customers. Although scanned objects obtained with the iPad Pro had higher deviations than objects scanned by the Artec Space Spider, the iPad Pro offers a way to scan at lower costs. However, certain color ranges do not pick up well. Specifically, black tends to be filtered out. Use of a coating can mitigate this. In a scenario in which a customer is preparing the scans, this seems to be impractical, though. Depending on the use case, it must be evaluated whether the scanning accuracy of an iPad Pro meets the requirements. In addition, more accurate CAD models can be achieved through a specific and adaptive scanning strategy. In addition, asymmetric and detailed objects can lead to more accurate CAD models. Due to the fact that the TrueDepth sensor is front-facing, it is currently suitable for scanning objects only to a limited extent. Scanning objects is made difficult due to the location of the sensor and the handling required as a result. Instead, it can be used to design products that are tailored to the customer’s head, for example. The integrated LiDAR scanner would provide a way out. However, the available accuracy is not yet sufficient for the manufacturing of customized products due to the low mesh resolution. A new iteration of integrated sensor technology or a software update granting more access to developers may provide a remedy here. Future work will show how the technologies perform under different lighting conditions and how different scan strategies or scan times affect the accuracy.