Artificial Intelligence for Forensic Image Analysis in Bullet Hole Comparison: A Preliminary Study

Abstract

The application of artificial intelligence within forensic image analysis marks a significant step forward for the non-destructive examination of evidence, a crucial practice for maintaining the integrity of a crime scene. While non-destructive testing (NDT) methods are established, the integration of AI, particularly for analyzing ballistic evidence, requires further exploration. This preliminary study directly addresses this gap by focusing on the use of deep learning to automate the analysis of bullet holes. This work investigated the performance of two state-of-the-art convolutional neural networks (CNNs), YOLOv8 and R-CNN, for detecting ballistic markings in digital images. The approach treats digital image analysis itself as a form of non-destructive testing, thereby preserving the original evidence. The findings demonstrate the potential of AI to augment forensic investigations by providing an objective, data-driven alternative to traditional assessments and increasing the efficiency of evidence processing. This research confirms the feasibility and relevance of leveraging advanced AI models to develop powerful new tools for Forensic Science. It is expected that this study will contribute worldwide to help (1) the police indict criminals and prove innocence; (2) the justice system judges and proves people guilty of their crimes.

1. Introduction

Non-destructive testing [1,2,3] is a major area of interest within the field of Forensic Science [4]. It is relevant because non-destructive testing is important to avoid altering the crime scene and to preserve evidence during artifact collection and analysis [5,6,7,8,9,10,11]. Evidence obtained from non-destructive testing is a major contributor to helping the police and justice system incriminate, judge, and prove guilty of a crime [9]. Comparative testing of ballistics [12] is a key strategy for forensic scientists because it allows forensic experts to identify the characteristics of firearms used to commit crimes. The most popular comparative ballistics testing compares the marks left on bullets when fired from the gun barrel. This test seeks to identify a firearm based on the correspondence between the marks of the projectiles found at the scene of the crime and the marks of the projectiles fired in a forensic laboratory with the alleged murder weapon. However, criminals often remove bullets and cartridges (cases) from crime scenes, as well as weapons, making this testing from internal ballistics [13] unfeasible.

On the other hand, removing surfaces targeted by gunfire at the scene of the crime sounds very unusual. Therefore, comparative ballistics testing focused on bullet holes is best suited for forensic cases dealing with the absence of bullets and cartridges (cases) at crime scenes. Another important test of ballistics compares images of bullet holes from surfaces targeted by gunfire. Several studies related to this type of testing, using different approaches and with different purposes, can be found in the scientific literature, as in [14,15,16,17,18,19,20,21,22]. This testing seeks to identify characteristics (such as caliber and type) of a firearm based on the correspondence between the image of the bullet holes found at the crime scene and images of previously categorized bullet holes composing a dataset, which is studied in terminal ballistics [13]. Debate continues about the best strategies that utilize artificial intelligence for the management of comparative testing of ballistics.

Figure 1 shows the main steps of the terminal ballistics test. It is performed in four steps: evidence collection, bullet hole detection, classification, and matching. Artificial intelligence is usually applied to the last three steps, and each step can apply a different artificial intelligence strategy if necessary. The complete testing is so complex that it is divided here into two parts, as shown in Figure 1. The current study fits into the first part, which is more directly related to non-destructive testing than the second part.

In the first part, the evidence collection step must follow some practices to preserve evidence at the crime scene and in forensic laboratories. There are many publications that describe guidelines for preserving evidence during the collection and analysis of artifacts found at crime scenes, such as [6,7,8,9,10,11]. Regarding the second step, several articles express a wide variety of strategies for detecting bullet holes in digital images based on artificial intelligence and other sources [13,23,24,25,26,27]. However, to the knowledge of the authors, none of the previously published papers presents testing that utilizes artificial intelligence to compare bullet holes in surfaces targeted by gunfire within the context of terminal ballistics as a non-destructive technique for Forensic Science.

This paper highlights the importance of addressing bullet hole detection by artificial intelligence as a non-destructive testing method. In this sense, artificial intelligence was applied to data from diversified datasets to investigate the accuracy of the proposed strategy. For that, convolutional neural networks, Recursive (R-CNN) and You Only Look Once (YOLOV8) [1,28,29], were used to train and detect bullet holes found in digital images.

It is important to note that the evidence collection step can be performed with an ultrasonic technique or by charge-coupled device (CCD) sensors. While the ultrasonic technique can, for example, use antisymmetric Lamb waves, as seen in [30], the CCD sensors detect electromagnetic waves across a wavelength range, as seen in [1,31]. This work was performed with images acquired by CCD sensors because the datasets were already available in the literature.

This paper innovates by addressing the topic of comparing bullet holes in surfaces targeted by gunfire by using artificial intelligence in the context of terminal ballistics for Forensic Science as a non-destructive testing method, as presented in Figure 2.

The main contribution of this paper is to be a reference to researchers and forensic experts to ensure that comparative testing will really be performed by AI-based tools as a non-destructive testing method to detect bullet holes. This is a central issue for solving crimes aided by artificial intelligence since a not rigorous investigation can impact the accuracy of the results achieved by artificial intelligence and, consequently, the crime-solving.

This paper is organized as follows: Section 2 presents information about principles, rules, and protocols in the field of Forensic Science, general concepts regarding convolutional neural networks, and an introduction to the You Only Look Once algorithm. Section 3 describes the methodology. Section 4 presents the results, and Section 5 presents conclusions and suggestions for future studies.

2. Background

2.1. Non-Destructive Testing in Forensic Ballistics

2.1.1. Forensic Ballistics

Forensic ballistics investigates not only firearms and their respective ammunition but also the phenomena and effects caused by shots from these weapons [13]. It is an applied science that aims to clarify the circumstances of the use of firearms while also understanding and solving crimes. It is related to different types of testing that serve to determine, among other characteristics, the caliber and type of weapon used, the number of shots fired, the distance of the shot or the trajectory of the bullet, both in relation to the target surface, and the shooter’s position.

Forensic ballistics testing is grouped and classified into four different groups: internal ballistics, intermediate ballistics, external ballistics, and terminal ballistics [13], as described below.

• Internal ballistics investigates the phenomena that occur internally within the firearm.
• Intermediate ballistics (or transition ballistics) investigates the phenomena and behavior of the projectile, influenced by the gases remaining from the shot, immediately after leaving the gun barrel.
• External ballistics investigates the behavior of the bullet as it travels through the air in the distance between the weapon and the target surface of the shot.
• Terminal ballistics investigates the phenomena, behavior, and effects of the bullet as it collides with and pierces the target surface of the shot.

2.1.2. Evidence Collection in Forensic Science

When commenting on the guidelines used in Forensic Science to ensure that testing is non-destructive, in this work, the evidence collection step was not separated from the artifact analysis steps. These steps were kept as part of the same set because many of the professionals who investigate crime scenes are part of the crime laboratory staff (administratively) [4]. Furthermore, forensic scientists increasingly participate in crime scene investigations to assist crime scene investigators (CSIs) in their tasks [4]. Therefore, the widely-held belief that many people linked to Forensic Science still have not considered investigations carried out at the crime scene as part of Forensic Science tends to become increasingly outdated.

Even important publications in the area of Forensic Science, such as [4,5,6,7,8,9,10,11], include at least one chapter or section on collecting evidence at crime scenes. There are several different types of publications, such as awareness protocols for non-forensic personnel [6], collection guides [7,8], manuals [9,10], and online test banks [11]. Together, these publications seek to raise awareness among police departments around the world about the need and importance of applying guidelines for conducting forensic investigations in criminal investigation activities.

2.1.3. Relevance

Although the content of these publications may appear redundant or obvious to some forensic scientists in developed countries, these same guidelines still remain at least somewhat distant from the reality of many developing countries, which suffer from a high crime rate and a low number of forensic experts to investigate all crimes occurring within the respective territorial borders. Furthermore, these same countries have low criminal conviction rates due to the high rate of unsolved or incorrectly solved crimes. In these countries, impunity often serves as an incentive to commit crimes. Therefore, this study presents itself as an important and necessary publication to try to change this reality across the world by raising awareness of the police and justice systems in these countries. This time, however, it approaches this issue in an innovative way to the knowledge of the authors, i.e., in the context of non-destructive testing carried out by artificial intelligence to help resolve a terminal ballistic problem.

2.1.4. Guidelines for Carrying out Non-Destructive Testing

According to the guidelines for carrying out forensic examinations [6,7,8,9,10,11], some of which are briefly discussed, upon discovering the occurrence of a criminal act, an ordinary citizen, a rescuer, or even an unprepared police officer may, even without meaning to, alter the crime scene. Interference with the evidence present at the crime scene can be mitigated if there is intense work to raise awareness among teams that carry out non-forensic work [6]. Therefore, crime scene preservation work should begin as soon as possible. To do this, the crime scene must be isolated, and only personnel essential to the investigation should be authorized to enter this perimeter. These precautions are essential to preserve the integrity of the crime scene, thus avoiding contamination, loss, or destruction of evidence.

According to these guidelines, each crime scene is unique. Therefore, the requirements for its investigation may require adaptations or changes, as new evidence can be recognized during the investigation of the scene. Therefore, the assignment of responsibilities and tasks must be dynamic and flexible but always planned to meet the specifics of the unique scenario. Therefore, for each crime scene, experts must organize themselves and prepare their functions in order to seek the veracity of the facts.

Often, the fidelity of facts can be hidden in a small and seemingly insignificant clue at first glance, such as a simple strand of hair, but which, after forensic analysis, may prove to be essential for solving the crime. Therefore, by assigning responsibilities to experts, limiting the number of people at the crime scene is paramount. Moreover, adequate communication is critical to prevent these clues from being neglected, lost, contaminated, compromised, or even destroyed. In parallel, false clues can be introduced accidentally, disorienting experts and influencing the final outcome of the investigation.

The considerations made in the previous paragraph are valid both for activities carried out at the crime scene, as well as for the handling of instruments and packaging used for handling samples analyzed in forensic laboratories. Therefore, unpreparedness in attending and staying in these places may cause a loss of evidence or compromise the analysis process that would determine someone’s guilt or innocence.

2.2. Introduction to Convolutional Neural Networks

Deep learning leads to complexity problems where there are large datasets available. It is an artificial intelligence subfield focused on models of large neural networks that make data-driven decisions [32]. Deep learning seeks to emulate human cognitive processes in machines, aiming to make machines intelligent [33]. The essential human capacity for learning forms the basis for Machine Learning (ML), a field dedicated to reducing human effort across various tasks by enabling machines to learn from past experiences [34]. The ML methods are improving the state-of-the-art in different applications, such as speech recognition and object detection [35]. ML encompasses three primary learning paradigms: supervised, unsupervised, and semi-supervised learning. Traditional ML techniques often require a process of feature extraction, demanding domain-specific knowledge and posing challenges in selecting appropriate features for given problems. Deep learning addresses these challenges by automating the extraction of significant features.

Convolutional Neural Network (CNN) is a type of deep learning algorithm that makes use of a feedforward architecture designed to adapt to environmental changes, maintaining a desired equilibrium state. The CNNs can be defined as hierarchical feature detectors inspired by biological systems, capable of learning highly abstract features and accurately detecting objects [36]. CNNs are favored over classical neural network models for several reasons: weight sharing reduces the number of parameters needing training, leading to better generalization; the integration of classification and feature extraction simplifies the process; and the implementation of large networks is more manageable with CNNs compared to general artificial neural networks [37,38,39].

Structural Overview of Convolutional Neural Networks

An artificial neural network model normally comprises a single input and output layer with multiple hidden layers [40]. Each neuron processes an input vector X and applies a function F with a W weight vector to produce an output vector Y. It is represented by Equation (1) [41].

$$F\left(X,W\right)=Y$$

(1)

It is worth highlighting that the W vector indicates the strength of interconnections between neurons in adjacent layers and, so, it is essential for image classification tasks. The general CNN model includes four primary components: convolution layers, pooling layers, activation functions, and fully connected layers, as presented in Figure 3.

In image classification, the CNN model receives an image as its input and outputs a class label based on extracted features. Neurons in subsequent layers connect to those in preceding layers through receptive fields, which are responsible for extracting local features of the input image [42]. Receptive fields form weight vectors that remain consistent across all points in a plane, ensuring the detection of similar features at different locations within the input, as illustrated in Figure 4 [43].

The weight vectors work as a kernel sliding over input vectors to generate feature maps in a process called convolution. This sliding, both horizontally and vertically, constitutes the convolution operation. It extracts multiple features from a single input image layer, creating distinct feature maps and significantly reducing the number of parameters to be trained due to the receptive fields mentioned [45].

2.3. The YOLO (You Only Look Once) Algorithm

Object detection systems modify classifiers to identify objects by evaluating locations and scales within test images. Traditional systems, such as deformable parts models (DPM), utilize a sliding window for the classifier to traverse uniformly spaced locations across the image [46].

Modern approaches, such as Recursive Convolutional Neural Networks (R-CNNs), employ region analysis methods to generate potential bounding boxes, followed by classification and post-processing to refine the detected boxes, eliminating duplicates and re-labeling them based on detected objects. However, since it is a recursive process, the result of a previous step is used as input for the next step until the desired objective is achieved. In this way, this process can be slow and difficult to optimize [47].

The You Only Look Once (YOLO) algorithm redefines object detection as a single regression problem, obtaining the bounding box coordinates directly from the input image pixels without the need for recursive steps. YOLO uses a single CNN to predict multiple bounding boxes and class probabilities simultaneously, streamlining the detection process. Training on full images, YOLO optimizes detection performance directly [1,28,29].

YOLO integrates all object detection components into a unified neural network. It utilizes features from the entire image to predict each bounding box, operating globally across the entire image and all contained objects [28]. This fact enables YOLO to process end-to-end inputs at real-time speeds while maintaining high accuracy.

The algorithm splits the input image into an $S\times S$ grid, as presented in Figure 5. Each grid cell, if containing the center of an object, becomes responsible for representing that object. Each cell has information about its $BB$ bounding boxes and its corresponding confidence scores, which indicates the model’s certainty that a box contains an object and the accuracy of the bounding box itself. Cells without objects should have zero confidence scores, whereas cells with objects should have confidence scores equivalent to the intersection over union (IOU) between predicted and ground truth boxes. IOU is calculated using the Jaccard index, as shown by Equation (2). Confidence score is formally defined by Equation (3) [28].

$$IOU={\displaystyle \frac{\left(A\cap B\right)}{\left(A\cup B\right)}}$$

(2)

$$\mathit{Confidence}=\mathrm{Pr}\left(Object\right)\ast {IOU}_{predicted}^{g.truth}$$

(3)

Each bounding box has five parameters of prediction: $x$, $y$, $w$, $h$, and its confidence score. The $\left(x,y\right)$ coordinates represent the box center, while $w$ and $h$ represent, respectively, width and height relative to the entire image. Additionally, each grid cell has associated a $C$ conditional class probability, conditioned on the grid cell containing the object. The conditional class probability $C$ is represented by Equation (4) [28]. Regardless of the number of bounding boxes $BB$, only one set of class probabilities is predicted per grid cell.

$$C=Pr\left({Class}_i|Object\right)$$

(4)

3. Materials and Methods

3.1. Materials

3.1.1. Forensic Equipment and Techniques for Creating Datasets

When a forensic analysis method is performed based on visual evidence, the way images are taken at the crime scene can make the method either non-destructive or destructive. The equipment used by the forensic expert is also relevant. Therefore, this subsection focuses on how visual evidence must be collected. Forensic equipment and techniques are described here.

Regarding the equipment, in addition to wearing gloves, shoe covers, masks, caps, coats, overalls, etc., it is necessary to use a camera with at least 35 mm film. It is also necessary to choose a fixed focal length lens that is twice your camera’s normal focal length. In order to control the depth of field, you must select either Manual Mode or Aperture Priority and set the ISO option to 400 ISO. To the end of capturing the image in its correct color, choose the proper White Balance. Select an f/stop setting of f/16 or f/22. To the end of avoiding the camera from vibrating, use a cable or electronic release and a tripod.

You should store the images in the RAW uncompressed file format (minimum of 24 bits for color images and 8 bits for grayscale images). The minimum resolution for an image is 1000 ppi when taking into consideration a calibration to the actual size (1:1). The highest resolution can be achieved by filling the viewfinder with the investigated object and scaling in landscape orientation. The maximum area of capture must be calculated by dividing the amount of horizontal and vertical pixels on the camera sensor by 1000.

Regarding techniques, for each crime scene, it is necessary to start collecting visual evidence by strictly following the guidelines for carrying out non-destructive testing, as described in Section 2.1.4. It is important because collecting visual evidence is only considered a non-destructive activity if the forensic expert is careful when taking photographs without disturbing or destroying any evidence at the crime scene. After that, also for each crime scene, four sets of images are normally taken, as described below.

The first set of images must show the locations of the bullet holes in the scene and how each bullet hole relates to its surroundings, which may include other bullet holes, items of evidence, or objects in its vicinity. In this study, there is a certain alignment between this first set of images and the dataset (1) due to some similarities between them. For example, in Dataset (1), each mid-view photograph shows bullet holes and their neighborhood.

The second set of images must show each bullet hole itself. In other words, this second set must only consist of close-up view photographs. A close-up view photograph must clearly show the close-up of the bullet hole to make its details more apparent. In this study, there is a certain alignment between Dataset (2) and the second set of images due to some similarities between them. For example, Dataset (2) is comprised of many close-up view photographs of single holes, marks, or stains caused by bullets or other sources. Furthermore, some images in Dataset (2) show some similarities with the first set of images. For example, Dataset (2) is also composed of other images placed between close-up view and mid-view photographs.

The images for composing these two datasets must be obtained at the beginning of the investigation at the crime scene. This must occur before other forensic experts begin collecting physical evidence. These two image datasets are very important because they are the only datasets that can ensure that testing is non-destructive. Regarding the third and fourth datasets, the images for composing these two datasets must only be taken after the other forensic experts have investigated all the evidence at the crime scene.

The third set of images should also show the positions of the bullet holes at the crime scene and how each bullet hole relates to its vicinity. It must also only consist of mid-range view photographs. The difference between the third set of images and the first one is the presence of numbering devices. It is important that the bullet holes are individually numbered. This numbering clearly differentiates one bullet hole from another and indicates the labeled position of each bullet hole in relation to other parts of the immediate area of the crime scene.

The fourth set of images must also show each bullet hole itself. It must also only consist of close-up view photographs. The difference between the fourth set of images and the second one is the presence of a numbering device and a scale. The numbering device identifies the bullet hole, and the scale shows its relative size in the photograph. The scale must be placed on the same plane as the surface containing the bullet hole. This makes the scale useful for measurements and prevents the forensic expert from introducing distortion into the photograph [9,48,49,50].

3.1.2. Datasets

It is expected to call more attention to information related to ensuring that tests must be performed in a truly non-destructive manner than to technical details of ballistics. It is very important that forensic experts have prior knowledge of the characteristics that must be present in the images they intend to take at the crime scene. This prevents forensic experts from returning to the crime scene unnecessarily.

To this end, it would also be very important for forensic experts to have visual references of the desired image characteristics for the datasets. There should be a dataset composed of examples of images that may favor forensic analysis and another dataset composed of examples of images that may disfavor forensic analysis. Therefore, this paper performs experiments based on two sets of images taken under different conditions, Datasets (1) and (2).

Dataset (1) is composed of images taken under ideal conditions, while Dataset (2) is composed of images taken under non-ideal conditions. The decision to perform experiments based on these two sets of images taken under different conditions was inspired by the scientific literature. There are several articles recently published that describe different uses of datasets under ideal and non-ideal conditions, such as [51,52,53,54]. For example, a discussion of AI-based object detection performed under ideal conditions (e.g., stable weather or adequate lighting) and non-ideal conditions (e.g., adverse weather or insufficient lighting) is described in [51].

Some advantages of using these two datasets are the possibility of (1) presenting a dataset composed of examples of images that favor forensic analysis and another dataset composed of examples of images unfavorable to forensic analysis; (2) carrying out experiments based on two different scenarios related to NDT: controlled and non-controlled; and (3) finding out results related to both scenarios. The controlled scenario represents the ideal conditions achieved by forensic experts as they strictly adhere to forensic guidelines when taking photographs at crime scenes. The non-controlled scenario represents the non-ideal conditions witnessed by forensic experts as they neglect forensic guidelines when taking photographs at crime scenes.

With the proposed experiments, it is expected to find out what results some CNNs would achieve when trying to detect bullet holes on these two datasets. It also helps the researchers to focus their attention on the desired image characteristics, such as uniformity, homogeneity, standardization, quality, resolution, coloration, stains, textures, dirt, lighting conditions, angle of penetration of the projectile into the surface, distance and angle of image acquisition, etc.

With this intention, Dataset (1) presents images obtained under controlled conditions, with little variation in the angle of penetration of the projectiles, and in the image capture distance. On the other hand, Dataset (2) presents images obtained under non-controlled conditions, that is, non-uniform, non-homogeneous, non-standardized images, of low quality and resolution, with varied colors, with different types of surfaces, with variations in the penetration angle of the projectiles, with varied lighting conditions, and images obtained by different sensors and configurations, as well as positioned at different distances from the depicted surfaces.

In this sense, the experiments performed in this study were based on Datasets (1) and (2) to find out if it is possible to detect bullet holes in different situations using images with different characteristics. Furthermore, it would be desirable to have at least one of the two datasets as a visual reference for forensic experts on what the set of images (dataset) should or should not look like to be obtained at a crime scene.

Datasets (1) and (2) are also used to contribute to the generation of results related to the detection of bullet holes that can serve as references in NDT. These reference results can guide forensic experts on how to obtain one-time images that will help testing achieve high accuracy and precision.

A bullet hole image dataset tends to be very small and limited since it is difficult to find a set of images taken under ideal conditions that meet the characteristics mentioned. In addition, each crime scene usually has a small number of bullet holes, normally caused by the firing of small firearms such as pistols and revolvers. Consequently, it is expected that this limited dataset tends to contribute to inflated results related to metrics such as accuracy and precision. Even so, such results would be very important for NDT because they would be a reference for other studies to analyze whether their results are close to the results obtained based on the use of this dataset, which is a reference.

On the other hand, a dataset tends to be large and diverse when used as a visual reference for what a set of images should not look like to be taken at a crime scene to favor forensic analysis. This dataset is usually large and diverse because it is easy to find a set of images taken under non-ideal conditions that meet the characteristics mentioned above. Consequently, it is expected that this large and diverse dataset will tend to contribute to results presenting low values in terms of accuracy and precision. Even so, such results would be very important for NDT because they would be a reference for other studies to analyze whether their results overcome the results obtained based on the use of this dataset, which is also a reference.

For NDT, the best terminal ballistics method or the best AI-based method is not important if it can be related to an activity that has a chance of being destructive. For NDT, the most important thing is that the method used ensures truly non-destructive and useful testing. Therefore, the focus of this manuscript is on showing how to ensure truly non-destructive testing (related to the context of Forensic Science) rather than on showing which is the best AI method to be applied to solve a terminal ballistics problem.

3.2. Methods

Figure 6 shows a flowchart that summarizes the non-destructive testing performed in this paper by artificial intelligence for Forensic Science. The content of Figure 6 corresponds to the first part of the same testing presented in Figure 1. In the flowchart, it is possible to identify the two main phases of the methodology: evidence collection and bullet hole detection.

3.2.1. Evidence Collection

The first phase shown by the flowchart in Figure 6 represents how evidence must be collected to allow the scientific community to consider the testing truly non-destructive. In other words, this paper presents the ideal way to carry out the first phase of a forensic investigation (see Section 2.1.4). However, in this study, datasets that already consisted of images of bullet holes were used. Therefore, it was not necessary for us to collect evidence directly at crime scenes. Furthermore, for this study, performing the evidence-collection phase is of minor importance because the images in the datasets used are adequate to achieve this paper’s objectives.

3.2.2. Bullet Hole Detection

Bullet hole detection was performed on two different datasets in this study. In total 50 images of shooting targets taken under controlled conditions compose Dataset (1), as shown in Figure 7. In Dataset (1), there are patterns and regularities in the images. In total, 184 images taken under non-controlled conditions comprise Dataset (2); among them are 138 images showing bullet holes, as shown in Figure 8. In Dataset (2), images show a wide range of surfaces, environments, and lighting conditions.

Image Fetch and Preprocessing Using Digital Image Processing

At the beginning of this phase, images were fetched from each dataset (Dataset (1) and Dataset (2)). Then, all images from both datasets were automatically or manually cropped using the web tool Bulk Image Crop with an aspect ratio of 1:1 achieving a consistent size of 640 × 640 pixels. In this study, the images were resized to 640 × 640 pixels in order to achieve a viable cost–benefit balance involving processing time and demand for computational resources. This was necessary for this study because it had certain constraints on funding and computational resources to perform the current research. When the main focus of the research is terminal ballistic tests, the ideal is not to change the original dimensions of the images so as not to influence the results. However, this study focuses on the important aspects to ensure testing is non-destructive. This study does not aim to present the best or most innovative method to solve a terminal ballistic problem. Therefore, although resizing images to 640 × 640 pixels is a limitation in this research, this is of minor importance for the presented study, as this limitation does not directly impact the aspects that are truly important for NDT.

After that, the sampling was carried out using the Roboflow2 tool, which provides image labeling tools for various machine learning models, such as YOLOV8 and R-CNN. Image labeling consists of defining samples and assigning them a label. Samples are the regions within an image that contain the desired object, e.g., a bullet hole. Figure 9 shows an example of how to define the bounding boxes for the bullet hole label, representing the bullet holes left by the shooter.

The entire labeling process was performed manually for both datasets. All images in Dataset (1) and 75% in Dataset (2) contain bullet holes, most of which were therefore labeled as bullet holes. The sampling is an important step before model training because a neural network can learn based on labeled samples during the process of training.

CNN Model Application

The next step was to train the models capable of detecting the labels defined in the previous step. For each dataset (Dataset (1) and Dataset (2)), two different models were trained: YOLOV8 and R-CNN, resulting in four scenarios. To train YOLOV8 or R-CNN models on the input data and to have a better analysis of the model’s accuracy, the method of separating data into training, validation, and test sets was used [33]. To this end, both datasets were divided as follows: 70% of their images were used for the training process, 20% of their images were used for the validation process, and 10% of their images were used for the test process. In other words, 70% of the data is used for parameter learning, while 20% is used for validation, where the error committed is used to define how the learnable parameters will have their values modified at each learning epoch. Additionally, 10% of the data is only used to estimate the generalization power of the model after training.

For R-CNN models, the following were used: (1) The R-101-FPN and X-101-32x8d-FPN backbones. (2) The max iterations hyperparameters (epochs) between 256, 512, 1024, and 2048. (3) The number of images per batch between 2 and 8. For YOLOV8 models, the following were used: (1) The nano, small, and medium backbones. (2) The hyperparameters of epochs between 256, 512, and 1024. (3) The “batch size” between 16 and 32. Each training process produces a file of weights as output. The file of weights is used by the neural network to detect the trained labels. Thus, the file corresponding to the weights used to detect the bullet holes was obtained. Other information about the training step, such as performance and accuracy metrics, will be presented and discussed in Section 4.

Having the CNN models trained, the next task was performed by invoking the YOLOV8 and R-CNN models with the weights to detect the bullet holes and storing the result containing the coordinates of the bounding boxes of detected objects and their respective confidence levels. As can be seen in Figure 10, there is a possibility of occurrence of the undesirable effect of multiple detections on the same object. To solve this problem, non-maximum suppression was performed, as explained in the next paragraph.

The non-maximum suppression algorithm [55] is often used in computer vision to resolve duplicates like the one found in Figure 10. Taking $BB$ and $S$ as sets of bounding box coordinates and their respective confidence scores and N_t as the overlap limit, the algorithm returns $BB$ and $S$ containing the filtered bounding boxes and their respective confidence scores. Summarizing, the algorithm traverses in b_i in $BB$, descending order of their respective confidence scores, and tests whether its intersection over the union with the remaining b_i+1 in $BB$ is greater than the limit N_t, eliminating b_i+1 and its confidence score if applicable. The intersection in or of the union applied to two-dimensional objects is given by the Jaccard index [56], which is the ratio between the union of their areas and their intersection (see Equation (2)). This index indicates how much the two objects overlap and was set at N_t = 0.5.

The bullet holes detected are represented by the $x_1$, $y_1$, $x_2$, and $y_2$ coordinates of their northwest and southeast anchors. The equation of the center of the line between these two points was used to determine the center of the bullet holes, with their radii being set at a constant value of 25 pixels. In other words, the position of the center of the bullet hole is calculated by Equation (5), taking into consideration the center of a straight line in the plane and applied to the straight line that connects the northwest $x_1$ and $y_1$ and southeast $x_2$ and $y_2$ anchors of the object’s bounding box identified.

$$f\left(\left(x_1,y_1\right),\left(x_2,y_2\right)\right)=\left({\displaystyle \frac{\left(x_1+x_2\right)}{2}},{\displaystyle \frac{\left(y_1+y_2\right)}{2}}\right)$$

(5)

3.2.3. Border Between the Parts of This Study

The first part of this study focuses on guidelines for preserving evidence at the crime scene to ensure that the testing is truly non-destructive. Furthermore, the first part of this study conducts bullet hole detection experiments. These experiments are carried out based on two scenarios related to NDT. Both the datasets used in the experiments and their results are intended to serve as references in NDT. The datasets are composed of close-up or mid-range view photographs without any addition of external elements such as scales or numbering devices in order to maintain consistency with NDT practices. Paying close attention to the first part of this study and adhering to its guidelines is critical in NDT.

The second part of this study focuses on comparing bullet hole sizes. The comparison occurs between two images at a time, namely (1) one that comprises the fourth set of images and (2) a reference image that shows the surface of a specific material (e.g., metal, glass, wood, etc.) containing a hole caused by a bullet of previously known caliber. The comparison must take into account the material the target surface is made of and the type of ammunition, as these factors can cause small variations in the size of the holes for the same caliber. The images compared must be close-up view photographs, each showing a bullet hole accompanied by a scale. The scale is used to measure, for example, the diameter of the bullet hole. Bullet holes with similar measurements in the compared photographs are more likely to belong to the same caliber. Validation of the comparisons can be performed based on the metrics Mann–Whitney U test [17,57,58,59] or Kruskal–Wallis H test [21,60,61,62,63].

Mann–Whitney U Test

Mann–Whitney U test is a non-parametric statistical test of the null hypothesis. It evaluates if the probability of $X_i$ being greater than $Y_i$ is different from the probability of $Y_i$ being greater than $X_i$, i.e., $P\left(X_i>Y_i\right)\ne P\left(Y_i>X_i\right)$. The values $X_i$ and $Y_i$ are randomly selected from two populations. Equation (6) expresses Mann–Whitney U test [17,57,58,59]. Equation (6) considers two sample groups ($X$ and $Y$) and their respective samples $i$ ($X_i$ e $Y_i$), being $1\le i\le n_1$ and $1\le i\le n_2$, respectively. In other words, the first sample group is composed of the samples $X_1,\dots ,X_{n1}$. The second sample group is composed of the samples $Y_1,\dots ,Y_{n2}$.

$$U_1=n_1{n}_2+{\displaystyle \frac{n_1\left(n_1+1\right)}{2}}-{\mathrm{R}}_1,U_2=n_1{n}_2+{\displaystyle \frac{n_2\left(n_2+1\right)}{2}}-{\mathrm{R}}_2$$

(6)

For Equation (6), the sums of the ranks in the first and second sample groups are represented by $R_1$ and $R_2$, respectively. All samples from both groups must be ranked. Rank 1 represents the lowest value. Rank $n_1+n_2$ represents the highest value.

The Mann–Whitney U test can be extended to work with more than two groups. In this case, the test is known as the Kruskal–Wallis H test [21,60,61,62,63].

Kruskal–Wallis H Test

The Kruskal–Wallis H test is a non-parametric statistical test that assesses whether samples originate from the same distribution. It compares two or more independent samples. Samples can be the same or different sizes. All data from all groups are ranked together, and the sum of the ranks is obtained for each sample. In other words, the H test ignores group membership when ranking data from 1 to $N$. Equation (7) expresses the Kruskal–Wallis H test [21,60,61,62,63].

$$H={\displaystyle \frac{12}{N\left(N+1\right)}}\sum _{i=1}^C{\displaystyle \frac{R_{i^2}}{n_i}}-3\left(N+1\right)$$

(7)

For Equation (7), $C$ is the number of samples; $n_i$ is the number of observations in the ith sample; $N=\sum n_i$ is the number of observations in all samples combined; and $R_i$ is the sum of the ranks in the ith sample.

The Kruskal–Wallis H test is also known as one-way ANOVA on ranks.

4. Results and Discussion

One of the ways to measure the performance of a detection model is by using a confusion matrix of their labels [64]. A confusion matrix consists of the following components: true positive (TP), when the detected value is equal to its true positive value; true negative (TN) when the projected value is equal to the actual value, but it is negative; false positive (FP), when the anticipated value was incorrectly predicted as positive, which is actually negative; and false negative (FN), when the anticipated value is incorrectly predicted as negative when in reality its value is positive.

Figure 11 and Figure 12 present the confusion matrices for the labeled bullet holes, taking into consideration the training datasets. The confusion matrices were generated by the training processes of YOLOV8 and R-CNN, respectively. The columns represent the true values, thereby making the label the true positive and the background the true negative. The rows represent the predicted values, thereby making the label the true predicted and the background the false predicted.

The Cartesian product of the predicted true with the true value results in the true positive. According to Figure 11a and Figure 12a, the trained models showed a true positive closer to 100% success rate for Dataset (1). Consequently, this implies lower values of true negative, false positive, and false negative. These results are achieved by the models’ self-assessment, which is based on the confidence scores implemented internally. On the other hand, according to Figure 11b and Figure 12b, it is observed that the true positives are higher values than the errors obtained. However, the number of false positives obtained using the R-CNN model suggests difficulty in some cases in differentiating background from bullet holes.

Given the confusion matrix, accuracy, precision, recall, and F1 score can be calculated [64]. Accuracy, represented by Equation (8), indicates the result’s efficiency. It can be useful in establishing the eligibility of the evaluated model. Equation (9) establishes the precision, which measures the relevancy of results. Recall quantifies truly relevant results and is defined by Equation (10). The F1 score enables evaluating the quality of a classification model, especially when the classes are imbalanced. The F1 score is defined by Equation (11).

$$Accuracy={\displaystyle \frac{TP+TN}{TP+FP+FN+TN}}$$

(8)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(9)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(10)

$$F1 score={\displaystyle \frac{TP}{TP+\frac{1}{2}\left(FP+FN\right)}}$$

(11)

Table 1. Results achieved based on the use of the metrics accuracy, precision, recall, and F1 score.

	Metric	Dataset (1)				Dataset (2)
		Train	Val.	Test	Gen.	Train	Val.	Test	Gen.
YOLOV8	Accuracy	0.995	0.983	1.000	0.993	1.000	0.459	0.463	0.759
	Precision	1.000	0.983	1.000	0.997	1.000	0.873	0.926	0.966
	Recall	0.995	1.000	1.000	0.997	1.000	0.492	0.481	0.780
	F1 score	0.997	0.992	1.000	0.997	1.000	0.629	0.633	0.863
R-CNN	Accuracy	1.000	1.000	1.000	1.000	0.848	0.385	0.500	0.651
	Precision	1.000	1.000	1.000	1.000	0.848	0.640	0.660	0.778
	Recall	1.000	1.000	1.000	1.000	1.000	0.492	0.673	0.799
	F1 score	1.000	1.000	1.000	1.000	0.918	0.556	0.667	0.788

presents results achieved based on the use of the metrics accuracy, precision, recall, and F1 score. The results were obtained for both datasets using the YOLOV8 and R-CNN network models. Table 1 presents the results calculated for training, validation, and test sets, as well as a general result considering all sets. The YOLOV8 model achieved 0.993 and 0.759 accuracy values, respectively, for Dataset (1) and (2). The R-CNN model achieved 1.0 and 0.651 accuracy values, respectively, for Dataset (1) and (2). It can be seen that the YOLOV8 model achieved better results for Dataset (2) when compared to the R-CNN model, which achieved better results for the data from Dataset (1).

Table 2. Accuracy and precision values of the tested models compared to values obtained by other studies.

Authors	Model	Accuracy	Precision
The experiments performed on Dataset (1)	YOLOV8	0.993	0.997
	R-CNN	1.000	1.000
The experiments performed on Dataset (2)	YOLOV8	0.759	0.966
	R-CNN	0.651	0.778
Butt et al. [65]	YOLOV8n	-	0.921
	YOLOV8s	-	0.947
	YOLOV8m	-	0.937
Du et al. [66]	Faster R-CNN	-	0.632
	Series Network	-	0.835
Vilchez and Mauricio [67]	R-CNN	0.976	0.995
Widayaka et al. [68]	Image Processing	0.910	-

presents the accuracy and precision values of each model tested, representing the correct predictions on the used datasets, compared to values obtained by other studies.

5. Conclusions

This paper addresses the use of artificial intelligence to detect bullet holes in surfaces targeted by gunfire as non-destructive testing. For this purpose, the YOLOV8 and R-CNN convolutional neural networks were applied to train image recognition models capable of detecting bullet holes.

This study performed bullet hole detection based on two datasets created under two different scenarios: controlled and non-controlled, Datasets (1) and (2), respectively. When images taken under ideal conditions are used, the detection process tends to achieve significant results. When images taken under non-ideal conditions are used, the detection process tends to achieve moderate results due to the high complexity of the images. This fact reinforces the need to take images under controlled conditions.

It was noticed that the experiments performed achieved accuracy values of 99.3% and 75.9%, respectively, for Dataset (1) and (2) when using the YOLOV8 model; 100% and 65.1%, respectively, for Dataset (1) and (2) when using the R-CNN model. Therefore, the achieved results related to Dataset (1) align well with the results presented by other AI-based tools and the scientific community, such as [65,66,67,68].

It is noteworthy that the high accuracy achieved by bullet hole detections on Dataset (1) suggests that bullet hole evidence was well preserved, and its photographs were taken adequately in most cases.

It is important to note that comparative testing of bullet holes can only be considered non-destructive if the bullet hole and all other evidence have been adequately preserved at the crime scene. In this way, if the forensic expert destroys other evidence while collecting the bullet hole evidence, then the comparative testing of bullet holes is destructive. Therefore, a high level of accuracy in detecting bullet holes does not necessarily mean that it has non-destructive testing, even if the collection of bullet hole evidence itself was properly performed.

Furthermore, the more destructive the testing, the greater the possibility that the crime will not be solved. Therefore, the application of non-destructive testing in Forensic Science (including the field of Forensic Engineering) implies that crimes can be solved correctly and, consequently, justice can be served more often.

In the future, studies should investigate other image preprocessing strategies and check whether they are capable of improving the learning conditions of the models used to detect bullet holes. A future study will describe the second part of the comparative testing of ballistics presented in Figure 1.

This study provides important guidelines that can be applied to a wide range of applications in Forensic Science. Therefore, it is expected that the guidelines described in this study will act as a reference for performing investigations that apply artificial intelligence to improve tests in Forensic Science. It is also expected that this study will contribute to helping (1) the police indict people for their respective crimes committed and (2) the justice system to apply fair sentences to criminals and identify innocent people. In other words, it is expected that artificial intelligence-based, non-destructive testing applied to Forensic Science will provide guidelines to help solve crimes and bring justice around the world.