Calculation of Morphological Characteristic Parameters of Sand Particles Based on Deep Learning

Abstract

For projects such as tailings ponds, slopes, and foundations, loose materials such as rock, slag, and sand, which are composed of particles, often have low cohesion and rely mainly on friction to maintain stability. The shear strength parameters, namely, the internal friction angle and cohesion, are the core parameters that describe the mechanical properties of materials and are directly related to the engineering stability of the above projects. The shear strength properties of loose media are related to the geometric morphological characteristics of particles. Particles with high irregularity will increase the bite and friction of the contact interface between particles, thereby affecting the overall peak shear strength of the material. This study takes sand as the research object. Based on the Mask R-CNN algorithm in deep learning, a sand particle image dataset consisting of single, contact, and sand surface particles is established. An image segmentation model that can identify particles on the surface of the sand layer and obtain the corresponding particle mask is trained; a Python 3.11.4 program is written to automatically calculate seven characteristic parameters of particle morphological characteristics parameters, including the Feret major diameter, the particle Feret minor diameter, the particle aspect ratio, the particle roundness, the comprehensive shape coefficient, the roughness, and the convexity through the particle mask. This method can obtain the overall morphological characteristics of sand particles in real time and is a particle processing method that is a prerequisite for the subsequent rapid prediction of the strength properties of granular materials.

1. Introduction

For projects such as tailings ponds, slopes, and foundations, loose media such as rock blocks, slag, and sand are often involved. These loose rock and soil bodies composed of particles show the characteristics of looseness, compressibility, and low tensile and shear strength (low cohesion) in mechanical properties. Therefore, loose rock and soil bodies are more likely to reach a critical state, that is, the solid–fluid transformation point of granular matter, and be destroyed, which, in turn, leads to geological disasters such as landslides and mudslides [1,2,3,4]. When the loose rock and soil body is in a static state, it exhibits the mechanical properties of solid matter and can stably withstand external loads. It mainly relies on friction, that is, the shear resistance of the material, to maintain stability. Therefore, the shear strength parameters, namely, the internal friction angle and cohesion, are the core parameters that describe the mechanical properties of materials and are directly related to the engineering stability of the above-mentioned projects.

In the existing research on the shear strength of dispersed medium materials, the shape parameters of dispersed medium particles are important factors that affect their shear strength. Some researchers, such as Javier E. Necochea, Esteban Sáez and Kevin J. Hanley [5], directly established spherical particles and ellipsoidal particles through the discrete element method to simulate the direct shear test of granular materials. Zhong Zhou, Zhenxiao Li, Jiuzhou Zhang, et al. [6] designed four simple particle types with different roundness and established corresponding numerical test models. The influence of particle roundness on the macroscopic mechanical properties of sand was studied through triaxial shear simulation tests. Irani N, Salimi M, Golestaneh P, et al. [7] went a step further and established 680 groups of particle models with different structures and aspect ratios for discrete element simulation. Zhou Y [8] and Nie J [9] obtained the real particle morphology based on CT scanning reconstruction technology and introduced the model into the discrete element simulation test. It was found that the real particle morphology can show a stronger shear resistance and dilatancy effect than the traditional spherical unit. The greater the content of irregular particles, the stronger the bite between particles, and the sample thus shows stronger macroscopic shear strength. Another group of researchers started to conduct experiments. Wu J [10] obtained twelve different samples by mixing glass beads, Fujian standard sand and glass sand for triaxial consolidation drainage tests and quantified the aspect ratio, convexity and sphericity of the particles into the overall regularity (OR) index. The test results showed that the shear strength index decreased with the increase in OR and showed an obvious linear trend. Xiao, Y. et al. [11] conducted drainage triaxial tests on sandy soil composed of crushed glass beads (CG) and glass beads (GBs) to study the effects of particle size and particle size shape of different types of sand on the strength and shear dilatancy of soil [12]. The study showed that under the same particle shape, as the average particle size increases, the maximum friction angle and critical friction angle of circular GB sand both increase, while the critical friction angle of angular CG sand tends to decrease.

High-resolution cameras, microscopes, CT, X-ray, convolutional neural networks and other equipment or technology [13,14,15,16] have been commonly used to obtain images of granular materials and calculate high-precision particle morphological parameters. However, these particle image processing methods require that the particles be placed on a monochrome background and that the particles be separated as much as possible. The number of particles that can be processed is small, and the manpower and material resources consumed are high. They cannot be applied to complex environments where particles are stacked in reality. Although some studies have attempted to determine grain size distributions from suboptimal conditioned photos, accurately extracting complex morphological parameters from stacked or naturally lit environments using traditional image processing remains highly challenging [17]. With the development of artificial intelligence technology, image instance segmentation methods based on deep learning technology make it possible to quickly and massively identify particles in complex environments and process them to obtain their morphological information. Existing studies have used instance segmentation technology to obtain particle shapes and calculate the major or minor diameter of particles, and relatively accurate results have been obtained. However, most of these studies using instance segmentation algorithms are limited to particle grading analysis without further detailing particle morphology and forming a mapping relationship with the mechanical behavior of particles [18,19,20,21,22]. In this study, sand was used as the research object, and the Mask R-CNN algorithm model, written based on the Pytorch framework, was used to train an instance segmentation model for sand particles on the sand surface. The sand particle morphological characteristic parameter calculation program was written in Python with the sand surface particle mask map identified by the model as input. The computable morphological parameters include commonly used feature parameters such as the maximum Feret diameter, minimum Feret diameter, aspect ratio, circularity, specific surface area, comprehensive shape factor, roughness, and convexity of the particles. The segmentation model achieves a computational accuracy exceeding 95%. The proposed method integrates instance segmentation with morphological processing to rapidly characterize the fundamental morphological properties of granular materials. This provides an effective particle preprocessing approach to facilitate the subsequent rapid prediction of the strength characteristics of infrastructure—such as gravel roads and granular slopes (e.g., tailings ponds, soil dumps, and waste rock dumps)—as well as the analysis of road trafficability. The overall workflow of the proposed method is illustrated in Figure 1.

2. Morphological Identification and Characteristic Parameter Calculation Method of Sand Particles

2.1. Instance Segmentation Algorithm

With the development of artificial intelligence technology today, deep learning-based instance segmentation algorithms, such as Mask R-CNN, GCNet, PANAT, BlendMask, and YOLO, have their own advantages in recognition accuracy, stability, and speed. Considering that the real-time judgment of the support force of the road ahead when an off-road vehicle is running on sand requires high recognition speed and recognition accuracy, this paper selects the Mask R-CNN algorithm with both recognition speed and recognition accuracy, and the backbone network is modified to Resnet-101 [23] to achieve pixel-level segmentation of sand particles on the road ahead and extract the particle mask map for the next step of particle morphology processing [24].

Mask R-CNN is an instance segmentation algorithm based on the improvement of Faster R-CNN [25,26]. The algorithm framework includes a series of modules, including a backbone convolutional network for extracting image features, a region proposal network for generating candidate regions, a feature alignment module for feature mapping, a fully convolutional neural network for classification, and a mask network for generating accurate target boundaries. The overall network structure is shown in Figure 2. The main difference between Mask R-CNN and Faster R-CNN is that Mask R-CNN replaces the region of interest Pooling in Faster R-CNN with the region of interest Align and adds a mask network branch after the module. In the step of resizing the region of interest image, the feature alignment module of Mask R-CNN uses bilinear interpolation to process the image feature value, which can achieve more accurate feature mapping compared to the feature pooling module of Faster R-CNN, which uses rounding down to process the image feature value. A schematic diagram of the bilinear interpolation method is shown in Figure 3. The calculation formula is shown in Equation (1).

$$f=\left(1-u\right)\left(1-v\right)f_1+u\left(1-v\right)f_2+v\left(1-u\right)f_3+uv{f}_4$$

(1)

The feature map generated by the region proposal network (RPN) is represented by black grid points $f_1$, $f_2$, $f_3$, and $f_4$, with a unit distance between adjacent points. The center of each cell within the orange box corresponds to the target mapped value $f$ after alignment, where $u$ and $v$ denote the horizontal and vertical distances between the mapped point and the black grid points, respectively. The added mask branch enables the algorithm to further segment the target at the pixel level while achieving target detection to obtain the target’s accurate outline.

2.2. Particle Morphology Calculation Method

In different research fields such as medicine [27], materials science [28], geology [29], etc., there have been a lot of studies on particle shape. These studies involve a lot of quantitative particle shape characterization methods [30]. In these studies, due to the difficulties in particle sampling and parameter calculation, relatively few parameters were used to characterize particle morphology. In order to describe the particle morphology as comprehensively as possible, this study selected seven commonly used characteristic parameters, including particle Feret major diameter, particle Feret minor diameter, particle aspect ratio, particle roundness, particle specific surface area, comprehensive shape coefficient, roughness and convexity, for particle morphology description. The definition and calculation method of each characteristic are shown in Equations (2)–(6):

$$F_e={\displaystyle \frac{k_a}{k_b}}$$

(2)

where $k_a$ is the particle max feret diameter, that is, the maximum distance between two points on a straight line passing through the center of the particle and intersecting the particle profile; $k_b$ is the article min feret diameter, that is, the minimum distance between two points on a straight line passing through the center of the particle and intersecting the particle profile; and $F_e$ is the particle aspect ratio, that is, the ratio of the particle’s long diameter to the particle’s short diameter.

$$F_c={\displaystyle \frac{\sqrt{4S\pi }}{L}}$$

(3)

where $S$ is the particle’s projected area; $L$ is the particle projection perimeter; and $F_c$ is the particle roundness. That is, the ratio of the equivalent circumference of the particle’s projected area to the actual projected circumference of the particle.

$$C_x={\displaystyle \frac{S}{S_{out}}}$$

(4)

where $S_{out}$ is the minimum circumscribed polygon area of particle projection and $C_x$ is particle convexity. That is, the ratio of the particle projection area to the minimum circumscribed polygon area of the particle projection.

$$R_x={\displaystyle \frac{L}{L_{out}}}$$

(5)

where $L_{out}$ is the minimum circumscribed polygon perimeter and $R_x$ is particle roughness. That is, the ratio of the particle projection area to the minimum circumscribed polygon area of the particle projection.

$$Z_x={\displaystyle \frac{F_c}{\sqrt{F_e}}}$$

(6)

where $Z_x$ is the comprehensive shape factor. That is, the ratio of particle roundness to the square root of the particle aspect ratio.

Microscopically, variations in these morphological parameters directly influence the interlocking behavior between particles, which macroscopically leads to distinct differences in the shear strength, dilatancy, and other related properties of granular materials [31,32,33].

3. Dataset Preparation

In instance segmentation algorithms, dataset establishment is an important part of implementing segmentation models. The quality and quantity of the datasets directly determine the performance of the trained model.

In order to enable the model to take into account the shape recognition of sand particles and the spatial relationship between sand particles, this study prepared three types of image data: single-particle images, contact particle images, and sand surface images as datasets for model training. The single-particle dataset is a dataset with only one sand particle in the image, and the dataset has about 400 images. The contact particle dataset [34] is a dataset of multiple particles with 2–4 touching or overlapping gravel in the image. The dataset has about 400 images. The sand surface dataset is a dataset consisting of images covered with multiple layers of sand particles. The dataset has about 100 images. The images in the dataset were all taken in the laboratory using a Canon EOS 5D Mark III (Canon Inc., Tokyo, Japan). The distance between the camera and the surface of the sand particles was kept constant, and the shooting light intensity was kept constant. The particle size of the prepared sand particles was in the range of 0.5–5 mm.

The sand particles comprising the training set were sourced from commercially available natural river sand. The predominant colors of these particles are white, yellowish, and yellowish-brown, interspersed with some black, red, and gray particles. During the dataset annotation process, the primary focus was exclusively on the morphological features of the surface particles, disregarding the color variations among them. The sand particles utilized in the dataset are illustrated in Figure 4.

For the image segmentation dataset of sand particles, the irregular shape of sand particles themselves, the contact and occlusion between sand particles and other issues make labeling work time-consuming and labor-intensive. Therefore, in this study, in order to improve the quality and efficiency of particle labeling, the dataset labeling methods include automatic labeling by an algorithm and manual labeling by software.

The single-particle dataset is automatically labeled. During the labeling process, the three channels of the color RGB image are first weighted and converted into a grayscale image. The weighting formula is as follows:

$$Gs=0.2989\times R+0.5870\times G+0.1140\times B$$

(7)

where $Gs$ is the gray value of the converted image; $R$ is the red channel value; $G$ is the green channel value; and $B$ is the blue channel value.

After obtaining the grayscale image, the image is converted into a binary image with a background range of 0 and a particle range of 1 by setting a threshold, namely, the mask image [34]. The segmentation threshold of each single-particle image is automatically calculated by the maximum inter-class variance method implemented in Python, namely, the Otsu algorithm [35]. The algorithm traverses the grayscale value interval, takes a grayscale value as the pre-segmentation threshold each time, and calculates the inter-class variance of the foreground and background after segmentation based on the threshold. The pre-segmentation threshold that maximizes the inter-class variance during the traversal process is taken as the final segmentation threshold. The calculation formula for the inter-class variance of the foreground and background of the image is as follows:

$$g=w_0{\left(u_0-u\right)}^2+w_1{\left(u_1-u\right)}^2$$

(8)

where $g$ is the variance value between image classes after segmentation; $w_0$ is the ratio of the foreground to the image after segmentation; $w_1$ is the ratio of the background to the image after segmentation; $u_0$ is the foreground average grayscale; $u_1$ is the average grayscale of the background after segmentation; and $u$ is the total average grayscale of the image.

The annotation effect and process are shown in Figure 5:

The contact particle dataset and the sand surface dataset were manually annotated with particle outlines and assigned category labels using labelme 5.4.1, a general software program for instance segmentation dataset annotation. During annotation, only particles that were not obscured or particles that only had a small part obscured were annotated. The annotation results are shown in Figure 6 and Figure 7:

The labeled datasets are converted to the Common Objects in Context (COCO) dataset format for model training. The dataset is randomly divided into a training set and a validation set at a ratio of 8:2.

4. Model Training Results

The model was trained for 100 epochs using an NVIDIA GeForce RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA). The initial learning rate was set to 0.004, and was halved every 10 training epochs. The total training time was approximately 32 h. The evolution of the learning rate and loss value over the training epochs is shown in Figure 8.

It can be seen that after about 60 training rounds, the loss value of the training model remains basically unchanged. The final training loss converges to approximately 0.05, while the learning rate approaches zero. The low values of both metrics indicate that the model training has stabilized. The recognition and segmentation results obtained by the test are shown in Figure 9 and Figure 10:

The segmentation results demonstrate that the trained Mask R-CNN sand particle segmentation model successfully extracts the vast majority of surface particles using masks while effectively excluding underlying occluded particles. This performance aligns perfectly with the expected outcomes of the model.

In this study, four groups of graded sand particles (1 kg per group) were configured to collect surface images of sand particles. The mass percentage of each group after mechanical screening is shown in

Table 1. Sand particle ratio table.

Group	1–2.5 mm	2.5–5 mm	5–10 mm
1	20%	60%	20%
2	60%	20%	20%
3	40%	60%	0
4	60%	40%	0

:

The surface photos of sand with different gradations were put into the model for particle prediction, and the image size was controlled to 900 × 1000. The average inference time required for each image and the average number of particles obtained by segmentation are shown in

Table 2. Inference time and identification particle number.

Group	Time/s	Number
1	0.0622	23.52
2	0.2336	46.18
3	0.2508	39.81
4	0.2358	50.44

:

4.1. Model Quality Verification

In order to understand the specific accuracy of the training model, it is necessary to verify the particle images obtained by model segmentation. Since the contact and sand surface dataset are manually labeled, there must be subjective errors in the labeling results. In addition, the case segmentation model in this study was trained to obtain as many sand particles as possible and their exact morphological parameters. Therefore, the model was verified by comparing the sand particle mask obtained by model segmentation with the sand particle mask obtained by monochromatic background particle segmentation using the Otsu algorithm. The verification steps are as follows:

(a)

Single-particle images of 10 sand particles are collected, and their binary mask images are calculated and generated.

(b)

These particles are randomly placed on the surface of the sand, and each particle is placed so that its two-dimensional projection shape is consistent with the single-particle image.

(c)

The sand surface is photographed to obtain image data, and the image data is input into the trained segmentation model to obtain the mask image of the sand particles predicted by the model.

(d)

The particle max feret diameter, min feret diameter, perimeter and area values of the mask images obtained by the two methods are calculated and compared, and the relative error is calculated.

The specific particle tracking approach and the corresponding recognition results are illustrated in Figure 11 and Figure 12, respectively. The calculation results of the relative errors of ten particle masks are shown in

Table 3. Relative error table of the sand particle segmentation model.

Feature	$\mathit{S}$	$\mathit{L}$	${\mathit{k}}_{\mathit{a}}$	${\mathit{k}}_{\mathit{b}}$
error	3.80%	1.74%	2.44%	3.50%

:

It can be seen from Table 3 that the average relative errors of the four predicted values of area, perimeter, max feret diameter and min feret diameter are all within 5%. That is, the prediction accuracy is higher than 95%, and the model’s prediction accuracy is high.

4.2. Calculation of Morphological Characteristics of Sand Particles

When there are enough identified particles, the morphological characteristics of the sand particles obtained by segmentation can represent the overall morphological characteristics of the sand. Therefore, photos were taken at multiple locations on the sand surface, and the middle part with good focus was taken as the input image of the instance segmentation model. After obtaining the segmentation mask of the sand particles, their morphological characteristic parameters were calculated one by one, the characteristic values of all the identified particles were summarized, and the mean was calculated.

To ensure the accuracy of the analysis, it is necessary to confirm the minimum number of particles required for the analysis. Therefore, four groups of graded sand particles were photographed and sampled, and the surface particle masks were obtained by inputting the trained segmentation model. The characteristic parameters of the particle group were calculated under different mask numbers. Taking the Feret long diameter and Feret short diameter as examples, the curve of the change rate of the average Feret diameter with the change in the number of particles was plotted, as shown in Figure 13.

As shown in Figure 13a–d, after the number of particles in these four groups reaches 450, the rate of change in the average Feret diameter with the number of particles is less than 1%. The Canon Mark III 5D used for image sampling in this study obtains an image size of 3840 × 3840 each time it takes a photo. After cropping the area with poor edge focus, the image size is 2700 × 3000. As shown in Table 2, in group 1, with the fewest number of particles to be identified, a photo with a size of 2700 × 3000 can identify about 207 particles. Therefore, each time the sand surface is photographed and sampled, at least three photos need to be taken for calculation.

Under 450 particles, the mean values of particle characteristic values of different grading groups are calculated, as shown in

Table 4. Sand particle characteristic parameter calculation 1.

Group	${\mathit{k}}_{\mathit{a}}$/mm	${\mathit{k}}_{\mathit{b}}$/mm	${\mathit{F}}_{\mathit{e}}$	${\mathit{F}}_{\mathit{c}}$
group 1	3.68	5.49	1.52	0.0311
group 2	2.32	3.50	1.54	0.0495
group 3	2.92	4.32	1.51	0.0388
group 4	2.44	3.65	1.53	0.0456

and

Table 5. Sand particle characteristic parameter calculation 2.

Group	${\mathit{Z}}_{\mathit{x}}$	${\mathit{R}}_{\mathit{x}}$	${\mathit{C}}_{\mathit{x}}$
group 1	0.0257	33.09	0.983
group 2	0.0407	21.15	0.990
group 3	0.0319	26.21	0.985
group 4	0.0373	22.18	0.989

:

In summary, the image analysis technology based on the Mask R-CNN algorithm in deep learning can quickly measure the shape characteristic parameters of particle groups, such as Feret’s long diameter, particle Feret’s short diameter, particle aspect ratio, particle roundness, comprehensive shape coefficient, roughness and convexity.

5. Conclusions

Based on the Mask R-CNN instance segmentation algorithm in deep learning, this paper has written a Python program that can independently realize the accurate and rapid recognition and segmentation of sand particles in sandy soil roads and calculate the shape characteristic parameters of the segmented sand particles, such as the Feret major diameter, Feret minor diameter, aspect ratio, particle roundness, comprehensive shape coefficient, roughness and convexity. The study has drawn the following conclusions:

(1)

The average inference time required for photos with a size of 900 × 1000 is related to the average number of sand particles identified and the particle gradation. For photos of different gradations, the maximum inference time required is 0.25 s, and the minimum is 0.08 s. The maximum number of identified particles is 50.4, and the minimum is 23.5.

(2)

By comparing the mask image obtained by predicting the same particle on the sand surface with the mask image calculated by the Otsu algorithm under the single-particle situation, the relative errors of the four predicted values are all within 5%. That is, the prediction accuracy is higher than 95%, and the model’s prediction accuracy is high.

(3)

When the number of identified particles reaches 450, the characteristic value of the particle group changes basically steadily, with a change rate of less than 1%. The number of images required to identify the required number of particles is three.

This method, which combines instance segmentation with morphological processing, can quickly describe the overall morphological characteristics of granular materials and provide a preliminary particle processing method for the subsequent rapid prediction of the strength properties of granular materials.