CNN-MLP-Based Configurable Robotic Arm for Smart Agriculture

Abstract

Amidst escalating global populations and dwindling arable lands, enhancing agricultural productivity and sustainability is imperative. Addressing the inefficiencies of traditional agriculture, which struggles to meet the demands of large-scale production, this paper introduces a highly configurable smart agricultural robotic arm system (CARA), engineered using convolutional neural networks and multilayer perceptron. CARA integrates a highly configurable robotic arm, an image acquisition module, and a deep processing center, embodying the convergence of advanced robotics and artificial intelligence to facilitate precise and efficient agricultural tasks including harvesting, pesticide application, and crop inspection. Rigorous experimental validations confirm that the system significantly enhances operational efficiency, adapts seamlessly to diverse agricultural contexts, and bolsters the precision and sustainability of farming practices. This study not only underscores the vital role of intelligent automation in modern agriculture but also sets a precedent for future agricultural innovations.

1. Introduction

The agricultural sector is facing unprecedented challenges due to the increasing global population and the relative decline in arable land. To meet the growing demand for food, enhancing agricultural productivity and sustainability is imperative [1,2,3]. Traditional agriculture, which heavily relies on manual labor, is inefficient and cannot meet the demands of large-scale production. Agriculture is rapidly transitioning towards automation and intelligence, driven by technological innovations [4,5].

Smart agriculture, which is at the forefront of the agricultural technological revolution, utilizes advanced information and robotics technologies to achieve precision management and automation in agricultural production [6,7]. Smart agriculture integrates the Internet of Things (IoT), big data, and artificial intelligence to monitor crop growth, fertilize precisely, irrigate automatically, and control pests [8,9].

Robotic arms are a critical component of automation technology and play a vital role in intelligent agriculture. Robots can execute precise tasks such as plant picking, fruit sorting, and packaging with ease [10,11,12]. They significantly reduce manual labor and production costs while enhancing accuracy and efficiency [13,14,15]. However, accomplishing these complex tasks requires more than mere mechanical operations. It necessitates the support of advanced artificial intelligence technologies like deep learning to endow robotic arms with visual recognition and decision-making capabilities [16,17].

The integration of convolutional neural networks (CNN) and multilayer perceptron (MLP) in deep learning has demonstrated remarkable capabilities in image recognition and processing [18,19,20,21,22]. In the context of intelligent agriculture, these technologies empower robotic arms to accurately identify various crops and their growth stages, facilitating precise operations on specific crops. To deploy agricultural technologies effectively, a comprehensive system framework that integrates robotic arm hardware and deep learning software is necessary [4,23,24].

This paper introduces a novel configurable agricultural smart robotic arm system (CARA), which incorporates a highly configurable robotic arm (HCRA), image acquisition module (IAM), and deep processing center (DPC). The CARA system utilizes a deep learning model that combines CNN with MLP to autonomously perform agricultural tasks, including picking, pesticide spraying, and crop inspection. It is highly adaptable and can be quickly configured to meet the specific needs of different crops and operational environments. The CARA system has demonstrated significant potential and practical value in intelligent agricultural production through extensive research and experimental validation. This is a significant step towards more efficient, precise, and sustainable agricultural development.

2. Materials and Methods

The CARA is predicated on flexibility and adaptability, enabling it to perform a variety of tasks with precision and efficiency. As depicted in Figure 1a, the CARA is principally composed of three key components: the IAM, the HCRA, and the DPC. The CARA operates in the primary processing stage of agricultural produce, handling tasks such as the picking and bagging of fruits, and pesticide spraying on vegetables. At the heart of CARA’s hardware configuration lie the HCRA and the IAM (Figure 1b), with the software component being constituted by the DPC. This section provides a comprehensive overview of the design and functionality employed in the development of the HCRA, the design and integration of the DPC with the IAM, and the holistic experimental deployment of CARA. Each selected component and strategic decision have been elaborately detailed to underscore their substantial contributions to the system’s configurability and operational efficiency. Figure 1c shows the actual demonstration of tree pest and disease detection.

2.1. Structure and Configuration of HCRA

To achieve success in smart agriculture, selecting a lightweight and highly configurable robotic arm is crucial [25,26,27,28,29]. As depicted in Figure 1b, the arm comprises rigid links, an end effector, and joints. The mechanical structure of the robotic arm is composed of rigid links that are connected by joints. It is important to note that the system’s end effector is highly configurable and can perform a multitude of intelligent tasks necessary for farm operations. The programming pendant enables visual control of the robotic arm and provides software requirements for external control.

Table 1. Hardware parameters of robotic arm.

Parameter	Value
Degrees of freedom (DOF)	6
Maximum operational radius (mm)	625
Weight (kg)	16
Load (kg)	3
Mounting diameter (mm)	φ140
Repeatability (mm)	±0.02
End effector velocity (m/s)	≤1.9
Rated power (W)	150
Peak power (W)	1000
Working temperature range (°C)	0–50
Working environment humidity	25–90%
Insulation method	Arbitrary angle
Protection level	IP54

and

Table 2. Parameters of each joint of the robotic arm.

Joint Motion Parameters	Motion Limit (°)	Maximum Angular Velocity (°/s)
Joint 1	±360	178°/s
Joint 2	±360	178°/s
Joint 3	±360	178°/s
Joint 4	±360	237°/s
Joint 5	±360	237°/s
Joint 6	±360	237°/s

summarize the physical attributes of the robotic arm, which has been designed to meet the agility and configurability required for different farm work environments. The maximum velocity of the robotic arm’s end effector represents the theoretical peak angular speed achievable under optimal conditions, distinct from the commonly discussed operational angular velocities which are subject to various practical constraints.

The incorporation of a six-degrees-of-freedom (6-DOF) robotic arm into smart agriculture capitalizes on its established utility across diverse industrial applications. Characterized by its comprehensive validation in multifaceted operational settings, the 6-DOF system excels in executing intricate and precision-driven tasks. Its adoption reflects a commitment to the principles of standardization that resonate with the prevailing industrial automation ecosystem, ensuring a high degree of reliability and adaptability essential for the demanding environments of intelligent agricultural operations. The prevalent use of this configuration in industry highlights its capabilities for exceptional repeatability and accuracy, critical attributes that address the stringent requirements of agricultural productivity and efficiency. Consequently, employing a 6-DOF robotic arm in our system is a strategic choice, aimed at harnessing a well-established mechanical infrastructure to augment the technological sophistication and operational efficacy of agricultural automation initiatives.

The joints of the robotic arm are designed with the capability to rotate ±360 degrees. However, the achievable range of motion may be restricted based on the application environment. Specifically, in standard operational settings, limitations are typically observed in the second, third, and fifth degrees of freedom. This discussion is confined to describing the basic structural and performance characteristics of the robotic arm without delving into the nuances of environmental constraints on its operation. The focus here is to outline the core capabilities as a baseline, with empirical assessments provided to substantiate the mechanical arm’s foundational configurations, ensuring clarity in its basic operational parameters without considering the variability introduced by different deployment contexts.

Figure 2a displays a detailed dimensional model of the Highly Configurable Robotic Arm (HCRA), providing a granular view of the arm’s precise geometrical specifications. To achieve a comprehensive understanding of the arm’s structural and mechanical performance, a high level of detail is required. The HCRA has been modeled in SolidWorks and Adams, as shown in Figure 2b,c. This modeling represents the initial step towards iterative design improvements and functional enhancements, laying a solid foundation for the continuous development of the robotic arm’s capabilities. The HCRA’s workspace, as shown in Figure 2d, optimizes coverage and maneuverability for agricultural tasks. Additionally, Figure 2e displays the arm in various configurations, demonstrating the HCRA’s versatility and adaptability to diverse operational demands encountered in agricultural settings.

In discussions of the mathematics underlying a robotic arm, the focus typically centers on kinematics, dynamics, and control strategies [30,31,32,33,34,35,36,37]. Among these, the Denavit–Hartenberg (D-H) parameter transformation matrix serves as a method to describe the relative position and orientation of links and joints. This method utilizes four parameters: the link length $a_i$, the link twist angle ${\alpha }_i$, the link offset $d_i$, and the joint angle ${\theta }_i$. For the transformation from one link to the next, the transformation matrix $T_i$ can be expressed Equation (1).

$$T_i=\left[\begin{array}{cccc}\cos {\theta }_i& -\sin {\theta }_i\cos {\alpha }_i& \sin {\theta }_i\sin {\alpha }_i& a_i\cos {\theta }_i\\ \sin {\theta }_i& \cos {\theta }_i\cos {\alpha }_i& -\cos {\theta }_i\sin {\alpha }_i& a_i\sin {\theta }_i\\ 0& \sin {\alpha }_i& \cos {\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right].$$

(1)

The forward kinematics equations of a robotic arm are employed to compute the position and orientation of the end effector based on given joint angles. If the robotic arm has $n$ joints, its forward kinematics equation can be represented by the multiplication of the transformation matrices of all joints as Equation (2):

$$T=T_1\cdot T_2\cdot \cdot \cdot T_n.$$

(2)

This resulting matrix $T$ describes the position and orientation of the end effector relative to the base frame.

The Jacobian matrix $J$ delineates the relationship between the velocity of the end effector and the joint velocities. For a six-degree-of-freedom robotic arm, the Jacobian matrix can be divided into the linear velocity part $J_v$ and the angular velocity part $J_{\omega }$ as Equation (3):

$$J=\left[\begin{array}{c}{J}_v\\ J_{\omega }\end{array}\right].$$

(3)

Here, $J_v$ and $J_{\omega }$ correspond to the dependencies of the end effector’s linear and angular velocities on the joint velocities, respectively.

PID (Proportional–Integral–Derivative) control is a commonly used control strategy for motion control of robotic arms. The objective of a PID controller is to reduce the discrepancy between the target and actual positions, and its control law can be expressed as Equation (4):

$$u\left(t\right)=K_{p}e\left(t\right)+K_i\int e\left(t\right)dt+K_d{\displaystyle \frac{de\left(t\right)}{dt}}$$

(4)

Here, $u\left(t\right)$ represents the control input (such as force or torque), $e\left(t\right)$ is the position error, and $K_p$, $K_i$, and $K_d$ are the proportional, integral, and derivative gains, respectively.

Due to the maturity and ubiquity of Proportional–Integral–Derivative (PID) control in automation, this study does not specifically test or extensively discuss PID performance. PID controllers, known for their robustness and effectiveness in maintaining system stability and optimizing performance, are well-documented across various applications. Numerous studies have validated their efficiency in real-world scenarios, establishing a solid foundation of understanding. Therefore, detailed discussions on PID tuning are considered unnecessary, as the focus is on presenting innovative advancements beyond this conventional control technique [38,39].

In conclusion, the HCRA will improve operational efficiency in smart agriculture, representing significant progress in automating complex and varied farm tasks. The deployment of this innovative technology demonstrates a strong commitment to enhancing productivity through intelligent mechanization, thereby increasing the precision and sustainability of agricultural practices.

2.2. Structure and Configuration of IAM and DPC

The CARA framework heavily relies on IAM for its visual perception capabilities. An efficient IAM module has been designed and implemented within the framework, as demonstrated in Figure 1b The module includes function buttons, switches, cameras, control circuits, and interfaces. This design optimizes the integration and layout of hardware components. The comprehensive and efficient control circuit ensures precise image acquisition capabilities. The intuitive and convenient interface provided by function buttons and switches allows users to activate the image acquisition process with ease. As the core component of image acquisition, camera captures visual information within the mechanical working environment. Camera captures signals that control circuit processes. The image acquisition process is optimized through pre-assembled and pre-programmed algorithms, ensuring high-quality and efficient transmission of image data. The IAM interface effectively transmits the image data to the DPC, allowing flexible deployment on computers or deep learning development boards. The DPC utilizes a highly efficient deep learning model that seamlessly integrates CNN (Figure 3a) and MLP (Figure 3b) to accurately process and recognize visual information of targets in even the most complex working environments. The DPC utilizes a highly efficient deep learning model that seamlessly integrates CNN and MLP to accurately process and recognize visual information of targets in even the most complex working environments. This model has been specifically designed to significantly enhance the end effector’s capabilities and performance. This deep learning model effectively extracts features from collected images and achieves high-precision target recognition by learning from a large amount of training data.

The proposed hybrid model integrates the feature extraction capabilities of CNN with the spatial and sequential processing strengths of MLP through MLPMixer and Permutator modules. Utilizing the Visual Geometry Group Network (VGG) for initial feature extraction, the architecture employs convolutional layers and Region of Interest (RoI) pooling to generate fixed-size feature representations, irrespective of object scales. Diverging from traditional CNN paths, this model introduces a bifurcation in which feature maps are concurrently processed by the MLPMixer, which reorganizes features across spatial dimensions to enhance global image context comprehension, and the Permutator, which applies a segment-wise permutation strategy to improve integration and interpretation of feature segments. This dual approach enables a deeper understanding of the interrelationships among input space components, significantly refining the model’s analytical capabilities [40,41,42,43,44,45].

This approach to feature processing involves two pathways that enhance the model’s representational capacity and flexibility in handling complex spatial relationships and feature interactions. The architecture consolidates the features from both pathways after processing through the MLP. These features are then harmonized through layer normalization and dimensionality reduction techniques. This consolidation unifies the diverse and rich feature sets into a coherent representation that is suitable for the final prediction stages.

The model’s forward pass directs the unified feature representation into dedicated heads for classification and bounding box regression. The components output final object class predictions and precise bounding box coordinates, utilizing the enriched feature set produced by the hybrid architecture. The design balances the depth and breadth of feature processing, ensuring adequate capture and utilization of both local and global image contexts for object detection tasks.

To provide a detailed and academic exposition of the mathematical principles underpinning CNN and MLP, we delve into the foundational operations, transformations, and learning mechanisms that characterize these two pivotal architectures in the domain of deep learning.

The convolution operation is central to a CNN’s functionality. For an input $\mathit{X}$ and a filter $\mathit{F}$, the convolution operation $\left(\mathit{i}\mathbf{,}\mathit{j}\right)$ at position $\left(\mathit{X}\mathbf{\ast }\mathit{F}\right)\left(\mathit{i}\mathbf{,}\mathit{j}\right)$ is computed as Equation (5).

$$\mathit{S}\left(\mathit{i}\mathbf{,}\mathit{j}\right)=\left(\mathit{X}\mathbf{\ast }\mathit{F}\right)\left(\mathit{i}\mathbf{,}\mathit{j}\right)={\mathbf{\sum }}_{\mathit{m}}{\mathbf{\sum }}_{\mathit{n}}\left(\mathit{i}+\mathit{m}\mathbf{,}\mathit{j}+\mathit{n}\right)\mathit{F}\left(\mathit{m}\mathbf{,}\mathit{n}\right).$$

(5)

This equation implies that for each position $\left(\mathit{i}\mathbf{,}\mathit{j}\right)$, the output $\mathit{S}$ is the sum of element-wise products of the filter $\mathit{F}$ with the input $\mathit{X}$. This operation is repeated across the entire input, producing a feature map that highlights the presence of features the filter is designed to detect.

After each convolution operation, the feature map is passed through an activation function to introduce non-linearity. The Rectified Linear Unit (ReLU) activation function is defined as Equation (6).

$$\mathbf{ReLU}\left(\mathit{x}\right)=\mathit{m}\mathit{a}\mathit{x}\left(\mathbf{0}\mathbf{,}\mathit{x}\right)\mathbf{.}$$

(6)

This function retains only positive values and sets all negative values to zero. It is essential for enabling the network to model complex functions and capture patterns like edges and corners in images.

To reduce the spatial dimensions of the feature map and to make the representation smaller and more manageable, a pooling layer is used. The max pooling operation is defined as Equation (7).

$$\mathit{P}\left(\mathit{i}\mathbf{,}\mathit{j}\right)=\underset{\mathit{k}\mathbf{,}\mathit{l}\mathbf{\in } \mathbf{window}}{\mathit{m}\mathit{a}\mathit{x}}\mathit{X}\left(\mathit{i}\mathbf{\cdot }\mathit{s}+\mathit{k}\mathbf{,}\mathit{j}\mathbf{\cdot }\mathit{s}+\mathit{l}\right)\mathbf{.}$$

(7)

This equation selects the maximum value in a specified window of the feature map, effectively downsampling the feature map to retain only the most significant features, thereby providing the network with translational invariance.

In an MLP, each neuron in a layer $\mathit{l}$ computes an output based on its input weights $\mathit{w}$, input features $\mathit{x}$, and bias $\mathit{b}$. The operation is defined by Equation (8):

$${\mathit{y}}^{\left(\mathit{l}\right)}=\mathit{\sigma }\left(\mathbf{\sum }{\mathit{w}}_{\mathit{i}}^{\left(\mathit{l}\right)}{\mathit{x}}_{\mathit{i}}+{\mathit{b}}^{\left(\mathit{l}\right)}\right)\mathbf{.}$$

(8)

This equation represents the weighted sum of the inputs plus a bias term, passed through a non-linear activation function $\mathit{\sigma }$, which is typically a sigmoid or ReLU function. This step allows the MLP to capture complex relationships between inputs and outputs.

The overall output of an MLP is obtained by propagating the input through multiple layers, each composed of neurons as described in Equation (8). The final output for a network with $\mathit{L}$ layers is Equation (9).

$${\mathit{O}}^{\left(\mathit{L}\right)}=\mathit{\sigma }\left({\mathit{W}}^{\left(\mathit{L}\right)}\mathit{\sigma }\left(\mathbf{\dots }\mathit{\sigma }\left({\mathit{W}}^{\left(\mathbf{2}\right)}\mathit{\sigma }\left({\mathit{W}}^{\left(\mathbf{1}\right)}\mathit{X}+{\mathit{B}}^{\left(\mathbf{1}\right)}\right)+{\mathit{B}}^{\left(\mathbf{2}\right)}\right)\mathbf{\dots }\right)+{\mathit{B}}^{\left(\mathit{L}\right)}\right)\mathbf{.}$$

(9)

This recursive equation represents the composition of functions across multiple layers, transforming the input $\mathit{X}$ into a final output ${\mathit{O}}^{\left(\mathit{L}\right)}$ through successive applications of linear and non-linear operations.

Learning in neural networks is accomplished by adjusting the weights $\mathit{W}$ to minimize a loss function $\mathcal{L}$. The update rule for weights in gradient descent is defined by Equation (10).

$${\mathit{W}}_{\mathrm{new}}^{\left(\mathit{l}\right)}={\mathit{W}}^{\left(\mathit{l}\right)}\mathbf{-}\mathit{\eta }{\displaystyle \frac{\mathit{\partial }\mathcal{L}}{\mathit{\partial }{\mathit{W}}^{\left(\mathit{l}\right)}}}.$$

(10)

Similarly, the biases $\mathit{B}$ are updated by Equation (11).

$${\mathit{B}}_{\mathrm{new}}^{\left(\mathit{l}\right)}={\mathit{B}}^{\left(\mathit{l}\right)}\mathbf{-}\mathit{\eta }{\displaystyle \frac{\mathit{\partial }\mathcal{L}}{\mathit{\partial }{\mathit{B}}^{\left(\mathit{l}\right)}}}.$$

(11)

Both Equations (10) and (11) describe how the parameters of the network are adjusted in the direction that most reduces the loss. The partial derivatives ${\displaystyle \frac{\mathit{\partial }\mathcal{L}}{\mathit{\partial }{\mathit{W}}^{\left(\mathit{l}\right)}}}$ and ${\displaystyle \frac{\mathit{\partial }\mathcal{L}}{\mathit{\partial }{\mathit{B}}^{\left(\mathit{l}\right)}}}$ represent the gradient of the loss function with respect to the weights and η is the learning rate.

The equations encapsulate the essence of CNNs and MLPs, where convolution operations extract spatial features, activation functions introduce non-linearity, pooling layers downsample data, and neurons in MLPs build complex mappings from inputs to outputs, all iteratively refined through backpropagation to minimize a loss function.

2.3. Experimental Scheme

Tomatoes, apples, oranges, pears, and corn are crucial components of the CARA identification, processing, and harvesting experimental deployment, as depicted in Figure 4. CARA is a highly integrated structure consisting of IAM, DPC, and HARC. The IAM and HARC are indispensable hardware elements of the system. The DPC expertly coordinates the multitude of images captured by IAM across various HARC positions, executing a range of intricate tasks. Hardware-software integration is essential for accurately harvesting, processing, and identifying a variety of agricultural produce, such as vegetables, fruits, and cereal crops.

The IAM, located at the extremity of HARC, is equipped with real-time image acquisition capabilities. It interfaces with the DPC, which processes the collected image signals and dispatches behavioral commands to HARC for various processing tasks. The system’s capacity to smoothly switch between soft and rigid robotic claws to meet the harvesting needs of various fruits and vegetables is demonstrated in Figure 4b,c. This modular approach enables customization of the mechanical arm for different agricultural tasks and reduces the time and resources required for retrofitting machinery for different crop types.

Figure 4d demonstrates the successful completion of a fruit-grasping task by a soft robotic gripper developed by the Intelligent Sensing Laboratory at China Agricultural University’s College of Engineering. The gripper, made of liquid silicone casting, is mounted at the end-effector interface of the HARC. The IAM–DPC communication, with a power rating of 2 watts, enables IAM to transmit image sampling data to the DPC every 0.2 s for image processing, storage, and command execution.

In response to the observations raised, it is imperative to elucidate the operational dynamics between the neural network and the robotic manipulator within our system. The primary function of the neural network, beyond mere recognition of tomatoes or other fruits, extends to assessing their suitability for grasping based on size, orientation, and ripeness. Critically, the network determines the spatial positioning of the target relative to the gripper, an essential step for facilitating precise manipulation.

To bridge the gap between image recognition and physical action, our system leverages the Robot Operating System (ROS), a robust middleware renowned for its utility in robotics applications. ROS facilitates a seamless integration of computer vision and control algorithms, enabling real-time communication between the neural network and the robotic arm. This integration is crucial for translating the spatial coordinates identified by the neural network into actionable commands that guide the gripper’s movements. By employing ROS, we ensure that the gripper approaches, aligns with, and securely grasps the fruit, accomplishing this with a level of precision that manual control could not easily replicate.

This system architecture not only enhances the accuracy of the grasping process but also exemplifies the practical application of combining advanced neural network processing with traditional robotic control to achieve efficient automated agricultural tasks.

Analyzing the motion of the robotic arm in the experimental scenario improves its structure for enhanced stability during operation. The completeness and efficiency of image acquisition, as well as the accuracy of the CNN–MLP algorithm in processing images, are thoroughly evaluated through IAM and DPC performances. Finally, CARA’s ability to perform processing tasks is evaluated by placing various experimental materials on the table and testing the arm’s grasping response to commands, image processing, and arm motion. This sequence of experiments measures the system’s operational integrity, high degree of configurability, and component synergy. This assessment determines the system’s ability to operate autonomously with minimal human intervention and complete tasks accurately. Feedback on these outcomes is critical for the precision and reliability of CARA during operation, which is essential in the field of modern precision agriculture.

3. Results and Discussion

In the context of experimental scenarios, this section offers a detailed analysis and discussion on the performance of the robotic arm within a simulation environment, the efficiency of the CNN-MLP model, and the capability of the CARA system in processing various agricultural products.

3.1. Performance Analysis of the HARC

To optimize the design of a robotic arm for processing vegetables, fruits, and grain crops, it is crucial to conduct a performance analysis. This analysis helps with necessary evaluations throughout the research process. Tests conducted within a simulation environment can significantly reduce costs by eliminating the need for physical prototypes or testing apparatuses. To obtain accurate answers with minimal risk and loss associated with physical testing, only the robotic arm modeling and necessary parameter measurements are required. The simulation environment used in the testing process allows for rapid changes to the design parameters of the robotic arm, enabling the identification of the optimal design. This flexibility is difficult to achieve in real-world settings. Simulations can replicate dangerous or extreme operational scenarios that may not be safely achievable in real-world situations. They also allow for extensive and repeated testing of various working environments and conditions ensuring statistical reliability and robustness. This is often not feasible in terms of time and cost in actual settings.

The experiment modeled the robotic arm using Adams2020 simulation software, incorporating its actual dimensions and mass to provide reliable and comprehensive simulation data. The collected data on the relative angular acceleration of each joint of the robotic arm under a dynamic friction coefficient of 5 are shown in Figure 5a. The analysis was limited to the five movable joints as a movable joint was not used on the actuator’s end during deployment. The robotic arm demonstrated exceptional stability throughout the process, exhibiting minimal relative angular acceleration. The force changes over time for the five different joints studied are displayed in Figure 5b through five distinct curves. These forces represent the loads that the robotic arm must overcome when it is unloaded. The torque changes in the joints of the unloaded robotic arm are stable and consistent, with no significant fluctuations. Figure 5c displays the change in torque over time, with five curves corresponding to the torque on different joints. Torque is crucial for ensuring precise and efficient movements of the robotic arm. Furthermore, the variation in torque for each joint remains consistently stable, with no significant changes in numerical values.

In summary, these charts demonstrate the precise kinematic and dynamic performance of the robotic arm under specific dynamic friction coefficients. The consistent or slightly fluctuating curves indicate that the forces and torques are stable and controllable during operation. These data are crucial for evaluating the effectiveness of the robotic arm in different applications.

3.2. Performance Analysis of DPC

In accordance with experimental paradigms, a stringent analysis of the CNN coupled with the MLP model’s performance has been conducted. Images harvested by the IAM were leveraged and coupled with Class Activation Mapping (CAM) heatmap visualization, image augmentation techniques, and loss metric analysis, to assess the overall efficacy of the DPC integrated with CNN + MLP within the CARA framework.

The employment of CAM heatmaps furnishes an attention visualization of regions within the input image that significantly influences the convolutional neural network’s output. This analytical process is integral to comprehending the model’s focal points, ensuring that they genuinely concentrate on pertinent features for accurate prediction. Figure 6a depicts the CAM heatmap visualization analysis using tomatoes as a case study. The CAM heatmaps overlaid on tomato images elucidate the model’s perceived key areas for identifying and classifying objects across various scenarios. The distinct features affirm the viability of the DPC in the image processing stage within CARA. Notably, as illustrated in Figure 6b, image augmentation plays an indispensable role in the model’s robustness. By artificially expanding the dataset with transformed versions of the input images comprising flips, affine transformations, contrast adjustments, rotations, and brightness modifications diversity is infused into the training regimen. This augmentation process exemplified with tomatoes expanded the original dataset from 800 to over 4000 images, mitigating overfitting and enhancing the model’s generalization capability. The original and augmented images contribute to presenting the model with assorted perspectives, simulating the plethora of environmental conditions potentially encountered in practical applications—an essential measure to heighten IAM + DPC performance.

In the evaluation of the DPC, a critical component of our dataset collection involved simulating real-world agricultural challenges to test the robustness of the system under diverse and adverse conditions. The dataset included scenarios with occluded branches and foliage, overlapping fruits, and varying degrees of sunlight interference, which are commonplace in dynamic agricultural environments. To enhance the complexity and realism of the experimental settings, strategic modifications such as the introduction of artificial occlusions and adjustments in image clarity were employed. These modifications were designed to mimic natural obstructions and visual impairments that typically occur in field conditions. This approach not only tested the DPC’s capability to process and analyze images under suboptimal conditions but also helped in developing a more resilient model by exposing it to a wider array of variables that affect visual perception in agricultural operations. The enriched dataset thereby facilitated a comprehensive assessment of the DPC’s performance, ensuring its effectiveness in real-world agricultural applications where such challenges are prevalent.

The analytical procedure of loss involves two principal metrics: classification loss (Figure 6c) and bounding box regression loss (Figure 6d). Classification loss appraises the model’s precision in discerning the correct object categories within the image while bounding box regression loss quantifies the accuracy in object localization. As depicted, the convergence of these two metrics evidences the model’s learning trajectory. The steady decrement in loss values across epochs signifies the model’s escalating adeptness in distinguishing and accurately positioning target objects.

The training environment for the CNN–MLP model, integral to this investigation, was configured with a robust hardware suite to ensure optimal performance and reproducibility of results. The computational core of the system consisted of an Intel i9-12900 Central Processing Unit (CPU), which operates at a frequency of 3.19 GHz, providing the necessary processing power for managing the computationally intensive tasks associated with neural network training. This was supplemented by an NVIDIA GeForce RTX 4090 Graphics Processing Unit (GPU) equipped with 24 GB of video memory, offering expansive parallel processing capabilities that are crucial for decreasing the training durations of complex deep learning models.

The setup was further enhanced with 64 GB of Random Access Memory (RAM) under a Windows 10 operating system, which facilitated the efficient handling of large-scale data operations required by advanced deep learning frameworks. PyTorch 1.12 was selected as the deep learning framework due to its flexibility and dynamic computation features, supported by an active community. The Compute Unified Device Architecture (CUDA) 11.7 was employed to maximize GPU efficiency, facilitating more effective numerical computations and faster iteration of models.

All coding and script executions for the model were conducted in the Visual Studio Code (VSCode) Integrated Development Environment (IDE), using Python 3.8. This environment was chosen for its extensive support for multiple programming extensions and tools, enhancing coding efficiency and simplifying the model development process. This carefully engineered computational framework supports the rigorous demands of the CNN-MLP model training, ensuring comprehensive utilization of available computational resources to fulfill the research objectives effectively.

The analyses of CAM heatmap scrutiny, the image augmentation process, and the diminishing trend in loss metrics collectively facilitate a comprehensive performance evaluation. The dissected descent in loss illustrates enhancements in the model’s competence to categorize and precisely situate bounding boxes around objects of interest. This decline in loss is corroborated by the CAM heatmaps, wherein focal points are aligned with pivotal areas of interest in the images, signifying the model’s alignment to salient features. The execution of the image augmentation process is a testament to the necessity of bolstering robustness in deep learning—a practice well executed herein. Overall, these insights confirm the proficiency of the CNN-MLP model in agricultural applications and lay a cornerstone for the implementation of reliable and efficient automated processes within the CARA system.

3.3. Performance Analysis of IAM

The IAM performance assessment heavily relies on the reliability of image acquisition, which is fundamental to the system’s overall functionality. It is crucial to note that an evaluation based solely on IAM-captured images may not fully capture its effectiveness within the system. Hence, a more refined approach was adopted to ensure a comprehensive evaluation. Figure 7 shows that the DPC’s accuracy highlights its ability to accurately identify information related to agricultural products from the collected images. This process allows for objective categorization and accurate classification rates of the identified target crops. This evaluation methodology ensures a highly accurate reflection of IAM’s adaptability and efficiency in the integrated system. IAM plays a crucial role in enhancing the precision and reliability of agricultural product identification and processing.

3.4. Evaluation of CARA

CARA was successfully deployed at a farm in Shunyi, Beijing, where fruit identification tests were conducted to assess its performance with the involvement of farm staff and students participating in this study. Figure 8 demonstrates the high and stable recognition capabilities of the CNN–MLP model deployed by CARA for various vegetables, fruits, and cereal crops at different iterations, with the best performance achieved around 200 cycles.

Table 3. Performance pre and post CARA deployment.

ID	Content	Before Implementation	After Implementation
1	Grasping or processing capabilities	Null	Precise grasping or processing
2	Image acquisition functionality	Null	Efficient and rapid image extraction
3	Image processing capabilities	Null	Accurate image processing
4	Image-integrated robotic arm control functionality	Null	Precise and stable control
5	High-configurability end-effector motion capabilities	Null	Motion stability and high configurability

displays the performance metrics of CARA before and after its deployment, showcasing impressive results. Additionally,

Table 4. Suggestions for CARA improvements.

ID	Suggestion	Suggestion Type
1	Further enhance the stability and robustness of robotic arm movements	Robotic arm functionality
2	Enrich the variety and functionality of end-effectors	Robotic arm functionality
3	Further improve recognition frequency and response rate	Image recognition capabilities
4	Expand CARA’s dimensions and functionalities	Full system performance
5	Further broaden the recognizable range of agricultural products	Full system performance

provides valuable suggestions for CARA’s improvement, as recommended by agricultural product processing personnel.

A comprehensive performance evaluation of the robotic arm was conducted to validate its operational efficiency and precision in executing tasks typical of agricultural automation. Key performance metrics included a peak operating speed of approximately 1.8 m/s and an average time to successfully grasp objects at about 3 s, indicating the system’s rapid response and processing capabilities. The end effector demonstrated exceptional repeatability with a positioning accuracy of about 0.1 mm. Over an extensive series of 200 rigorous field tests, the system achieved a 99% success rate in grasping, with only two unsuccessful attempts, highlighting its reliability and effectiveness in practical applications. The employment of a soft, non-destructive grasping mechanism significantly contributed to minimizing damage during operations, thereby preserving the integrity of delicate agricultural produce. Furthermore, the system’s average power consumption was recorded at approximately 200 watts, demonstrating an optimal balance between energy efficiency and performance output. These metrics not only confirm the robustness of the robotic arm but also underscore its suitability for deployment in settings that demand high precision and minimal impact on the products.

CARA’s micro and macro aspects were comprehensively analyzed by integrating various evaluation metrics. These included the performance of the robotic arm in a simulation environment, the efficacy of the CNN–MLP model, the IAM camera’s capability to capture images of diverse agricultural products, and the accuracy of the CARA system in grasping agricultural produce. This performance evaluation establishes CARA’s high proficiency in executing precise and intelligent tasks essential for smart agriculture. Its advanced automation and intelligence significantly enhance agricultural productivity, offering substantial improvements over traditional farming tools.

4. Conclusions

The deployment of the CARA system in smart agriculture environments signifies a transformative advancement in precision and efficiency. CARA demonstrated exceptional capability of automating a wide range of agricultural tasks, significantly enhancing productivity and sustainability. The experimental analysis conducted was thorough and extensive, confirming the system’s robust design and its application of advanced CNN-MLP algorithms. These algorithms facilitate precise task execution, adaptability across diverse crops and operational contexts, and require minimal human intervention. The precision of the CARA system is notably high, with its design enabling reliable performance across varied agricultural environments. Additionally, the system’s broad applicability suggests a significant potential for widespread use in different agricultural settings. We anticipate that future research will concentrate on further optimizing these algorithms to extend their applicability and incorporating autonomous decision-making processes. The successful implementation of the CARA system underscores the promising potential of integrating robotics and AI in agriculture, setting the stage for a more sustainable and efficient future in the field.