Responsible AI in Farming: A Multi-Criteria Framework for Sustainable Technology Design

Abstract

The continuous fusion of artificial intelligence (AI) and autonomous farming machinery (e.g., drones and field robots) provides a significant shift in the daily work experience of farmers. Faced with new technological developments, many risks and opportunities arise that need to be carefully translated into technological requirements to enable a sustainable production environment. Analyzing the complex relationship between social, ecological, and technological dependencies is a crucial step to understanding the different perspectives and systemic effects of technological functionalities. By providing a comprehensive overview of the state of the art, this article qualitatively analyzes the potential impact of AI on the autonomy of farmers and the technological developments to mitigate the risks. Fair data management practices, transparent AI approaches, and designs for an intuitive user experience are presented as key mechanisms for supporting responsible model development. Based on the defined social, technological, and ecological challenges in AI development, the knowledge to provide a high-level framework for the responsible creation of AI technologies is further systematized. By focusing on the multifaceted relationships and their effects on the autonomy of farmers, this article exemplifies the complex design decisions that must be faced in creating trustworthy and responsible AI tools.

1. Introduction

Recent advances in the fields of robotics, the Internet of Things, and above all artificial intelligence (AI) are now providing a new innovation ground for the next era of farming, often called Farming 4.0 (or Farming 5.0 in some cases). This overarching term combines research fields like precision farming, precision livestock farming, or smart farming, which are all driven by the technological advances in the information and communication technology (ICT) sector. It is thought that the broad application of novel technologies supports the achievement of various political declarations and societal goals (e.g., the Green Deal, Paris Agreement, SDGs, and Farm to Fork Strategy) [1,2,3], providing critical mitigation and adaptation capacities to tackle global issues such as greenhouse gas emissions, acidification, or eutrophication. This should be achieved by assisting stakeholders in tackling the aforementioned challenges whilst ensuring productivity goals by enabling real-time monitoring, simulation, and automation capabilities through high-fidelity models and bidirectional information flows [3,4]. Despite the manifold benefits of novel developments in Farming 4.0 [5,6], technology introduction without consideration for the needs of users may actually hinder sustainability efforts [5,7], highlighting the importance of responsible technology development within complex social-technological-ecological systems (STES). This is especially the case for AI, as the maturity of the technology and the diverse and scalable application fields will be a critical part of many future farming technologies, with a global compound annual growth rate of 23.1 percent until 2028 [8]. As farmers’ new roles, skills, and requirements in such an environment are changing rapidly, frameworks and design requirements are needed to understand the manifold implications of AI and guide responsible implementation to maintain autonomy and limit potentially adverse effects. Specific technological design recommendations only support a fixed set of problems. Therefore, this article focuses on expanding the understanding of multivariate dependencies in technological integration and emphasizes the importance of prioritizing human-coupled environmental actions. It presents the ability to use an STES perspective to define technological requirements for responsible AI development, exemplified by AI’s effects and mitigation potential on farmers’ autonomy. By following the structure of a “systematization of knowledge approach”, this article systematizes the threat landscape, exemplifies technological developments to mitigate the risks, provides a high-level framework for technology development, and elaborates on future work in this field.

In this context, this work aims to explore four specific subtopics in four sections:

• The concept of autonomy and the introduction of the STES approach;
• The systemic effects of AI and autonomous technologies and systems on the autonomy of farmers and current challenges in AI research for the preservation of farmers’ autonomy;
• The principles, techniques, and methodologies for the trustworthy deployment of an AI framework;
• The increase in alignment of AI systems to user needs by design and implementation from an STES perspective.

To accomplish this, the initial section introduces autonomy and the design principles of STES. The second section is organized from the STES perspective and offers a comprehensive overview of the threat landscape concerning responsible AI development. The third section illustrates the potential of technological design choices to tackle the aforementioned challenges. The fourth section discusses the high-level framework for establishing requirements within a responsible technology design process.

2. Farmers’ Autonomy within a Complex Socio-Technological-Ecological Environment

Artificial intelligence plays a crucial role in the transition to Farming 4.0, as it substantially transforms various aspects of crop and livestock farming. Technologies such as machine learning, computer vision, and robotic automation are revolutionizing practices ranging from crop monitoring to animal husbandry [9]. However, while these technologies promise substantial benefits, they threaten farmers’ autonomy as they are causing novel social, technological, and ecological dependencies.

2.1. Concept of Autonomy

Depending on the domain and the subject under study, different notions of autonomy are present. The concept of “autonomy in farming” is multifaceted as well and can thereby also be viewed from different perspectives. It can refer to farmers’ ability to make independent decisions about their agricultural processes or maintain their financial independence [10]. It can also refer to the autonomous systems on the field and their ability to achieve certain goals [11]. In practice, there is a range of complex social, economic, legal, ethical, and environmental interactions (e.g., weather variability, pests, and water availability) that may positively or negatively affect the perceived freedom of choice and the ability to achieve one’s goals. However, the definition of autonomy is not limited to the “perceived ability” to make a decision but also in the absolute sense, as regulations, environmental and economic constraints, cultural practices, biased predispositions, or social embedding can limit one’s ability to make independent and beneficial choices. Thereby, autonomy is influenced by a variety of aspects, such as trust in the technology and one’s abilities, financial stability, access to information and markets, digital literacy, security and privacy of systems, as well as economic, social, technological, and psychological dependencies in all forms [12,13,14,15,16]. Therefore, the rapid integration of AI technologies into the daily lives of farmers could threaten this autonomy in various ways. For instance, AI-driven decision-making tools could limit independent control over agricultural practices by increasing technological blindness or negatively influencing the costs associated with adopting and maintaining new technologies that can only be managed by external service providers [7].

Table 1. Critical factors affecting farmers’ autonomy.

Lack of...	Potential Effects on Autonomy
Technological Knowledge	Trusting technological decisions too much, strong dependence on providers for support, limited cost-effectiveness of technology [17,18,19]
Access and Affordability	Lower economic flexibility and production effectiveness [18]
Data Ownership and Privacy	Increase in vulnerability against cyber threats [20], lowered trust in automated decisions, lack of technology adoption [21,22], increased dependence on data providers [23]
Data and Algorithmic Transparency	Understanding and utilizing AI decisions, lack of trust [24]
Standardization	Limits interoperability of devices and accuracy of ML models [25,26], thereby decreasing trust, explainability, and economic exploitability
Security	Lowers trust, increases vulnerability to cyberattacks, limits economic exploitation [20,27]
Customization	Low interoperability with farm infrastructure and use case adaptation [18,21]
Reliability and Maintainability	Increase in economic variability and dependency on providers, loss of trust [28]
Understanding Farmers’ Needs	Limits technology adoption, lowers trust in technology [21,26], loss of localized knowledge and trust [19,21]

gives a high-level overview of the exemplified cause–effect relationships that influence farmers’ autonomy.

2.2. STES Approach

Many researchers [29,30,31] pointed out the dangers of technology implementation in ecological environments without having a systematic understanding of the associated human–environment interactions and a conceptual framework that governs technology introduction [32]. The farming ecosystem is a complex web of relationships between people, technologies, and the environment. Implementing AI can alter these relationships, leading to unwanted consequences that can directly and indirectly affect farmers’ decision-making processes and, ultimately, autonomy. The bridging of a socio-ecological system (SES) approach with a socio-technological (STS) approach creates a novel playground that investigates the intermediary role of technology within a socio-technological-environment system (STES) nexus. By inspecting the complex interactions in this multi-dimensional environment, an integrated technology (assessment) approach can account for technology-related dynamics to avoid simplistic, reductionist, or deterministic explanations [29]. As an STES approach recognizes the complexity of these dynamics (uncertainty, feedback cycles, emergent phenomena, etc.), scientific progress in this area can lead to novel frameworks and approaches to designing, integrating, and maintaining technological developments and fostering sustainable and resilient management of environmental and technological systems [31]. As many unjust algorithmic developments (bias, discrimination, harm, etc.) revolve around purely technical solutions [32], the STES approach tries to tackle this challenge by putting the human and his or her actions in the center to pave the way for human-centered technology requirements as shown in Figure 1. By understanding that technology and its goals are implemented in a dynamic setting of societal, personal, and environmental needs, it assesses the dynamic of the system components from the start. It is important to understand the different political consequences of framing the goals of AI implementation, to acknowledge a certain lack of understanding of all perspectives, and to be aware of one’s personal viewpoint. In doing so, the development and implementation of responsible AI systems must explore how intentional and unintentional technical mediation may result in ambiguous outcomes and feedback that displace or relocate but do not remove negative consequences. Based on this evaluation, it identifies strategic interventions and ways of changing relationships such that these create well-being for humans and non-humans alike [29]. Finally, as the implemented STES environment is continuously changing, measures for re-evaluating the intended impact are necessary.

3. Current Challenges of AI in the Autonomy of Farmers from an STES Perspective

To assess how AI could lead to unwanted effects on the daily lives of farmers, we follow the proposed STES aspects and provide detailed risk scenarios from the social, technological, and environmental perspectives. As these perspectives are often inherently connected in a dynamic setting, a clear cut between the individual aspects is not always possible. Nevertheless, the following sections highlight the intertwined mechanisms and provide an understanding of the multifaceted challenges AI-based technology design currently faces to provide responsible and sustainable implementation.

3.1. Social Aspects

The deployment of AI technologies is set within a web of social entities that are driven by economic incentives to a considerable degree. Technology providers, farmers, third-party entities, and the farming community are interested in maximizing their individual objectives, thereby providing a variety of requirements for the technologies that sometimes compete against each other (e.g., economic scalability vs. crop diversity). As the providers drive the design of the technology, the risk for the users and broader farming communities is more prominent. Vendors tend to centralize data storage to continuously develop their algorithms (vendor lock-in) over data management platforms [28]. The economic gains of those data are on the side of the technology provider, while the farmers are promised better products, which in turn are sold at higher prices [28]. This threatens the long-term economic stability of farmers. The implications of rights management on trust in the context of data storage were also highlighted as a prominent factor in the survey conducted by Fleming et al. [33].

Another factor is the distribution and access to knowledge of the systems [34]. Centralizing specialized knowledge at the vendor site enables a continuous business model based on maintaining and upgrading the deployed systems. Conversely, this increases farmers’ dependency on AI businesses as the sole knowledge providers. Limited access to knowledge for farmers results in low maintainability and adaptation of novel technologies and ultimately in reduced capabilities to judge the AI decision process. This in turn either lowers trust or results in trusting the decision output too much, with both limiting economic exploitation of the technology. Furthermore, working with AI-based technologies creates the need for new skills in the farm workforce (for programming digital systems and software, data analytics, or machine and hardware maintenance) [28]. At the same time, switching to automated technologies based on complex decision processes allows a shift to a more steady workforce. As repetitive and simple tasks may be annihilated, fewer migrant and part-time workers are needed, as they are potentially replaced with fewer high-skilled, full-time work personnel [35,36]. This creates novel requirements for technology design in terms of the user experience, as the deployed models must be designed for the specific user group or should at least be adaptable to different groups (e.g., language and visualization). Depending on the geographical area, different work requirements will lead to a non-homogeneous target audience. If the design of the AI is not adaptable to different skill levels of users, then it systematically widens the gap between demographic groups. Furthermore, various geographical and demographic differences must be accounted for to avoid social disparities, such as long-standing racial inequities [28], gender differences [37,38], and also disparities in farm size [33]. The respective models and tools are often designed for large-scale producers [26,39] as those are the farms that adopt digital technologies at higher rates [40] and may further widen an already existing digital gap between farming populations [41].

3.2. Technical Aspects

The data-driven modeling aspect of AI provides a variety of technological factors that create uncertainties in farmers’ decision-making processes. Along the AI development pipeline, this already starts with creating biases during data generation or its processing. During collection, a bias in data can be introduced by sampling solely in specific regions or types of farms, therefore expressing the ecological environment and social practices of a limited area (sampling bias). Due to the limitation to certain regions, the data might exclude different demographic properties (exclusion bias) or focus too much on crops with a high economic value (selection bias) [42]. Such biases limit the ability of farmers to infer actions based on generalized suggestions by the AI [43] and may produce a disadvantage for areas or crops of less economic interest (e.g., central African countries) [26]. The diversity of measurement tools and practices may also lead to bias during data generation (measurement bias) or limited accuracy of the models in a different data environment (interoperability issue). Depending on the instruments, data may be in a different format, aggregated at different levels, or produced in a different environment (e.g., soil depth, water content, and time of day).

Further bias can be introduced in the data processing phase of the model [44] through the selection of the model or its parameters (e.g., number of cross-validations, size of the train-test split, regularization functions, or data selection) as well as by making wrong assumptions about the modeling subject [42]. Other algorithmic biases may be created by the optimization function used by the model, the user interface, and the feedback loop between the system and its users [45]. Such algorithmic biases can significantly impact the model’s efficiency, trustworthiness, and deployability. This situation is aggravated if several models are incorporated into the system that exchange data between them or build on the decisions of each other [7]. This complex data flow creates an additional challenge for the already limited transparency and explainability of deployed models [32]. Based on this data complexity, the ability of algorithms to continuously learn by interacting with their technical, social, and environmental environment (e.g., reinforcement learning) further aggravates direct and hidden model dependencies. Such dependencies lower the transparency of the system, leading to higher costs and decreased maintainability of products based on AI [46]. These hidden dependencies potentially initiate a cascade of unwanted feedback loops, unknown emergent effects of the models, or large-scale fragility of the system. If those issues are not addressed in the design phase, hten those design weaknesses may further increase the economic dependency of farmers on the vendors, the uncertainty in the decision-making process, and trust issues toward the technology.

Another important aspect is the amount of data available. In most cases, data will be sparse and scarce, and then deep learning techniques cannot be used [47,48]. A similar problem happens in health applications, and thus we can borrow techniques from that field to handle this issue [49].

3.3. Environmental Aspects

As mentioned in the previous section, AI algorithms are often trained on a particular set of environmental conditions (e.g., soil type or climatic region) and redeployed in novel agricultural settings. Also, designers of such algorithms may lack the knowledge of traditional farm operators and make changes to the algorithm that are not aligned with personal or regional farm management practices (e.g., irrigation techniques or tillage practices). This increases the risk of unexpected behavior of the models, as the data input do not match the pretrained model or will be processed in a way that does not meet the individual user’s needs. This can lead to unknown and hidden biophysical feedback loops [46,50]. This decrease in accuracy may severely affect the trust in the deployed models, the system’s maintainability and explainability, farmers’ ability to make the best decisions for their farms, and the interoperability of AI technologies. Therefore, the ecological compatibility of the AI decision-making processes must be ensured both at the system and algorithm design levels [51], with a focus on developing clear operational limits that follow transparently defined metrics [52]. Furthermore, the model might learn indirect environmental or social key aspects (e.g., soil health, water cycle, and sowing patterns), which may lead to emergent effects from the model or leakage of sensitive information of the farms to the model providers or even other farmers.

4. Mitigation through Technological Developments

Despite the manifold challenges that surround AI technologies in the complex web of social, ecological, and technological interactions, responsible design could also leverage its ability to mediate the needs and requirements of each realm and provide a platform for the geographically diverse engagement of farmers [53]. Most AI systems are designed to optimize the biophysical realm set by the user’s goals (e.g., the feeding of animals or watering schedule for crops) [4,7]. The ability to achieve this is bound to the ecological and social feedback it obtains as inputs, as well as presenting the monitored and optimized subject meaningfully to the user. Therefore, this represents a crucial point that links and navigates the perception of the user and his or her farm, includes diverse stakeholders, and represents the data in a manner that is responsible [54,55]. The following section provides three examples of leveraging its capabilities to increase responsible and sustainable use of the data, algorithmic, and visualization levels (see Figure 2).

4.1. FAIR Data Principles

The findable, accessible, interoperable, and reusable (FAIR) data principles are a data management framework and open system design methodology with a particular emphasis on machine actionability in addition to supporting its reuse by individuals [25]. These principles not only apply to data in a more narrow sense but also include algorithms, tools, and workflows that have been part of the data generation and processing pipeline. In this context, Section 4.1 also integrates general data management design strategies to foster responsible use in AI environments.

Machine actionability is a vital process to the data principle and is particularly important in automated data processing. As AI systems can combine and automatically process diverse data sources, this autonomous agent must have the capacity to carry out the following processes:

• Correctly identify the type of data;
• Determine if it is worth using for the current task by analyzing the metadata or data elements;
• Understand any licensing, legal, or accessibility constraints associated with the data;
• Implements them correctly for the specified task [25].

By making the data findable through unique identifiers, diverse metadata, and common standards, the data provider ensures technical interoperability [56] for all geographically distributed AI service providers and their users but also enables clear visualization for transparent AI processes, as the context of the data and its limitations can be displayed (e.g., data generation method, experiment set-up, and data provider). This is also vital for analyzing bias in the data, as the enhanced capabilities for the machine to understand the context within which the data are generated may provide further information for identifying critical and sensitive areas in the data. Also, subsets of the data or changes in the data (dynamic data) should be uniquely identifiable and linked to the original data [57]. Changes to the data compared with its original source should be described in the metadata to enhance traceability, knowledge extraction, and the avoidance of potential biases.

Providing accessibility to data via open and free protocols enables fair chances to find the necessary information for one’s needs [58]. Accessibility does not necessarily imply open access to the data, as this could lead to other ethical concerns. However, it should clearly state under which modalities the data can be accessed. Also, the metadata should be openly visible even after the data are no longer available to enhance transparency and reproducibility [56]. In general, access to data should be as open as possible without introducing privacy concerns [59]. Data practices that enable privacy-aware access should be implemented in the AI processing pipeline to leverage the learning capabilities of the algorithms and better scalability and interoperability for the system. This includes practices such as anonymization of the data, synthetization, and federated machine learning algorithms. Anonymization describes the process of transforming the data in a way that sensitive information is hidden [60]. Synthetization of data is a range of methods to create an artificial data set that inherits the characteristics and structure of the original data [61]. Federated learning, on the other hand, is a machine learning approach that builds its model on decentralized and isolated data, thereby providing a security- and privacy-aware model training process [26,62,63]. All three methodological examples can be used to enhance data availability for a diverse set of stakeholders with limited open data access or without enough data to train a model for their needs. Furthermore, providing AI services incorporating federated learning may positively affect technology adoption for a broader set of stakeholders, as existing privacy concerns are mitigated [15].

Interoperable standards enhance the usability of the data [25] and leverage competencies to develop accurate models and foster farm system understanding. This will be achieved with qualified references to other (meta)data that increase the knowledge creation capabilities of AI services to provide enhanced and trustworthy decisions for the farmer. Interoperability of the data is also crucial for M2M communication [64]. Compatible data sharing between robots can increase the autonomy, flexibility, and interaction between robots, thereby supporting the overall autonomy of the farmer. Using a shared vocabulary for knowledge representation further strengthens the interpretability of the data [56]. This enhances the explainability of AI systems, enabling farmers to make more informed decisions independently.

Lastly, fostering reusability of data increases interoperability, as domain standards increase long-term use and ease of integration in different technological systems. This is a critical factor, as scholars [18] and farmers [65] alike highlighted the negative impacts on usability and technology adoption if interoperability is limited in smart farming technologies. The plurality of data descriptions ensures effective replication and (re)use in different farming environments, as the context of the data and its limitations can be better judged. This enables faster identification of bias and errors in the data and mitigates the risk of models’ unexpected behavior. Furthermore, such standards strengthen long-term maintenance capabilities and the economic autonomy of farmers, as they do not rely on outsourced and unknown data sources. As not all data must be entirely available for farmers, clear data usage licenses and detailed provenance enable farmers to understand the origin of the data, its authenticity, as well as the legal terms that can be used. This may positively impact confidence in the data and models and enable trust.

4.2. Transparent AI

Transparent AI is defined in this context as a system that provides information about its decision-making process and constitutes a high-level concept that incorporates the approaches of interpretable AI or explainable AI [24,66]. The first uses models that are inherently interpretable to humans [67], whereas the second approach provides explanations for its predictions to the user [68], primarily for models that are too complex to understand directly. In combination, the transparent AI approach not only focuses on explaining the model outputs but also provides information about the model pipeline. This creates novel potentials for human-in-the-loop design after model and product deployment [69].

Trusting AI decision processes too much is a serious concern in most professions. Understanding the underlying criteria and attributes of an AI through interpretable or explained models can facilitate trust in one’s own and the models’ decisions and enhance collaboration by combining the farmers’ knowledge with the data provided by the AI. A farmer may confirm his or her management hypothesis or the AI-based solution by analyzing the contributing factors directly in the model (interpretable AI) or through visualized explanations (explainable AI). Visualizations such as bar charts are often used for identifying the importance of attributes [70], heatmaps can show what parts of pictures have been used by the model [71], partial dependence plots enable an overview of how different values of attributes influence decision outcomes [72], or scatter plot diagrams can help users to see relationships or similarities between model outputs [73]. Interpretable models, on the other hand, can give direct access to the probability distribution of features (naive Bayes) and the importance of individual criteria (linear or logistic regression models and decision trees), provide a logic-based structure that allows following the decision process of the model (decision trees and covering algorithms) [43,67,74], or enable one to visualize the distribution of possible decision lists (Bayesian rule list) [75]. It is important to mention that interpretable models often come at the price of limited scalability to large data sets (i.e., many attributes) and complex decision processes (i.e., nonlinear relationships between attributes) and should therefore be handled with care [67,74].

Both explainable and interpretable models can help detect biases (see Figure 2), errors, or hidden dependencies in the decision process [24,43,68,71], such as by identifying sensitive attributes and their importance to the decision outcome (e.g., gender, race, height, and postal code). By understanding the underlying decision process and the importance of individual attributes, farmers can increase their decision-making competence, farm understanding, and privacy assessment [24]. Developing a system that is also able to learn from farmers’ feedback (e.g., reinforcement learning) could further help to strengthen collaboration by aligning the outputs and decision processes with unique farm management practices [76] or local biophysical differences (e.g., calibrating the model for a priori unknown biophysical feedbacks). This provides customizable management of AI systems for individual farm use cases [21]. As users are often inconsistent in their feedback or may inflict errors and biases [45,77], particular guidance and merge criteria must be developed to ensure correct model development (e.g., an error message if an input conflicts with the expected values or has a negative effect on the accuracy).

4.3. Intuitive User Experience

The user experience, with a particular focus on user interfaces, provides the critical link between the AI-based farm management tool and the user [77]. Design strategies should therefore be chosen to enhance the system’s mediating capabilities, optimizing various requirements such as understandability, usability, customizability, transparency, performance, or effectiveness. This is a challenge as the requirements change depending on the practitioner attributes (e.g., social background, education, use case, or goals), highlighting the need for inclusive design strategies to present information in a valuable and relevant way [78].

Depending on the area of employment, the designers of such systems must have a profound understanding of the local workforce and potential future changes due to technology implementation. Migrant farm workers or minorities dominate the labor market in many areas. Understanding the knowledge gap that is present and providing information representations that are understandable, enable learning, ease configuration, and allow simple reconfiguration are crucial to ensuring unwanted social effects. This can be ensured in the program by visual or auditory aids, context awareness of the program, automated translation services, a comprehensive help section and clear documentation, clear and consistent design, interactive designs and visualizations that enable customizability, the inclusion of real-time support like a large language model designed to answer questions and provide an explanation about functionality, as well as the availability of human support [76,77]. The ability of context awareness and adaptive user interfaces could provide further support by measuring human capabilities during program interaction (e.g., eye tracking, blink reflex, and time on a page), providing suggestions for the sensory adaptation of the interface, and displaying additional material or presentations of information to enable teaching and training [79]. In general, customization of user interfaces for respective AI models enables the ease of use for a wide variety of users, enhances data analytic capacities, and facilitates interaction and system understanding as well as use case adaptation [18,76].

All of this requires a priori knowledge of the user base. User research and extensive software testing in combination with the local workforce is crucial for successful and inclusive technology design. Fleming et al. [33] highlighted the need to understand farmers’ key language, rules, norms, values, and assumptions to support technology adoption. It is stressed that gaps in equity, access, and distribution of benefits should be addressed by understanding the implicit and explicit implications of language use to navigate knowledge processing for farmers. As a shift from low-skilled manual labor to high-skilled technology workers is expected through the increasing use of AI, this creates the opportunity to provide designs that are equally engaging for different genders. It was shown that a difference in aesthetic (background image, font, and colors) or language features (distribution of masculine and female terms) could influence belonging or interest in the subject area [80].

An inclusive user experience should also support expert and non-expert users in their decision-making process, highlighting the need for advanced methods to display and manipulate data and algorithmic dependencies and biases (as seen in Figure 2) without an unsustainable cognitive workload [79]. The user should be able to select subsets of the data, adjust the model parameters, and relabel or add new data. The program the machine learning framework is integrated into must support clear, concise, and relevant information on the data, metadata, and data translation in parameter settings within the AI decision process to ensure trust and long-term software maintainability by the user or for communication to the service provider. Further research has established a link between efficiency [81], quality, model interpretability [82], and use case adaptation [21] if users are involved in the feature selection process of the model or can influence the model goals (e.g., precision vs. generalizability). Visualizations of the impact of features on the accuracy and model behavior (e.g., explanations given to the user) with the ability to undo actions and return to previous model states allow expert users to experiment with different strategies [83] with the potential to increase the accuracy, farm system understanding, trust in the model results, and redeployment in different farm settings [21].

5. AI Development Framework

Formulating technological requirements that enable the widespread integration of various factors is always difficult. Technologies can only adhere to some standards, as many requirements compete against each other (data accuracy vs. data privacy) or are limited by other constraints such as financial resources, implicit and hidden community standards in AI development, or individual preferences and world views. Therefore, this article does not provide deterministic recommendations for technology designs but strives to create an expanded view of the multivariate dependencies that come with unjust technological integration and highlights the need to put human-coupled environmental actions at the center. The inherent complexities of such have been exemplified based on farmers’ autonomy when using AI technologies while providing several design suggestions on how to combat detrimental effects. By providing a high-level framework on the social, technological, and ecological implications of the system design requirements, this article strives to support technology designers in creating responsible AI technologies and farmers in their awareness of potential risks.

The proposed framework in Figure 3 provides a first step to targeting responsible design choices not only for AI but for algorithmic development in general. It includes several key characteristics as defined by Schiff et al. [84]. The framework first targets an AI system’s overall scope to human well-being and then determines the lower-level sub-tools and methodologies. It directs attention to the knowledge and expertise needed to fulfill the required outputs, thereby advising the operationalization of the targets. By analyzing the social-technological-ecological dependencies within the embedded environment, as well as by asking questions about operationalization, it guides the attention to the system knowledge needed to achieve the desired AI attributes. In doing so, this framework is linked to the STES approach introduced in Section 2 (“AI in Complex Environments”). By putting symmetric attention on the technological functionality, its users, and the implemented environment (Steps 1–3), it acknowledges the complex dependencies of AI systems and robotics in farming environments. By defining the desired attributes of the AI implementation, its users, or the farm (Step 4), it automatically reflects the impact of different perspectives on farming practices and explores how technology shapes the human–nature relationship. Thereby, it narrates the dynamics that are inherent in the daily farm operations that have to adhere to a multitude of requirements whilst aiming for economic feasibility. Steps 5–7 seek to identify strategic and sustainable interventions to strengthen the system functionality and induce positive change.

Viewing this framework as an iterative and recursive process is imperative, as many of the proposed solutions could lead to novel issues for responsible technology design. Prior understanding of the social, technological, and ecological dependencies must therefore be a primary principle for making responsible decisions about architectural and conceptual designs. By encouraging the re-evaluation of the chosen design decisions at the respective STES level, the framework opens up the question about further mechanisms to strengthen the desired attributes (Step 5). For example, the proposed mechanism of user screening for adjusted support could lead to data privacy issues or a lack of trust if sensitive information about farm attributes is processed. Therefore, such mechanisms should be joined with other strategies that enhance privacy and trust (such as localized data processing and synthetizing data for centralized model learning). Further examples of how the technologies in Step 5 support responsible AI practices can be found in Section 5 (“Mitigation through Technological Developments”).

The framework provides easy-to-understand questions for moderately skilled users to obtain a high-level understanding of the complexities of designing responsible AI systems. It is flexible in its design, as it is not limited to a certain AI system (e.g., supervised, unsupervised, or reinforcement) or environment such as farming. However, the current framework targets the development of new AI architectures within the context of a multitude of requirements to support sustainable design decisions. Therefore, it is limited in its capacity to advise AI architects concerned with downstream AI life cycle phases (e.g., deployment and monitoring).

The differentiation between attributes and instruments (to achieve the attributes) is blurry to a certain degree. Some instruments can also be defined as attributes and vice versa, as they often have a bilateral influence. In general, attributes are considered to be of a higher hierarchy, as there is no direct and simple way to achieve them, and they are therefore often a mix of different instruments. Analyzing the synergies or trade-offs between attributes and instruments is critical in this context. As mentioned before, some instruments conflict with each other, whereas the combination of certain instruments could have a widespread positive effect greater than the sum of individual actions.

6. Future Work

As the development of AI and robotics is progressing at a fast pace [8], future work considering the effects of such technologies is critical to leverage the positive effects for society and the environment. The effects of such technologies on attributes like autonomy or resilience are hard to measure directly and can often only be assessed qualitatively or via indirect metrics (e.g., average age of the workforce or gross margin per hectare) [85]. Finding approaches that support technology assessment and validate the usefulness of such frameworks is therefore the subject of ongoing work. One potential success in this direction can be defined in the incorporation of robotics and AI in digital twins [86,87]. A digital twin is designed to approximate the behavior of an entity in virtual space based on continuous updates through the collection of data, models, and what-if simulations [51]. In this way, it creates the functionality to explore the effects of different design choices in a safe space. By creating a digital twin at the farm level, different attributes and qualities can be defined by metrics and simulated to support designers in assessing the effects of different design decisions. As defined in [7,51], it would support the validation of AI models, providing an understanding of the model behavior and its boundaries.

Further work is also needed to integrate fairness indicators into the model’s development as well as to display the characteristics of such in an informative way through user interfaces or transparent AI processes. Partial dependency plots or counterfactual explanations are some of the many solutions for investigating the impact of changed values on the decision outcome but are not adequate substitutes for in-depth fairness analysis in the model pipeline. Human audits and tests are one of the most common approaches to checking for data and model fairness [24]. Such a step is insufficient if users have control over their data generation processes and collaboratively improve the AI system for their own use case. Analysis tools that foster an understandable and explorative way for farmers to check for biases represent one of the future research challenges in this domain. This could support the integration of responsible principles in AI systems but should not be seen as a concept to shift the responsibility for trustworthy AI to the consumer.

Developing and implementing fair principles in different domains is still subject to ongoing research. Indicators and priorities have been designed for the data requirements to help with swift implementation for individual use cases. Not every fair data management principle is weighted the same in terms of importance as well as the number of indicators to achieve responsible data use [88]. This creates bias by itself in following the standards but also provides the possibility of having a theoretically fair data standard that is not interoperable, as those are marked less critical. Further research should address this gap by having different design strategies depending on the necessary attributes one wants to adhere to.

7. Conclusions

It was shown that the development of AI systems is facing a variety of social, ecological, and technological challenges and requirements. These factors must be addressed and mediated to create AI and robotic technologies that can be called responsible and trustworthy. Exemplified through the effects on the autonomy of farmers, several key aspects have been introduced to mitigate the risks along the entire AI pipeline. These include facilitating fair data principles, particularly for the data collection and cleaning of the AI development pipeline. Approaches in the era of transparent AI can enhance the explainability and interpretability of deployed algorithms to create a baseline for trust, understandability, maintainability, and interoperability. Designs that foster intuitive user experiences have been shown to support bias avoidance when interacting with algorithms. These designs also enhance AI systems by improving cognition, learning, and usability. Based on this review, an AI development framework has been developed that provides symmetric attention to social, ecological, and technological requirements and guides the AI architect in finding sustainable solutions in this complex and dynamic environment. Further work is needed to validate the proposed framework and to investigate the effects of different AI design choices on attributes like autonomy or resilience. With this, digital twins may play a crucial role by approximating the behavior of a technological entity in virtual space.