Insights on Multi-Agent Systems Applications for Supply Chain Management

: In this paper, we review relevant literature on the development of multi-agent systems applications for supply chain management. We give a general picture of the state of the art, showing the main applications developed using this novel methodology for analyzing diverse problems in industry. We also analyze generic frameworks for supply chain modelling, showing their main characteristics. We discuss the main topics addressed with this technique and the degree of development of the contributions.


Introduction
A Supply Chain Network (SCN) is referred to as a complex network of organizations that synchronizes a series of interrelated business processes, such as procurement, manufacturing, and distribution, to create values to final customers in the form of one or more families of related products or services [1]. Supply Chain Management (SCM) involves complex interactions among suppliers, manufacturers, distributors, third-party logistics providers, retailers, and customers. These entities operate subject to different sets of constraints and objectives. However, they are highly interdependent, and each decision made by any entity in the SCN impacts the other partners. As a result, improving the performance of the SCN highly depends on the willingness to collaborate of all the entities in the SCN and on their ability to coordinate activities within the SCN [2]. The numerous and varied interactions between entities in SCNs, as well as the characteristics of nonlinearity, make it challenging to analyze and predict their responses over time.
So far, considerable endeavors have been made to develop accurate SCN models in order to predict and improve their performance. The conventional models in the literature mainly focus on three levels: the strategic level, which includes location/allocation decisions, demand planning, distribution channel planning, etc.; the tactical level, which covers inventory control, production distribution coordination, order/freight consolidation, etc.; and the operational level, including vehicle routing/scheduling, workforce scheduling, record keeping, and packaging [3]. These previous researches provide beneficial insights on SCNs and SCM, but they mainly address problems from a microscopic view, i.e., focusing exclusively on the focal entity or on the relationships between the focal entity and its closer partner. Fewer contributions aim to analyze the performance of the whole SCN [2], since setting up a global model of the SCN is a challenge.
SCNs have often been conceptualized as simple linear systems represented by an event-dependent series of firms interacting through dyadic relationships [4]. However, this linear conception of sequential dyadic relationships, while appealing, oversimplifies and distorts the realities of modern SCNs [5], such as those mentioned in [6][7][8], and fails to adequately account for the interdependence

Simulation and MAS
Due to the complexity of SCM, it is very difficult for managers and decision-makers to predict the effects and consequences of implementing new management policies, and thus, to decide the best strategies to improve the performance of a real SCN. Hence, the development of SCN modeling tools is very helpful to managers and provides great benefits for enterprises. Traditional methods, like analytic models, classical operational research methods, continuous time differential equation models, and discrete time difference equation models, are not able to cope with the inherent complexity of SCNs, such as the high number of enterprises and all the interactions that take place between them, or the stochasticity and uncertainty present in most of their processes. For example, classical operational research methods are not always able to handle the inherent dynamic characteristics of SCNs [32,33]. Analogously, continuous time and discrete time difference equation models are not always suitable for analyzing complex SCN structures, given the high order of differential equations (one tier generally gives a 2nd-4th order system; 2 tiers even 2nd-6th order), which makes analytical analysis difficult [34,35]. Thus, different modeling techniques are required.
Simulation has rapidly become an important methodological approach to theory development in the literature focused on strategy, organizations, and SCN management. It allows modelers to capture the dynamics of complex systems due to its capability of handling their dynamics and stochastic behavior, and enables managers to analyze and evaluate the effects of alternative configurations of the system, see, e.g., [36][37][38][39][40][41][42][43][44][45]. In particular, there has been a great interest in modeling SCNs as MAS in the past years [21,23], because there is a natural correspondence between the different actors in a SCN and agents in a MAS simulation model. In fact, SCNs tend to be decentralized systems with participants acting independently, according to their own interests and policies [33]. MAS have the capacity to consider the interactions between large numbers of heterogeneous firms, allowing SCN managers to improve their understanding of the whole system and to predict the consequences of singular interventions on the global performance [5]. Therefore, MAS has turned into one of the most effective tools for modeling and analyzing modern SCNs [33,46].
One of the key features of MAS that allow for properly modeling complex systems is the bottom-up methodology, by which MAS models are developed. This methodology is based on a synthesizing philosophy, where the modeler assumes that he/she cannot understand the whole phenomenon of interest but can observe, on a microlevel, specific activities and processes (agents), and tries to understand their behavior. These agents interact and communicate with other agents and they form a coherent whole on a macrolevel, often emerging behaviors that cannot be predicted in advance. Classic top-down methodologies are based on the assumption that it is possible to measure and analyze the phenomenon of interest and, from that, decompose it correctly into different subunits, where the subproblems are solved separately [28], and for this reason, these methodologies are not suitable to deal with complex systems. The adoption of MAS has several additional benefits, such as an increased modeling realism (e.g., individual agents can be comparable to machines, vehicles, products, or groups of such, found in a real life context), heterogeneity (e.g., there is no need to aggregate different agents' behaviors into average variables), bounded rationality (e.g., agents have local information, having their own goals and policies), scalability, and flexibility.

MAS Applications for Supply Chain Management
A literature review has been carried out in order to study the state of the art on the applications of MAS to SCM. Table 1 summarizes the reviewed literature. It can be seen that MAS has been used for researching a wide variety of topics in SCM in the last two decades, such as scheduling, coordination between enterprises, information sharing, order fulfillment process (OFP), collaborative production planning, provider selection, remanufacturing, or resilience, among others (see Section 5 for a more detailed discussion). Table 1 also shows the degree of development of the reviewed works, the role that the agents play in the SCN models, and the software platforms used (if any). Remanufacturing High Resource Anylogic [95] Remanufacturing Medium Resource Netlogo [96] Remanufacturing High Enterprise Netlogo [97] Remanufacturing High Mixed Not provided [98] Remanufacturing High Mixed Java [99] Resilience High Enterprise Python [100] Resilience High Enterprise Netlogo [101] Resilience High Mixed Anylogic [102] Framework High Functional Repast [103] Resilience Medium Enterprise Python [104] Resilience Medium Enterprise Netlogo [105] Resilience Medium Enterprise Java [106] Resilience Medium Enterprise JADE The degree of development gives an idea of the maturity of the reviewed works. In this regard, we have considered three levels: -Low degree of development: the work presents theoretical models which are not implemented in any software platform, and hence, do not provide any results yet. -Medium degree of development: the work presents models that have been implemented in a software platform, and a simulation study has been performed. -High degree of development: the work presents models that have been implemented in real industry or used to solve a real problem, i.e., through a solid case study with empirical data.
Most of the revised literature has a medium degree of development, with a few works showing a high degree of development. In addition, most of the studies with a high degree of development are recent, showing the increasing interest in using this methodology for solving real problems in complex SCNs.
The role of agents determines the level of detail achieved in the SCN models, i.e., low level of detail (e.g., agents modeling enterprises) to high level of detail (e.g., agents modeling machines, trucks, and other resources). In models with a medium granularity, agents represent different functionalities (e.g., agents modeling departments in each enterprise).
The role of the agents is chosen depending on the problem subject of study. If the study focuses on the coordination between enterprises or collaborative planning, agents may play the role of enterprises, while in the case of a study focusing on the analysis of inventory management policies, order management, or OFP, agents may play a functional role. If the analysis concerns scheduling problems or resource allocation, the agents often play the role of physical resources. Finally, a mix of different roles is possible, allowing the analysis of problems at different levels of details, and increasing the flexibility and realism of the developed SCN models. Many of the revised studies adopt the enterprise role for the agents, as most of the revised MAS studies deal with the interactions between enterprises in the SCN. However, these studies are somehow restricted, since the enterprises are modeled with a low level of detail. The literature review reveals that mixing different roles (often functional agents mixed with enterprise agents) is the favorite choice in most of the studies, as it allows for developing more sophisticated SCN models, where some of the main departments and/or functional agents in the system are modelled as independent agents, and thus, these studies allow a deeper analysis of the complex interactions that arise between such departments and the enterprises in the SCNs. Finally, the studies adopting the resource role for the agents were mostly focused on specific problems/processes that take place within an entity, lacking connections with higher-level agents in the SCN.

MAS Frameworks for Supply Chain Management
The above review gives a general overview of the state of the art on MAS-based applications to specific SCM problems. This section focuses on MAS-based frameworks for SCN modeling. A framework is an identification of issues related to a specific domain or how information must be organized to serve a specific purpose [107]. Some of these frameworks are theoretical, while some others have been implemented in a MAS-based software for developing SCN simulators, serving as reference platforms for analyzing specific issues in SCM (e.g., the bullwhip effect, the ripple effect, etc.). A common feature of these MAS-based frameworks is that they have been designed to be generic in order to allow modeling of a wide variety of SCN configurations within the topic of interest. Thus, the generic elements that make up these frameworks can be customized to model specific SCN problems/configurations.
One of the first frameworks that appears in the literature is described by Swaminathan et al. [85]. The authors created a library of reusable agents to model the different enterprises in the SCN and objects to model the control policies for the simulation of material, information, and cash flows. Some concepts of this framework were used by IBM to develop a SCN reengineering tool. Julka et al. modeled the enterprises using only one generic agent (instead of using one agent per type of enterprise), and the behavior of such agent was defined through subagents representing internal departments [72]. This framework was implemented using ADE (Agent Development Environment), and its applicability was shown on a prototype decision support system to study the effects of internal policies, exogenous events, and plant modifications in a petroleum refinery. Dong et al. modeled the enterprises and their departments as agents, and the material, information and cash flows as objects [62]. They used Swarm to implement the model. Chatfield et al. presented SISCO: Simulator for Integrated Supply Chain Operations, for the storage, modeling, and generation of SCNs [46,56,57]. In SISCO, the user specifies the structure and policies of a SCN using a Graphical User Interface-based application, and then saves the SCN description in the open eXtensible Markup Language (XML)-based Supply Chain Modeling Language (SCML) format. SISCO automatically generates the simulation model when needed by mapping the contents of the SCML file to a library of supply-chain-oriented simulation classes. SISCO has been used in several studies to analyze the bullwhip effect in SCNs, see, e.g., [44,45]. Govindu and Chinnam developed a framework based on the SCOR (Supply Chain Operations Reference) model, that allows modeling of different segments of the SCN at either aggregated or detailed levels, resulting in models of hybrid resolution and facilitating the study of intra-and inter-organizational dynamics [68]. The framework was designed by an extensive library of organizational agents, SCN agents, behavior, and policy objects, and it was implemented in Java, using JADE (Java Agent DEvelopment) for agents' development. The authors used MASCF (Multi-Agent Supply Chain Framework), a generic methodological framework focusing on the analysis and design phases of development of SCN applications [108]. Kiralp and Venkatadri developed the DSOPP platform (Distributed Simulation of Order Promising Protocols) [74]. The framework was built around a scalable multiperiod optimization model that may be used across enterprises. Its goal was to show the feasibility of collaborative decision-making and the study of order promising and production planning in the SCN. Long et al. developed a framework with multilayers for modeling and distributed simulation of complex SCN, using JADE [33]. The enterprises were constructed by instantiation of generic agents. The platform supports multilayered simulation modeling and is capable of changing concept models with different granularities into simulation models. Dominguez and Framinan proposed a two-layer framework based on the SCOR model, where an enterprise agent is made up of a set of functional agents depending on its role in the SCN [59]. They implemented this framework using Swarm, and developed a SCN simulator named SCOPE, which allows for modeling and simulating the dynamics of complex SCN structures (see, e.g., [38,60,61]). Recently, a novel multimethod approach was proposed (i.e., MAS combined with other methodologies) for analyzing behavioral aspects of SCNs [37]. Tangpong et al. developed an agent-system contingency theory as a general multilevel theory of managerial decisions in the SCN and industrial marketing context, and proposed the use of experiment and survey as a methodological framework to facilitate the empirical efforts [88]. The study offers scholars a platform to systematically research behavioral SCN and industrial marketing. Shukla and Kiridena developed an advanced analytics framework for configuring SCNs in the context of distributed manufacturing [90]. The authors adopted a distributed MAS architecture to deploy fuzzy rough sets-based algorithms for knowledge elicitation and representation. The effectiveness of the proposed framework was demonstrated by benchmarking it with existent tools using an industry test case. Behdani et al. presented a MAS simulation framework for disruption management, providing a flexible modelling and simulation environment for decision-makers to experiment with different types of disruptions and disruption management strategies [102]. The framework was implemented using Repast, a Java-based open-source simulation toolkit for developing MAS models. Similar to other works (e.g. [59]), SCN models are generated for a specific case by customizing framework generic classes. The authors used the proposed simulator for analyzing the case of an international lube oil company. Finally, Yoo and Glardon proposed a framework for combining two different MAS-based simulation packages (i.e., JADE and Repast) for SCN modelling, and applied it to a real-world case study concerning a manufacturing enterprise [26].
Several key features for the design of a MAS-based framework for SCN modeling have been identified from the above literature. These key features are described next: -Reference model. In the above literature, only [68] and [59] used a robust and well-known model for SCN description (i.e., the SCOR model) to design the agents' structure of the framework. -Detailed manufacturing process. [33,59,72] modeled with detail the manufacturing process. -Stochastic processes. Many of the processes that take place in SCNs are often stochastic, like the transportation of goods (an inter-enterprise process) or the machine processes (intra-enterprise). The frameworks proposed by [33,46,56,57,59,88,102] included some of these uncertainties. -Reusability. In [59,72], authors exploited the reusability of agents and simplified the structure of the framework, with one generic-configurable agent to model each of the enterprises in the SCN, which can be customized with different functional agents. -Intra-enterprise and inter-enterprise process modeling. The frameworks developed by [59,62,68,72] are able to model and analyze intra-enterprise and inter-enterprise processes, which is a very valuable feature for SCN analysis, as it allows for analyzing the interactions that may arise between the microlevel (e.g., departments) and the macrolevel (e.g., enterprises) in complex SCNs. -External solver. Only [59,74] used an external solver/optimizer for solving linear programming models (like the production planning models).

Discussion and Conclusions
The decentralized nature of modern SCNs and most of the modern business processes (e.g., distributed production [109]) make the MAS methodology an outstanding and powerful tool for modelling and analyzing SCNs. The capability of MAS to build large models with many heterogeneous agents acting independently fits with the increased complexity of global SCNs, usually composed of a large number of actors and where companies outsource noncore processes to suppliers, which are often widely distributed and dispersed throughout the territory [110], resulting in complex systems with unpredicted behavior. The potential of MAS models has been exploited by the research community since the late 1990s in order to create more accurate models of SCNs that are able to capture all these complexities. These works have significantly contributed to increase our knowledge of the OFP or the inventory management (see, e.g., [48,52,70,71,[76][77][78][79]82]), common processes in SCN management that have experienced an increased complexity due to the decentralization of SCNs. This methodology has also been very attractive for analyzing different coordination mechanisms, such as information sharing and collaborative production planning (see, e.g., [47,48,51,54,55,60,61,[76][77][78]80,81,84]). In fact, MAS has unique capabilities to analyze coordination mechanisms among autonomous entities, as it has been demonstrated in other fields of research, such as the coordination of robots, traffic management, emergency evacuations, or the coordination of multiple sources of energy (see, e.g., [111][112][113][114]). Another interesting topic addressed by MAS in the management of global SCNs is the selection of providers among several competitors (see, e.g., [53,66,78,79,83,87,91]). Indeed, the capability of MAS models to allow negotiation and competition among agents with individual goals makes this methodology very attractive for researchers in this area. Also, the suitability of MAS to create complex SCN models has allowed for the investigation of the SCN disruptions in the extended supply network, i.e., to ties beyond a firm's direct suppliers and customers. As a direct consequence of the globalization and the interconnection of SCN nodes, they became less resilient to unexpected disruptions, and a failure in one SCN entity can potentially cascade across the whole network. Several authors have investigated this phenomenon in order to determine what factors make SCNs resilient and to propose mitigation strategies (see, e.g., [99][100][101][102][103][104][105][106]).
Nowadays, new complexities have emerged in the field of SCN management. The need for sustainable and green production systems requires balancing global and local efforts to satisfy customer needs without disturbing nature. Thus, sustainable SCNs require management policies that maximize the profit, and reduce consumption of resources and environmental pollution. Several studies, such as [91][92][93], have modelled these and other issues related to the development of sustainable SCNs using MAS. In addition to these, other authors have applied MAS to incorporate all the new processes that aim for a cleaner production, such as the collection of used products for recycling, remanufacturing, refurbishing, or repairing. The incorporation of these processes (also known as reverse logistics) into the SCN leads to a closed form of the SCN, which experiences a higher complexity due to a higher number of processes and the high uncertainty of the reverse flow [115], which, in turn, increases the uncertainty of the management processes. All these features are well suited for MAS modelling, allowing for analyzing the complexities of this new paradigm and studying the impact of the reverse flow of used products (and its associated processes to restore value) on the classic forward flow of new products. This is still an emerging area of research, and there are only a few studies using MAS [38,[94][95][96][97][98].
In conclusion, this review reveals the suitability of MAS for modelling decentralized SCNs, which are complex systems due to the high number of elements and the high and varied number of processes that take place between them. This review also highlights the versatility of this technique to analyze a wide variety of topics within the SCM, being specially interesting for modelling complex SCN structures (e.g., SCNs with many interrelated nodes or closed-loop SCNs) or complex interactions/processes (reverse logistics, disruption management, distributed production, etc.). This is mainly due to the capability of MAS for developing models with different levels of abstraction, where agents may represent resources (e.g., machines, trucks, etc.), enterprises, or a mix of these elements. The number of available software platforms for developing MAS models has increased in past years, making this technique more accessible for developing research works. As a drawback, there is no established methodology for developing MAS models. We have also reviewed a significant number of MAS-based frameworks for studying different topics within SCM. These frameworks can be customized to analyze a wide variety of SCN configurations. Some of them were implemented in a software, resulting in valuable tools for research.