Agent-Based Modeling of a Self-Organized Food Safety System

Abstract

“The wisdom of crowds” is often observed in social discourses and activities around us. The manifestations of it are, however, so intrinsically embedded and behaviorally accepted that an elaboration of a social phenomenon evidencing such wisdom is often considered a discovery; or at least an astonishing fact. One such scenario is explored here, namely, the conceptualization and modeling of a food safety system—a system directly related to social cognition. The first contribution of this paper is the re-evaluation of Knowles’s model towards a more conscious understanding of “the wisdom of crowds” effects on inspection and consumption behaviors. The second contribution is augmenting the model with social networking capabilities, which acts as a medium to spread information about stores and help consumers find uncontaminated stores. Simulation results revealed that stores respecting social cognition improve the effectiveness of the food safety system for consumers as well as for the stores. Simulation findings also revealed that active societies have the capability to self-organize effectively, even if they lack regulatory obligations.

1. Introduction

The food industry, a set of collective activities facilitating the supply and consumption of different food products, is very important across the globe. It is given special attention in developed countries, as food has a direct impact on the health of their citizens. Food safety has, therefore, became a growing concern globally [1,2,3,4]. As the population across different regions of the world grows unevenly and the disparity between rich and poor widens, the ultimate fallout is foreseeable. Simply, the countries which can spend more on food safety are less vulnerable—already having better systems and health-care facilities. Whereas the countries which have issues in their economics, governance, and health-care, are more likely to suffer. Therefore, the question—how can people self-organize to deal with the issue of food safety? particularly in under-developed and developing countries, is critical and worth exploring.

Scientific improvements not only facilitate standard harvesting and logistics processes but also safeguard against contamination of food in 17 stores. However, a lot needs to be done, as the global food distribution and supply chain have become very complex, in the recent past [5]. This is due to an increase in the population in general, and complex interactions between farmers, suppliers, traders and consumers in particular, which has led to several food safety crises recently such as the occurrence of bovine spongiform encephalopathy (mad cow) [6] and maple leaf foods listeria monocytogenes [7]. The former crises approximately infected 1000 new cows weekly. Long term impact of the crises reduced the beef market shares by 4.5%. The latter crises, which occurred in 2008, resulted in 57 confirmed cases and 23 deaths. This was due to serving meat to highly vulnerable people. The occurrences of such disasters initiated a debate regarding viable solutions. The solutions under considerations include introducing strict laws, re-thinking of prevalent industrial practices and adopting a fresh approach to understanding consumer behaviors [8].

Stores are usually not the center of attention when it comes to food safety. The main measures within the whole food chain take place before the store level. The assessment and eradication of food contamination, at the consumer level, is usually carried out using food inspection programs in which the food provided by the stores is the center of attention. It is, however, noted that the lack of standards has made the provision of resources for inspection and agreeing upon thresholds for contamination, a real challenge [9,10]. Similarly, the consumers and the psychological factors representing their perceptions [11,12] and attitudes [13] about food quality are also important [10].

The systems, which are social in nature, including food supply and consumption, are usually based on societal needs and constraints and they evolve with time. Most of them, however, are not ideal for several reasons, including the diversity of the human race, the inherent imbalance in human reasoning, the gap between scientific knowledge and its impact on individuals in terms of development and flaws in knowledge and judgments.

Computational Social Science (CSS) [14] is a comparatively a new field of study, believing in the use of computational procedures for solving social problems. It mimics physical systems by developing their models in such a way that computers understand and compute them. Agent-Based Modeling (ABM) [15] is one of the most widely used techniques for investigating CSS problems. In ABM, the real objects of interest are represented as agents, which provide an abstract representation of characteristics, actions and communication patterns between the objects. The agents sequentially interact with each other, materializing an evolving society by changing the system from one state to another as the time progresses.

Literature includes several models for food safety. Some of them are developed using ABM techniques [16]. Naturally, the entities of interest for a food safety system include farms, workers, consumers, warehouses, stores, trucks, and regulators. However, the problem scale and scope determines the relevancy and non-relevancy of the entities. For example, food safety from the consumer point of view focuses on the landscape, where the consumers interact with food items. Knowles [5] presented a simple model, aimed at investigating the impact of information sharing among stores and regulators regarding the quality of food consumed by consumers. This model uses three types of agents, namely, consumers, stores and regulators.

Knowles’ model implements two possible responses from stores to regulators. In the first case, a store signals a regulator on its own, when it becomes contaminated. In response, a regulator quickly arranges a visit to the store and sterilizes it. This work uses the word sterilize to show that the food safety problem is corrected. In the second case, no signals are generated and regulators visit the stores purely based on a random selection process. When a consumer encounters a contaminated store, irrespective of the case, they add the store to the list of bad stores and avoid visiting it again in future. This behavior represents an individualistic, self-interested decision rather than the concept of the wisdom of crowds [17]; a decision made by a crowd as a whole. The wisdom of the crowd can be implied by noting the number of consumers avoiding a particular store. However, the Knowles model does not consider the effect of consumers avoiding a store while taking a store’s decision. In fact, a store does not make a decision altogether. It goes into a contaminated state randomly unless it is visited by a regulator (either in response to a signal issued by the store or a random selection). During this time, a first-time visitor would register this store in their bad list permanently. The store is not allowed to respond to a decrease in the number of consumers.

This paper extends the Knowles model [5] for evaluating the notion of “the wisdom of crowds” against the conditions, which results in a safer and more dependable food system. This model incorporates stores and consumers’ responses to the wisdom of the crowd (the major limitation of Knowles model [5] described above). It takes effective and timely decisions for improving food quality. Stores are categorized as either “vigilant” or “indifferent”. The vigilant stores are made responsive. However, the indifferent stores remain the same. The extended model relates the contamination of a store with food quality. To make the model more realistic, the quality improves or degrades gradually instead of switching abruptly between the contaminated and sterilized states.

The proposed extended model is capable of investigating the impact of a store’s vigilance on the consumers’ health. We believe that a vigilant store would keep a large fraction of potential consumers (having no experience with the store) compared with an indifferent store, where the crowd interest is decreased when food quality is degraded. In response, a fairer distribution of consumers across the stores in the competition would improve the dependability of the overall system. We further consider social networking as a mean of disseminating information about “good_stores”. The consumers’ experiences make the store good or bad instead of a store signaling its contamination status as in the Knowles model. This information is generally private to consumers but can be shared with friends. We believe that information sharing on social networking would improve consumers’ experience as a whole.

The main motivation is to develop a model which can represent a more real-world situation by incorporating two important aspects. The first one is based on the fact that societies as a whole are capable of self-organization. The second one is the inclusion of the social network, which has become an important medium for information sharing. Given more explicitly, the following two hypotheses are developed to investigate the two aspects described above:

Hypothesis 1 (H1).

More vigilant crowds, stores, and consumers would not only improve the quality of consumed food but also ensure a fair distribution of consumers across the stores.

Hypothesis 2 (H2).

Social networking-based information sharing would further improve the consumers’ experience and thus the quality of the food.

Consumers and stores are both capable of self-organization. The validity of these hypotheses would be discussed in different situations by considering an interesting scenario in this work. It is believed that a society can emerge in better shape, even, in the absence of regulatory authorities. However, this is only possible when there is no monopoly/government control (the free market economy). Further, consumers must have the right to choose whatever they like and share their experiences openly on social media.

This paper studies the current as well as the proposed models of food safety and the rest of it is organized as follows. Section 2 provides the relevant literature and motivation for the work, presented in this paper. The proposed model, after introducing the current models, is presented in Section 3. Section 4 provides the evaluation results and the analysis of the simulation scenarios for both current and proposed models. The conclusion of this work is presented in Section 5, which ends this paper.

2. Background and Motivation

Bozkurt and Padilla [18] have presented a general overview of modeling and simulation applications solving real-life problems. However, their focus was on methodological, teleological, ontological and epistemological themes. The research conducted by Bledaa and Shackley [7] is more related to this work. It used ABM for studying the establishment of risk perceptions—one of the well-known models, social and theory-driven in nature, which is widely studied [19,20,21,22]. Many simulation models targeting different domains have been developed, such as the one recently developed by Ge et al. [23] for the supply chain management.

The literature includes many models in the domain of food safety; however, only a few of them have used the ABM technique. An agent-based supply chain model for simulating agri-food is presented in Reference [16]. Verwaart and Valeeva [24] have proposed an agent-based model for measuring the effects of incentives and interaction patterns for food safety. This model uses heterogeneous agents capable of using different options/choices for the motivation and openness towards communications. Talley’s thesis [25] used agent-based models for investigating various characteristics of consumer behaviors in spreading contamination. It utilized the intervening role of the public health department to stop spreading contamination. To inspect food safety, Knowles developed a novel and simple but exciting model [5]. Knowles’s model is extended in this work to investigate the social aspects of consumers and store owners. It is believed that the consumers and store owners accepting the existence of “the wisdom of crowds” can make good decisions. This is not only good for consumers and owners but at the same time beneficial for the whole society.

Routine Activity Theory (RAT) is one of the theories used to quantify peoples’ routines [26]. It uses a three-way causal relationship describing the occurrence of an event across spatial and temporal domains. The use of RAT in a criminal study, presented in Reference [27], motivated us to conceptually build our model based on RAT. This work created a problem analysis triangle, similar to that used in Reference [27], to visualize the entities and their abstract relationships, involved in food safety management paradigms across space and time. Since this work used Knowles model [5] as a groundwork, it started with representing the model using the RAT. The proposed model, then, grew on top of this groundwork.

Figure 1 illustrates the RAT triangle, where its three sides depict the three model entities, namely, the consumers, stores, and regulators. Regulators have a causal relationship with stores and they visit them randomly or by a signal, in the case of contamination. Consumers go to stores to buy food items. They may avoid going to a store if previously they had a bad experience with that store. Finally, the Knowles model [5] lacks the relationship between consumers and regulators as shown in Figure 1 using a broken line. For retaining consistency in evolving the models, the model in this work also avoided this relationship. The proposed model ignores the role of regulators completely and it is replaced with stores’ vigilance and dissemination of information. Further, it has considerably relaxed the specifications of spatial and temporal dimensions. Time is driven by simulation iterations in which the state of the simulation world at time t − 1 drives the status at time t. Space has no constraints at all and, thus, the entity to entity reach time is zero.

3. The Existing and Proposed Models

3.1. Models Overview

The basic purpose of the Knowles model is to provide insights into the role of society’s general sensibility towards a self-organized food safety system at the consumption level. This model has three types of agents that are consumers, stores and regulators. Figure 2 provides the conceptualization of the Knowles model, using a specific scenario for illustration purposes [5]. Figure 2a provides the initial settings, at time t = 0, with three instances of consumers and stores each. Further, it shows that there are two regulators considered in this scenario. Figure 2b depicts the status of the scenario at t = 1, where three consumers C1, C2, and C3 visiting three different stores S1, S2, and S3 in the given order. The stores providing good quality food are colored green (S1 and S3) while the one colored orange (S2) offers bad quality food. In response to visiting S2, C2 marks it as a bad store as presented in Figure 2c. S3 also turned to a bad store at t = 2, however, it signals and a regulator (R1) visits it at t = 3, which is described in Figure 2d. R2 randomly visits S2 at t = 4, as shown in Figure 2e. Both the stores are now sterilized after visits of the regulators. They, however, remain in bad lists of consumers, showing the impression they obtained during their visits while these stores were offering bad quality (see Figure 2f).

Figure 3 illustrates the conceptualization of the proposed model. Figure 3a shows the status of the system at t = 1, where three consumers (C1, C2, and C3) are visiting three different stores (S1, S2, and S3) in order. This is similar to the previous conceptualization for Knowles model, presented in Figure 2b. In this scenario, two stores (S2 and S3) are bad. However, S2, shown in a bigger size, is vigilant and thus it acts on its quality, shown by self-loop in Figure 3a. It becomes sterilized at t = 2, while S3 remains indifferent, as shown in Figure 3b. consumers visiting S2 would now find it a good store. C2 has a bad perception recorded for it previously, as shown in Figure 3b. However, C2 might visit S2 after changing the old recorded perception in response to sharing views/information with friends. Figure 3c depicts the removal of S2 from the list of bad stores maintained by C2.

The rest of this section describes the detailed implementation of the existing and proposed models, before their simulations, in the following section.

3.2. Knowles Model for Food Safety Inspections

In the Knowles model [5], the three agents, it considers, use different operations, to implement the system. The consumers execute the CONSUME and HEAL processes. The regulators apply the TEST operation while the stores act according to the SPREAD and SIGNAL methods. These operations are explained below:

• CONSUME and HEAL: A healthy consumer consumes food from a store. However, the consumer has to wait to get healthy, if he/she has consumed contaminated food. A consumer randomly selects an uncontaminated store in his/her proximity—within the accessible range, which is neither declared contaminated by the store/regulators nor listed in the list of bad stores based on the previous experience of the consumer. The uncontaminated stores are listed as good stores. The consumer is declared sick when he/she consumes food, whose contamination—a randomly selected plausible health hazard value, exceeds his/her immunity level. Consequently, the store is added to the list of bad stores. A sick consumer performs the healing process when his/her healing counter is less than the recommended threshold. Figure 4a illustrates the whole process of CONSUME AND HEAL operations.
• SPREAD and SIGNAL: A store may randomly become contaminated. When such a situation arises, the store may decide to use the SIGNAL process to indicate the regulators about its state being contaminated. Figure 4b illustrates the SPREAD AND SIGNAL processes.
• TEST: A regulator visits a randomly selected store, among the signaling stores, within his/her proximity. The proximity (the assigned accessible range) of a regulator is usually more than the proximity of a consumer. Otherwise, the regulator visits a randomly selected store in his/her proximity. The store, in response to a regulator’s visit, becomes sterilized. Figure 4c provides the Test procedure.

Although the Knowles model provides two choices from stores to regulators. A store, in the first case, signals the regulators, as it gets contaminated. A regulator, then, quickly visits the store to fix (sterilize) it. The contaminated store, in the second case, is purely visited by a regulator randomly. This is because the store does not initiate the signaling process itself. Ideally, the option in which the stores signal themselves is the best choice. The practicality of this option is, however, debatable. The Knowles model [5] is taken as a background for this work.

3.3. The Extended Knowles Model

Zia et al. [28] extended the basic Knowles model, which is presented in Reference [28]. It excluded the SIGNAL process and introduced a new process for the stores called UPDATE—for managing food quality. It makes the model capable of the quality control of the food served by the stores. The UPDATE process manages both the progressive improvement and degradation of the food quality. The improvement or degradation, once selected, evolves for quite some time until a directional switch occurs. It means that the system avoids an abrupt change from one possible status to another but they evolve with time. The UPDATE process was also made capable of reflecting the effect of the crowd—the number of consumers at a specific time, available at the store. When the food quality of a store, currently having “c” consumers, is less than a prescribed threshold, a vigilant store brings-in drastic changes in food quality. In response, the quality is quantitatively improved proportionally to the number of consumers (the crowd). Figure 5a illustrates these differences.

Evaluation of the quality of the food was, then, integrated with other processes as well. Such as, the SPREAD of contamination is no longer an unconditional decision. A store becomes `really’ contaminated if the quality of the store is less than quality_threshold (a static value between 0.1 to 0.9). Figure 5 also depicts the updated SPREAD process (see Figure 5c). Similar changes are also implemented in TEST by regulators in which a store’s quality needs to be above the threshold to declare it as sterilized. These changes in Knowles’s Model are depicted in Figure 5b.

3.4. The Proposed Model

The model proposed in this paper considers two aspects. In our previous model, described in subSection 3.3, and presented in Reference [28] when a consumer was unable to find a safe store (a store which is uncontaminated and not in the list of bad stores), it used to wander. Wandering is the process of choosing another random location at the current iteration, which searches for a safe candidate store in the next iteration. This process is counter-intuitive, which works similarly to a store that starts signaling when it gets contaminated. The wandering process is omitted in the proposed model. In the absence of the wandering process, it is, however, expected to increase the number of consumers who would fail to find a safe store. So, the obvious question is, how to compensate for this issue. To cope with this, this work suggests evaluating the potential of information dissemination among the consumers using a presumed existing social network.

To realize it, a network between agents is established at the start of the simulation. The agents establish links with each other described next. Let “friends” be a 2D array maintained by an agent, which contains all of its friends, according to the following specifications:

• total connections depend on the connectivity index—the percentage of people connected to each other.
• many connections are local which depends on local connections index.
• the remaining connections, quite a few in number, are remote.

Figure 6 shows a sample setup of connectivity network for the proposed model, in which blue and red colors represent local and remote connections correspondingly. It means that many consumers now have friends connected through a static network, which is created at the start of simulations. This helps consumers to take recommendations from friends regarding a store, when they cannot find stores in their proximities, instead of wandering. To support data management and possible recommendations using social networking, a consumer registers a good store in a list called “green_stores”—maintaining healthy stores found in the response of visiting them by the consumers. A consumer would, now, ask a randomly chosen friend regarding a random store from the list of “green_stores”—maintained by the selected friend, when no healthy store is found in its proximity. This facilitates the selection process. Figure 7 presents the proposed extended CONSUME process, where social networking support is available to consumers. The model presented in Figure 7 is described next.

A consumer consumes the food of a store, if, he/she is healthy, but must wait to get healthy again if he/she is sick after consuming contaminated food. For food consumption, the consumer chooses a random store in his/her proximity (defined by her range of accessibility), which is neither declared contaminated by one of the regulators/stores nor already listed in bad stores based on a consumer’s previous experience. When an uncontaminated store is chosen, it is listed as a good store. The store is also added to “green_stores”—the list of stores that the consumer visited personally and found healthy. Otherwise, the consumer is considered as sick, if, contamination—a random value of plausible health hazard, imposed by the food is more than his/her immunity. Consequently, this lists the chosen store in the list of bad stores. A consumer also performs healing, if, he/she is sick, and his/her counter of healing is still less than the prescribed threshold. If, during the selection, no store is found in the proximity, the consumer would ask a randomly chosen friend, who will suggest a random store from the list of “green_stores”.

4. Model Simulation and Results

4.1. Setup

The proposed work is implemented and evaluated in NetLogo [29]. The simulation space, used for this work, is a cellular grid, comprises of 33 × 33 cells. Agents perform calculations and act in a sequence, however, the order of the sequence is randomly shuffled for each iteration. This simulations work has two parts. The former concerns with the detailed evaluation of the Knowles model [5] and its extended version presented by Zia et al. [28]. The latter part is dedicated to evaluating the model that further extended the current model extension of Knowles model by Zia et al [28].

The first part considers five cases, where they implement scenarios with increasing complexity as described below:

• Case 1: represents the Knowles’s Model but without using the Signaling option.
• Case 2: represents the Knowles’s Model using the Signaling option.
• Case 3: includes the UPGRADE process, and the concept of progressive improvement or degradation of food quality.
• Case 4: considers the UPGRADE process along with the impact of crowd on progressive quality improvement by the vigilant stores only.
• Case 5: considers the UPGRADE process along with the impact of crowd on progressive quality improvement by all stores.

The second part of the simulations extends Case 5 to additional cases described below:

• Case 5A: is the same as Case 5.
• Case 5B: Case 5 but without the wandering process.
• Case 5C: uses friends network when no store is found in the consumer’s proximity.

Each scenario is run for 100 ticks. A simulation for a given set of parameters is repeated 100 times. This is used to determine a more realistic average response. Some parameters are fixed for the simulation in this work such as 100 stores, 10 regulators, and 1000 consumers. Similarly, it used a fixed value of 0.5 for the connectivity index while 80% connections were kept local.

A sequence diagram showing simulation flow for the Knowles model [5] is presented in Figure 8. The simulator initiates running the simulation. A randomly selected store, then, initiate the spreading process. Five stores, then, initiate the optional signaling process. All the consumers start the consume process except those who are sick. The sick consumers initiate the healing process, instead. Regulators perform tests of the stores at the end. The cases ranging from 3 to 5 perform updates and do not use signaling. Figure 9 depicts a sequence diagram for the flow of simulation in the current extended model proposed by Zia et al. [28].

4.2. Analysis and Discussion

Presumably, Case 1 and 2 are the worst and best cases (in the given order) among the five cases described in the first part of the simulations. In the absence of stores’ vigilance, case 2 does not, even, offer a fair mechanism for the distribution of consumers. Case 3 incorporates a mechanism for progressive up-gradation (improvement) and degradation of quality by stores. Case 4 extends case 3 by introducing the concept of vigilance in stores. Case 5 provides a hypothetically exciting scenario, in which all the stores are vigilant. So, case 5 seems the best among the cases from 3 to 5.

This work is analyzed based on four statistical parameters described below:

• Sick Consumers: represents the number of sick consumers at a given time.
• Contaminated Stores: provides the number of contaminated stores at a given time.
• Sterilized Stores: represents the number of sterilized stores at a given time.
• Stores Usage: represented in terms of, average value and standard deviation, based on 100 executions for a given scenario.

The simulation results revealed that our assumption about sick consumers and case 5 to be the best case among the cases from 3 to 5 turned out to be true. Case 5 also performed better than case 2. This is because signaling alone (adopted in case 2) is unable to guarantee the presence of a regulator in the store’s proximity. So, the quick sterilization of the stores was not possible. However, the vigilance of all stores in case 5 allowed stores to sense a decline in the number of consumers and thus provided a guaranteed self-correction mechanism to help stores not lose the majority of their consumer base. Therefore, at least for the current settings in these simulations, we conclude that an active society is capable of self-organizing, even in the absence of regulatory compulsions. Figure 10a illustrates these results, which compares the five cases. It is also clear from this graph that case 3 proved even worse than case 1. This is because the progressive improvement and degradation in quality impose an additional restriction on contaminated stores, thus delaying the change in their status from contaminated to sterilized. This claim is also validated by the results for contaminated stores presented in Figure 10b. The other cases (labeled 1, 2, 4, and 5) in this graph provided similar results, which are comparable with the results shown in Figure 10a.

The results presented in Figure 10c illustrate comparable trends in the sterilization of stores for cases ranging from 3 to 5. The number of sterilized stores is based on the number of contaminated stores. A large number of stores is sterilized if there are a large number of contaminated stores and vice versa. Case 1 and 2, however, resulted in values higher than the remaining three cases, which is not comparable with their corresponding contaminated stores, presented in Figure 10b. The reason is again the restriction imposed by the progressive quality update, which prevents stores from an abrupt change in status, from contaminated to sterilized.

The last parameter suggested to check the fairness in stores’ utilization and it is shown in terms of the average standard deviation of consumers who visited these stores. Figure 11 illustrates the simulation response for stores’ usage. It shows that case 5 has the highest value. The standard deviation in stores’ usage represents fairness in consumers’ presence at stores. Therefore, it verifies the hypothesis that “more vigilance in correction measures ensures a large fraction of interested population available as compared to indifferent attitude”. This may increase the robustness and dependability of the system. However, case 4 does not support this hypothesis, which has the least standard deviation among the cases ranging from 3 to 5. This suggests that the hypothesis might be rephrased as: “more vigilance in correction measures ensures a large fraction of interested population available as compared to indifferent attitude, provided that majority of the service providers are vigilant and sensitive to implications of “the wisdom of crowds””.

Figure 12 provides the simulation results for the extended scenarios of case 5 to investigate the impact of social networking on decision making. Case 5 described earlier is the same as Case 5A, while Case 5B is the same as case 5 but without the wandering process. Case 5C uses the friends’ network to get information about a good healthy store when no such store is available for the consumer in his/her proximity. This case, naturally, required an additional parameter, which shows the consumers who were unable to get a store when they needed it. This work termed this parameter as consumers with no store available.

This extension aimed at decreasing the number of consumers who are unable to find a suitable store in their proximity by taking help from their friends. The comparison of results presented in Figure 12 successfully achieved this goal. It is clear that case 5B has the worst outcome, however, this is understandable in the absence of a wandering mechanism. Even with wandering, the results described earlier revealed that a substantial amount of consumers were unable to find a healthy store. The proposed model, however, ensured that throughout the simulation, very few consumers were unable to find a good healthy store (see Case 5C in Figure 12). The rest of the results were quite similar to what was discussed earlier and are, therefore, not included.

5. Conclusions

This paper presented an agent-based model for managing food safety. It extended the simple model of food safety, proposed by Knowles [5], based on three entities of interest namely the consumers, stores, and regulators. A brief description of this extended model was published earlier in Reference [28]. However, this work further extended the proposed model and evaluated it in detail. The simulation results revealed that a large fraction of consumers had access to good quality food when stores adopted more vigilance on corrective measures. The self-organizing behavior of people was also learned to be an interesting finding, in this work. It was demonstrated that active societies are capable of self-organization, even if the regulatory restrictions do not exist.

The self-organization further extended its functional scope when the model was equipped with information sharing through social networking. The implications of these findings are considerable.

System components, which are capable of taking self-organized corrective measures result-in improved system stability. This is because these measures resist against actions which might lead the system towards instability.