NegoSim: A Modular and Extendable Automated Negotiation Simulation Platform Considering EUBOA

Abstract

In recent years, the research community has become increasingly interested in automated negotiation. It has been used in real-world systems such as autonomous vehicle transportation systems, smart grids, and e-commerce. Considering the broad range of applications, automated negotiation is of great use and interest to engineers, developers, and industry-focused researchers. Therefore, some researchers have developed platforms for automated negotiation. However, these systems do not provide complete control over the development of various protocols or analytic tools. This paper introduces NegoSim, a new negotiation simulator that introduces a new negotiation framework called EUBOA, though it also works on the well-known BOA framework. NegoSim is completely modular, with different APIs allowing users full control over developing and modifying automated negotiation agents, different assessments, protocols, and utility spaces, as well as the ability to create or modify different GUIs. NegoSim also includes predefined agents, protocols, utility space, and analytic tools, making it an ideal platform for researchers.

1. Introduction

Negotiation is one of the most important activities in human lives for resolving conflicts and reaching agreements, but few negotiators are professionals in this context, so automated negotiation (AN) has gained more interest in the research community in recent years [1]. Recent researchers have applied AN to real-world systems such as transportation systems for autonomous vehicles [2], smart grids [3], Wi-Fi channel assignment [4], e-commerce [5], and supply chain management [6].

Considering their broad potential application and recent research, AN platforms and applications are important for use by engineers, developers, and industry-focused researchers. Platforms in the AN field include Pocket Negotiator [7], Genius [8], and NegMAS [9]. Pocket Negotiator is a human-to-human negotiation support system that recommends bids to send to an opponent as well as the best time to accept the opponent’s offer, but it does not cover all components of AN. Genius is another platform (General Environment for Negotiation with Intelligent multi-purpose Usage Simulation). This platform provides researchers with functions for developing agents, as well as a good repository of various negotiation strategies. Since 2010, Genius has served as the primary platform for the Automated Negotiation Agents Competition (ANAC) [10,11,12]. Although Genius is one of the most well-known AN platforms, it does not provide any APIs for developing new protocols, instead restricting users to predefined protocols, which can make analyzing the agents difficult. Because both Pocket Negotiator and Genius use a static preference during the negotiation session, researchers developed NegMAS (Negotiations Managed by Agent Simulations/Negotiation MultiAgent System), one of the most recently developed platforms, to address this shortcoming. All of these platforms generate massive amounts of data, much of it redundant, for researchers to sort through. Negowiki [13] compiled a set of design rules for selecting the best protocol for the problem. The author describes a tool for generating a wide variety of negotiation scenarios, a set of high-level metrics for the characterization of negotiation scenarios, a testbed for evaluating protocol performance with various scenarios, and a repository of previously analyzed negotiation protocol performance data. It was, however, difficult to manage and to force it to stop updating. The above findings indicate that, while there are numerous highly useful platforms available, they do not provide users with complete control over the development of various protocols or analytic tools.

Analyzing the ANAC results revealed the evolution of strategies and important factors in the development of competition. Analyzing ANAC results reveals a stream of negotiation strategies and important factors for developing competition [12]. The BOA architecture is a critical framework in terms of individual component performance in ANAC competitions. This model identifies three components of a negotiation strategy: the bidding strategy, the opponent model, and the acceptance strategy [14]. This architecture assesses and compares the negotiation performance of using an advanced BOA framework, as well as providing an overview of the factors influencing the final performance. For example, high-performance opponent-modeling techniques have attracted significant attention in the field [15]. Furthermore, effective strategies can be achieved in competitions by combining the modules of the superior agent’s strategies, depending on the opponent’s strategies and negotiation environments. A fixed set of modules is used in several sophisticated existing agent strategies. Therefore, studies on negotiation strategies focusing on the modules have been deemed significant and influential. Recent negotiation strategies used the Q-learning reinforcement learning approach for the bidding component of the BOA framework [16].

Agents represent their users and negotiate on their behalf, so they should first understand their user’s preferences. However, most of the time, large domain sizes make it impossible for a user to rank all possible domain outcomes. It is also sometimes impossible to consider a mathematical model for some domains due to interdependencies between some domain issues [17]. Users prefer to determine the order of bids, and the agent must model its user (user modeling phase). However, when the user model must be completed, the agent may request the rankings of specific outcomes (elicitation phase). All of these challenges led to the development of the EUBOA (elicitation strategy, user modeling, bidding strategy, opponent modeling, acceptance strategy) framework. This framework simplifies the development of an agent in uncertain situations.

To address these shortcomings, the NegoSim negotiation simulator was proposed and developed. NegoSim provides APIs that allow for simple agent development; its well-structured design ensures that users encounter few problems. The only decision an agent’s developer must make is which bid to send. In contrast to other platforms, NegoSim allows users to create analysis modules with their preferred formulas and protocols, as opposed to platforms that limit users to predefined protocols and formulas. Because NegoSim is completely modular and extendable, users have full control over developing their own agents, protocols, and analytic tools, as well as modifying or creating GUIs. Notably, NegoSim does not require users to use the GUI; instead, they can use NegoSim as a library and write their code directly. Because of its perfect architecture, NegoSim is also simple to learn.

The main contribution of this paper is the introduction of a newly developed negotiation simulator that overcomes the shortcomings of previous platforms, along with descriptions of its new features and components. We would like to highlight the following contributions.

• This paper proposes NegoSim, a new negotiation simulator. Protocols, negotiating parties, and analytic tools are three critical components of each negotiation platform, and these three components are completely modular and extendable in NegoSim.
• We develop NegoSim and make it available to all users. This modularity gives users complete control over the development of negotiating agents, negotiation protocols, and analytic tools based on their specific needs.
• NegoSim introduces a new negotiation framework called EUBOA, although it is based on the underlying well-known BOA framework.
• These analytic tools enable researchers to examine massive amounts of redundant negotiation data. Examples of predefined agents, protocols, and analytic tools demonstrate that NegoSim is an appropriate platform for researchers.

The remainder of the paper is structured as follows: Section 2 describes the architecture of NegoSim and an agent; Section 3 describes the NegoSim environment, and Section 4 summarizes the paper.

2. NegoSim and Agent Architecture

Participants in NegoSim, like in the real world, sit at a table called the NegoTable and exchange offers; there are laws for exchanging offers known as the protocol. The protocol solicits bids from agents and places them on the table; it runs all processes in the negotiation, such as how and when negotiators must exchange offers, when the negotiation ends because an agreement is reached, noting that the deadline has been met, and negotiators walking away. The protocol owns the negotiation table, the clock (to control negotiation time), and all existing laws; each negotiation party can access it to ask questions, and the protocol checks the negotiation laws to determine an answer. Another important function of the protocol is analysis, which requires the analysis entity to analyze the negotiation session after receiving each party’s offer. As in opponent and user modeling, the analysis entity has access to the exchanged offers as well as each party’s estimations. See Figure 1.

As previously stated, the parties are the most important entities in a negotiation, which consists of three major components: bidding strategy, opponent modeling, and acceptance strategy. These three components were introduced as the BOA framework in [14]. NegoSim proposes a new framework called EUBOA.

Figure 2 shows the NegoSim architecture. The protocol, as shown in Figure 2, controls all communications during the negotiation session. Parties wait until the protocol requests that they submit a bid. When they receive the request, each party can ask the protocol any question about the negotiation session (remaining time, previous offers, etc.), and the protocol will respond following the negotiation session rules. Parties are then free to submit a bid, accept the opponent’s offer, or end the negotiation. The analysis entity examines each event during the negotiation session and at the end of the negotiation, saving the result as a Python serialized (pickled) file and printing the results in the terminal.

Five components of the NegoSim are modular, namely, the negotiation session and tournament GUI, parties, protocol, analytical toolbox, and utility spaces, giving users full control over their agents, protocols, GUIs, analytic tools, and utility spaces via simple APIs. There are some predefined negotiating agents, a negotiation protocol called the stack alternating offer protocol (SAOP) [18], some GUIs, and some analytic tools. These features make NegoSim user-friendly. Because NegoSim preference profiles are in XML file format, they are compatible with Genius preference profiles. As a predefined utility space in the current version of NegoSim, there is a linear additive utility space [19].

2.1. Agent

The main feature of NegoSim is providing a tool for training agents to negotiate with one another. Therefore, one of the most important entities in any AN session is an agent, which negotiates on behalf of its user against other agents. NegoSim supports two methods for creating an automated agent:

• Directly developing an agent using agent-development APIs: NegoSim provides some simple APIs for developing agents. The only thing the developer needs to consider is which bid to send when the protocol requests one (agents communicate with each other by exchanging offers via a protocol) without distracting the user with other issues. The developer must only consider making an offer, which is an initial offer to begin a negotiation, a counteroffer in response to the opponent, or ending the negotiation. The agent developers should then consider a method for determining the best time to accept the opponents. They can also create an opponent modeling entity to approximate the opponent’s preferences.
• Creating an agent with the BOA framework for certain conditions and the EUBOA framework for uncertain conditions, see Section 2.1.2.

2.1.1. Utility Space

Every agent has its own preferences and utility space, which maps bids to a real number in the range [0, 1] where 0 and 1 represent bids with the least and most desirability for the user, respectively. One of the most common utility spaces is the linear additive utility space, which is defined in NegoSim, but unlike other platforms, NegoSim does not limit the user to using the predefined utility spaces. Using the existing APIs, users can create their own utility space.

2.1.2. EUBOA Framework

As previously stated, each agent includes at least the BOA components, as well as an uncertainty condition in which the user’s preference is unclear, and NegoSim adds two more components. Users can create their agents directly in NegoSim or via BOA [14] or EUBOA. The EUBOA framework assists researchers in determining the efficacy of each component of the negotiation strategy. They can also devise a new negotiation strategy by determining the best combination of components. See Figure 3.

EUBOA is used when users do not want to or cannot reveal all of their preferences, and the agent must negotiate under uncertain conditions. Because all negotiators are familiar with the negotiation domain, NegoSim provides an initial preference profile in which the weight of all issues and the corresponding value of each issue are equal. This initial preference profile should be changed so that the elicitation strategy component first inquires about the initial order of bids:

$${\omega }_1 \preccurlyeq {\omega }_2 \preccurlyeq \dots \preccurlyeq {\omega }_d$$

(1)

In Equation (1), $\omega$ refers to a possible domain space outcome, and $\preccurlyeq$ determines the relative value of two outcomes; for example, ${\omega }_1 \preccurlyeq {\omega }_2$ means that the value of ${\omega }_2$ is greater than the value of ${\omega }_1$ in the utility space of user preferences. The user modeling component is then given these initial bid orders. This component modifies the initial preference profile to better understand user preferences. When user modeling needs to know the rank of a bid, it sends a request to the elicitation strategy component, which then asks the user about the bid’s rank.

Another component is opponent modeling, which uses the bid history on the negotiation table to create an opponent preference profile. Estimation components are provided to the analysis entity by both the user and opponent modeling components. The bidding strategy component receives both estimation components. This component, known as the next bid, uses these estimation components and the remaining time in the negotiation to determine the best bid to send to the opponent. The bidding strategy sends the discovered bid to the acceptance strategy, which decides whether or not to accept the opponent’s offer in response to the next bid. The procedure is outlined below:

• Elicitation strategy: This component solicits the user’s initial partial preferences as well as the modeling component question.

  a.

  Input: a possible outcome of domain space

  b.

  Output: the rank of the bid

• Estimation components: The user modeling and opponent modeling components comprise estimation components.

  c.

  User modeling: This component predicts the user’s preference profile.

  i.

  Input: orders the possible outcomes of domain space

  ii.

  Output: updates user preferences

  d.

  Opponent modeling: This component predicts the opponent’s preference profile.

  i.

  Input: opponent’s bids.

  ii.

  Output: the estimated opponent preference profile.

• Bidding strategy: This component determines the best bid to send to the opponent.

  e.

  Input: the user’s preference profile generated by the user modeling component, the opponent’s preference profile generated by the opponent modeling component, and the negotiation’s remaining time

  f.

  Output: a bid to send to the opponent.

• Acceptance strategy: This component determines whether or not to accept the opponent’s offer.

  g.

  Input: user preference profile generated by the user modeling component, opponent preference profile generated by opponent modeling, bid generated by the bidding strategy to send to the opponent, and remaining time in the negotiation

  h.

  Output: an action: accept the opponent’s offer, send a bid, or end the negotiation

NegoSim supports the implementation of BOA and EUBOA frameworks in certain and uncertain conditions, which is at the core of NegoSim. Furthermore, users can develop their own components and create all possible agents by combining these EUBOA model components.

Table 1. Example of existing EUBOA model.

Elicitation Strategy	User Model	Bidding Strategy	Opponent Model	Acceptance Strategy
$E_1$	$U_1$	$B_1$	$O_1$	$A_1$
$E_2$	-	$B_2$	-	-

shows two elicitation strategies (E₁, E₂), one user model (U₁), two bidding strategies (B₁, B₂), an opponent modeling (O₁), and an acceptance strategy (A₁). All possible agents using these components are agent1($E_1{U}_1{B}_1{O}_1{A}_1$), agent2($E_1{U}_1{B}_2{O}_1{A}_1$), agent3($E_2{U}_1{B}_1{O}_1{A}_1$), agent4($E_2{U}_1{B}_2{O}_1{A}_1$). All of these agents can be created in two ways using existing components: static and dynamic. In a static manner, users can create these agents one by one by combining these components. Dynamically, users can create all possible agents using five nested loops.

2.2. Protocol

The protocol is the most important entity in the NegoSim architecture. Protocol runs all negotiation processes (asking agents to send bids, controlling the time, checking negotiation results, etc.). NegoSim provides APIs that enable users to easily create and customize their protocols.

2.3. Negotiation Table

The negotiation parties and exchanged offers are listed in the NegoTable. The protocol owns the table, so when it receives a bid from a party, it converts it to an offer by adding the time the bid was received and then places it on the negotiation table. To arrive at the exchanged offers, each party must ask about the protocol, and the protocol must then respond to the party according to its rules. Because exchanged bids are stored on the negotiation table, this architecture achieves the need for developers to store them (if they need to).

2.4. Analysis Module

The analysis module is one of the advantages of NegoSim over other platforms; this module gives developers complete control over analyzing their agents without having to navigate large amounts of redundant data. Developers can use NegoSim’s APIs to create their own analysis modules. The protocol then calls for analysis, which tracks negotiation events like exchanged bids and parties’ estimations (for example, the opponent model and user model in uncertain conditions); it then analyzes the entire negotiation session/tournament. There are two kinds of analyses: negotiation sessions and tournaments. The tournament analysis uses the data from the negotiation session analysis. Researchers can use NegoSim’s predefined analysis modules for both negotiation sessions and tournaments.

3. Environment of NegoSim

As previously stated, the NegoSim GUI is modular, allowing users to create their own GUI using existing APIs, though NegoSim also provides predefined GUIs. The basic GUI shown in Figure 4 informs users about the existing components and entities in NegoSim. A user can also see all possible outcomes of a domain (e.g., Job domain) and their utility (utility space) concerning the selected preferences (Jobs_util1.xml and Jobs_util2.xml). The user can access other two existing GUIs via the basic GUI, one for negotiation sessions and the other for negotiation tournaments.

3.1. Negotiation Session

Figure 5 shows how the negotiation session is used to investigate how two parties negotiate with each other during the negotiation time. The user must first configure the negotiation session parameters before clicking the “Start Negotiation” button. When the negotiation session is finished, NegoSim displays the outcome to the user.

3.2. Negotiation Tournament

Figure 6 shows a negotiation tournament used to analyze the negotiating parties. The tournament, like a negotiation session, is divided into two main parts, depicted in the Figure by black and blue frames.

The user enters tournament parameters in the inside of the black frame and can set repetition numbers greater than 1. Following the conclusion of the tournament, the results are displayed as a table in the inside of blue frame. The user can, however, create table charts using the menu bar on the right side of the results window [20]; see Figure 7. The analysis tool, which users can easily adjust or develop separately, provides the negotiation session and tournament data.

4. Conclusions

AN has attracted the interest of the research community in recent years. AN has had several real-world applications, including autonomous vehicle transportation systems, smart grids, and e-commerce. This paper proposed and developed the NegoSim negotiation simulator. Protocols, negotiating parties, and analytic tools are three critical components of each negotiation platform, and these three components are completely modular and extendable in NegoSim. This modularity gives users complete control to develop negotiating agents, negotiation protocols, and analytic tools based on their specific needs. NegoSim, in particular, introduced a new negotiation framework called EUBOA, which works alongside the well-known BOA framework. These NegoSim analytic tools assist researchers in analyzing large amounts of negotiation logs to understand insightful findings. NegoSim has opened the source code of predefined agents, protocols, and analytic tools as a proper AN platform for researchers.