A General Statistical Physics Framework for Assignment Problems

Abstract

Linear assignment problems hold a pivotal role in combinatorial optimization, offering a broad spectrum of applications within the field of data sciences. They consist of assigning “agents” to “tasks” in a way that leads to a minimum total cost associated with the assignment. The assignment is balanced when the number of agents equals the number of tasks, with a one-to-one correspondence between agents and tasks, and it is and unbalanced otherwise. Additional options and constraints may be imposed, such as allowing agents to perform multiple tasks or allowing tasks to be performed by multiple agents. In this paper, we propose a novel framework that can solve all these assignment problems employing methodologies derived from the field of statistical physics. We describe this formalism in detail and validate all its assertions. A major part of this framework is the definition of a concave effective free energy function that encapsulates the constraints of the assignment problem within a finite temperature context. We demonstrate that this free energy monotonically decreases as a function of a parameter $\beta$ representing the inverse of temperature. As $\beta$ increases, the free energy converges to the optimal assignment cost. Furthermore, we demonstrate that when $\beta$ values are sufficiently large, the exact solution to the assignment problem can be derived by rounding off the elements of the computed assignment matrix to the nearest integer. We describe a computer implementation of our framework and illustrate its application to multi-task assignment problems for which the Hungarian algorithm is not applicable.

1. Introduction

An assignment problem can be interpreted as a problem of resource allocation in which a certain number of “tasks” are performed by some “agents” so that the total cost of pairing tasks with agents is minimized. There are many ways to constrain this problem, such as imposing one-to-one mapping (the balanced problem), searching for a specific number of assignments (the k-cardinality assignment problem), imposing constraints on the resources (the generalized assignment problem), or allowing agents to perform multiple tasks or reversely having some tasks performed by more than one agent. For reviews on the variety of assignment problems and the algorithms that have been proposed to solve them, we refer the reader to Refs. [1,2,3] and references therein. Assignment problems are fundamental combinatorial optimization problems. As such, solving such problems is a significant concern in operational research, economics, and data sciences, among other disciplines. Assignment problems have been and continue to be a focal point of investigation for mathematicians, statisticians, computer scientists, and even physicists. In this paper, we propose a general framework that regroups all the assignment problems mentioned above. We develop an approximate solution to this problem based on statistical physics and prove that it converges efficiently to an exact solution for non-degenerate as well as degenerate assignment problems.

Let us first define the assignment problem. Let T be the set of tasks and A the set of agents, with possibly different cardinalities $\left|T\right|=N_2$ and $\left|A\right|=N_1$. If we define $C(i,j)$ as the cost of assigning task i to the agent j, the balanced assignment problem can be stated as finding a transportation matrix G between T and A whose elements are non-negative integer numbers and that minimizes

$$U=\sum _{i\in T}\sum _{j\in A}C(i,j)G(i,j).$$

(1)

In the special case of $N_1=N_2$, with a one-to-one assignment of tasks to agents (the balanced assignment problem), G corresponds to a permutation of $\{1,\dots ,N\}$. In the more general case, $N_1$ and $N_2$ can be different (unbalanced assignment), the number of pairings k may be imposed with $0<k\le min(N_1,N_2)$ (k-cardinality assignment problem), and/or tasks and agents may be assigned to more than one agents and tasks, respectively. In those general cases, G remains binary but may not be a permutation matrix (in the specific case of the k-cardinality problem, for example, G is a partial permutation matrix of rank k).

As mentioned above, assignment problems are basically combinatorial optimization problems that can be solved numerically using a linear programming model or, more specifically, a linear integer programming method. Those methods, however, are theoretically $NP$-complete. There are fast heuristics that can solve such problems, such as Dantzig’s simplex method [4]. The assignment problem, however, is a specialized case of integer linear programming that can be recast and solved by an exact polynomial algorithm. For example, the most common algorithm for solving the balanced assignment problem has its origin in the work of Jacobi [5]. It was “rediscovered” 60 years later by [6] and is now dubbed the Hungarian algorithm. It is a global algorithm that iteratively identifies assignments between agents and tasks. It is polynomial in running time ($O\left(N^4\right)$ or $O\left(N^3\right)$ depending on the implementation). While it is serial by nature, there are many recent efforts to parallelize this algorithm (see, for example [7,8,9], and references therein).

The Hungarian algorithm can be adapted to many unbalanced assignment problems. If there are $N_2$ tasks and $N_1$ agents, with $N_2>N_1$, for example, it is possible to add $N_2-N_1$ “dummy” agents, solve the balanced assignment problem of size $N_2$, and then only retain the assignments to the actual $N_1$ agents. If each agent can perform up to M tasks, the textbook solution is then to create M copies of each agent prior to using the Hungarian algorithm. This approach, however, may not lead to the expected solution. Consider a problem with fiv tasks and three agents, with each agent allowed to perform up to three tasks. The option of creating three copies of each agent and then solving the corresponding unbalanced assignment problem may lead to one agent performing two tasks, a second agent performing three tasks, and the last agent not performing any task, possibly violating a constraint that each agent needs to perform at least one task. Several methods have been proposed to circumvent this problem, either by dividing the problem into multiple subproblems [10,11] or by using random algorithms such as the ant colony algorithms [12,13]. It is unclear whether these algorithms can identify the optimal solution (they have been shown to fail in some cases, see for example [14]) or how they scale for large problems.

In this paper, we propose a radically different approach to solving a general class of assignment problems using continuous systems. Our approach is motivated by statistical physics. It is a full generalization of the method developed in a previous paper to solve balanced assignment problems [15]. In the following, we will use the generic term “multi-assignment” to represent an assignment problem with agents possibly performing multiple tasks, or with tasks assigned to multiple agents. In this paper, we aim to

• Introduce and substantiate a continuous framework for addressing potentially multifaceted assignment problems involving multiple tasks and/or multiple agents, leveraging principles from statistical physics;
• Demonstrate that, under generic circumstances where the multi-assignment problem possesses a unique solution, the aforementioned framework ensures convergence to that solution with arbitrarily high accuracy, in terms of both cost (referred to as energy) and assignment matrix,
• Present a modified approach capable of identifying at least one solution for degenerate multi-assignment problems that feature multiple solutions;
• Demonstrate that the implementation of this framework can be efficiently parallelized.

We emphasize that this formalism is not a mere adaptation but a full generalization of the framework we develop for solving assignment problems [16]. In particular, new features of this paper are as follows:

• A method to account for the fact that each task can be assigned to a variable number of agents and, on the other hand, that each agent can be assigned a variable number of tasks. To that end, we introduce indicator functions over the sets of tasks and agents that are optimized along with the transportation plan.
• The establishment of proofs of validity and convergence of our algorithm for those general multi-assignment problems. The main results are provided in the text, while the proofs themselves are relegated to the Appendices for more clarity. For the balanced assignment problem, these proofs rely heavily on the fact that the corresponding transportation matrices are permutation matrices that are the extreme points of the well-characterized convex set of doubly stochastic matrices (according to the Birkhoff–von Neumann theorem [17,18]). For more general multi-assignment problems, the transportation matrices also belong to a convex set, with properties akin to the Birkhoff–von Neumann theorem (these properties are discussed in Appendix A). The use of these properties to derive the convergence of our algorithm for solving multi-assignment problems is new.

The paper is organized as follows. In Section 2 Section 3 and Section 4, we describe in detail the general framework we propose for solving multi-assignment problems. Proofs of all important properties of this framework are provided in the appendices. Section 5 briefly describes the implementation of the method in a C++ program, Matching V1.0. In Section 6, we present some applications, along with a comparison to current algorithms.

2. A General Formulation of the Assignment Problem

We consider two sets of points, $S_1$ and $S_2$, with cardinalities $N_1$ and $N_2$, respectively. We represent the cost of transportation between $S_1$ and $S_2$ as a matrix C whose elements $C(i,j)$ are positive, for all $(i,j)\in [1,N_1]\times [1,N_2]$. We set the number of assignments between points in $S_1$ and $S_2$ to be a strictly positive integer number k, a constant given as input to the problem. A point i is assigned to $n_1\left(i\right)$ points in $S_2$, with $n_1\left(i\right)$ belonging to $[p_{min}\left(i\right),\dots ,p_{max}\left(i\right)]$, where $p_{min}\left(i\right)$ and $p_{max}\left(i\right)$ are non-negative integers satisfying $0\le p_{min}\left(i\right)\le p_{max}\left(i\right)\le N_2$. Similarly, a point j is assigned to $n_2\left(j\right)$ points in $S_1$, with $n_2\left(j\right)$ belonging to $[q_{min}\left(j\right),\dots ,q_{max}\left(j\right)]$, where $q_{min}\left(j\right)$ and $q_{max}\left(j\right)$ are non-negative integers integers satisfying $0\le q_{min}\left(j\right)\le q_{max}\left(j\right)\le N_1$. Using the traditional “task-agent” for the formulation of an assignment problem, $S_1$ represents the tasks, and $S_2$ the agents. $n_1\left(i\right)$ is the number of agents needed to perform task i, while $n_2\left(j\right)$ is the number of tasks that can be performed by agent j. $p_{min}\left(i\right)$ and $p_{max}\left(i\right)$ represent the minimum and maximum number of agents that are assigned to a task i, respectively, while $q_{min}\left(j\right)$ and $q_{max}\left(j\right)$ represent the minimum and maximum number of tasks that can be assigned to an agent j, respectively. We use a general definition of these boundaries, in that we allow each task and each agent to have their own limits. The multi-assignment problem is then framed as the search for an assignment matrix G that defines the correspondence between points in $S_1$ and points in $S_2$. This matrix is found by minimizing the assignment cost U, defined as

$$U(G,C)=\sum _{i,j}G(i,j)C(i,j).$$

(2)

The summations encompass all i in $S_1$ and j in $S_2$. The objective is to locate the minimum of U given the values of $G(i,j)$ that adhere to the following constraints:

$$\begin{array}{c}\forall i,{\displaystyle \sum _j}G(i,j)=n_1\left(i\right)\hfill \\ \forall j,{\displaystyle \sum _i}G(i,j)=n_2\left(j\right)\hfill \\ {\displaystyle \sum _i\sum _j}G(i,j)=k\hfill \\ \forall (i,j),G(i,j)\in \{0,1\}\hfill \\ \forall \left(i\right),n_1\left(i\right)\in [p_{min}\left(i\right),\dots ,p_{max}\left(i\right)]\hfill \\ \forall \left(j\right),n_2\left(j\right)\in [q_{min}\left(j\right),\dots ,q_{max}\left(j\right)],\hfill \end{array}$$

(3)

where k is the actual number of task–agent pairs. The numbers $p_{min}\left(i\right)$, $p_{max}\left(i\right)$, $q_{min}\left(j\right)$, $q_{max}\left(j\right)$, and k are problem-specific and given. They do satisfy some constraints, such as $0\le p_{min}\left(i\right)\le p_{max}\left(i\right)\le N_2$ and $0\le q_{min}\left(j\right)\le q_{max}\left(j\right)\le N_1$, as described above, as well as $0<k\le {\sum }_{i=1}^{N_1}{p}_{min}\left(i\right)$ and $k\le {\sum }_{j=1}^{N_2}{q}_{min}\left(j\right)$. We store those numbers in four vectors, $P_{min}=(p_{min}\left(1\right),\dots ,p_{min}\left(N_1\right))$, $P_{max}=(p_{max}\left(1\right),\dots ,p_{max}\left(N_1\right))$, $Q_{min}=(q_{min}\left(1\right),\dots ,q_{min}\left(N_2\right))$, $Q_{max}=(q_{max}\left(1\right),\dots ,q_{max}\left(N_2\right))$.

Equation (3) recap most of the standard linear assignment problems:

(i)

If $k=N_1=N_2$ and $p_{min}\left(i\right)=p_{max}\left(i\right)=q_{min}\left(j\right)=q_{max}\left(j\right)=1$ for all i and j, we recover the balanced assignment problem,

(ii)

If $p_{min}\left(i\right)=q_{min}\left(j\right)=0$ and $p_{max}\left(i\right)=q_{max}\left(j\right)=1$ for all i and j, the equations correspond to the k-cardinality assignment problem [19,20,21].

In Equation (3), $G(i,j)$, $n_1\left(i\right)$ and $n_2\left(j\right)$ are unknown, defining a total of $N_1{N}_2+N_1+N_2$ variables. The solution to the general assignment problem is given by the functions $n_1^{*}$ and $n_2^{*}$ on $S_1$ and $S_2$, respectively, which identify which tasks and which agents are involved in the optimal assignment, the transportation matrix $G^{*}$ that defines these correspondences, and the minimum assignment cost $U^{*}=U(G^{*},C)$.

Optimizing Equation (2) subject to the constraints Equation (3) constitutes a discrete optimization challenge, specifically an integer linear programming problem. To address this challenge, we employ a statistical physics methodology, transforming it into a temperature-dependent problem involving real variables. The integer optimal solution is then obtained as the temperature approaches zero.

2.1. Effective Free Energy for the General Assignment Problem

Solving the multi-assignment problem entails determining the minimum of the function defined in Equation (2) across the potential mappings between the two discrete sets of points under consideration. Identifying this minimum is tantamount to identifying the most probable state of the system it characterizes. This “system” encompasses the various binary transportation plans, also referred to as assignment matrices, between $S_1$ and $S_2$ that adhere to the constraints Equation (3). Each state within this system corresponds to assignment matrix G with its associated energy $U(G,C)$ as defined in Equation (2). The probability $P\left(G\right)$ linked with an assignment matrix G is given by

$$P\left(G\right)=\frac{1}{Z\left(\beta \right)}{e}^{-\beta U(G,C)}.$$

(4)

In this equation, $\beta$ is the inverse temperature, namely $\beta =1/k_{B}T$, where $k_B$ is the Boltzmann constant and T is the temperature, and $Z\left(\beta \right)$ is the partition function computed over all states of the system. As this system is defined by the assignment matrices G and by the marginals of G, $n_1$, and $n_2$, the partition function is equal to

$$\begin{array}{ccc}\hfill Z_{\beta }& =& e^{-\beta {\mathcal{F}}_{\beta }}\hfill \\ & =& \sum _{G(i,j)=0}^1\sum _{n_1\left(i\right)=p_{min}\left(i\right)}^{p_{max}\left(i\right)}\sum _{n_2\left(j\right)=q_{min}\left(j\right)}^{q_{max}\left(j\right)}{e}^{-\beta U(G,C)},\hfill \end{array}$$

(5)

where $\mathcal{F}\left(\beta \right)$ is the free energy of the system. The practicality of this free energy is limited due to the inability to compute it explicitly. We develop a method for approximating it through the saddle point approximation technique.

Taking into account the constraints in Equation (3), the partition function can be written as

$$\begin{array}{ccc}& & Z_{\beta }=\sum _{G(i,j)=0}^1\sum _{n_1\left(i\right)=p_{min}\left(i\right)}^{p_{max}\left(i\right)}\sum _{n_2\left(j\right)=q_{min}\left(j\right)}^{q_{max}\left(j\right)}{e}^{-\beta \sum _{i,j}C(i,j)G(i,j)}\hfill \\ & & \prod _i\delta \left(\sum _{j}G(i,j)-n_1\left(i\right)\right)\prod _j\delta \left(\sum _{i}G(i,j)-n_2\left(j\right)\right)\hfill \\ & & \delta \left(\sum _{i,j}G(i,j)-k\right).\hfill \end{array}$$

(6)

The three sums impose that $G(i,j)$, $n_1\left(i\right)$, and $n_2\left(j\right)$ take integer values within some ranges defined by the constraints Equation (3). The additional constraints are imposed through the delta functions. We employ the Fourier representation of those delta functions, thereby introducing additional auxiliary variables x, $\lambda \left(i\right)$, and $\mu \left(j\right)$, with $i\in [1,N_1]$ and $j\in [1,N_2]$. The partition function is then given, after reorganization, by (up to a multiplicative constant) by

$$\begin{array}{ccc}\hfill Z_{\beta }=& & {\int }_{-\infty }^{+\infty }\prod _{i}d\lambda \left(i\right){\int }_{-\infty }^{+\infty }\prod _{j}d\mu \left(j\right){\int }_{-\infty }^{+\infty }dx{e}^{ß\beta kx}\hfill \\ & & \sum _{n_1\left(i\right)=p_{min}\left(i\right)}^{p_{max}\left(i\right)}\sum _{n_2\left(j\right)=q_{min}\left(j\right)}^{q_{max}\left(j\right)}{e}^{\beta \left(\sum _iß\lambda \left(i\right)n_1\left(i\right)+\sum _jß\mu \left(j\right)n_2\left(j\right)\right)}\hfill \\ & & \sum _{G(i,j)=0}^1{e}^{-\beta \sum _{i,j}G(i,j)(C(i,j)+ß\lambda \left(i\right)+ß\mu \left(j\right)+ßx)},\hfill \end{array}$$

(7)

where $ß$ is the imaginary unit (${ß}^2=-1$). Note that we have rescaled the variables x, $\mathit{\lambda }$, and $\mathit{\mu }$ by a factor $\beta$ for consistency with the energy term. Conducting the summations over the variables $G(i,j)$, $n_1\left(i\right)$, and $n_2\left(j\right)$, we get

$$Z_{\beta }={\int }_{-\infty }^{+\infty }\prod _{i}d\lambda \left(i\right){\int }_{-\infty }^{+\infty }\prod _{j}d{\mu }_j{\int }_{-\infty }^{+\infty }dx{e}^{-\beta F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)},$$

where $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$ is a functional, or effective free energy, that depends on the variables $\mathit{\lambda }$, $\mathit{\mu }$, and x and is defined by

$$\begin{array}{ccc}& & F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)=\hfill \\ & & -\frac{1}{\beta }\sum _{i}ln\left(\frac{e^{ß\beta p_{min}\left(i\right)\lambda \left(i\right)}-e^{ß\beta (p_{max}\left(i\right)+1)\lambda \left(i\right)}}{1-e^{ß\beta \lambda \left(i\right)}}\right)\hfill \\ & & -\frac{1}{\beta }\sum _{j}ln\left(\frac{e^{ß\beta q_{min}\left(j\right)\mu \left(j\right)}-e^{ß\beta (q_{max}\left(j\right)+1)\mu \left(j\right)}}{1-e^{ß\beta \mu \left(j\right)}}\right)\hfill \\ & & -\frac{1}{\beta }\sum _{i,j}ln\left[1+e^{-\beta \left(C(i,j)+ß\lambda \left(i\right)+ß\mu \left(l\right)+ßx\right)}\right]-ßkx.\hfill \end{array}$$

(8)

In contrast to the expression of the internal energy U defined in Equation (2) that depends on the $N_1{N}_2+N_1+N_2$ constrained binary variables $G(i,j)$, $n_1\left(i\right)$, and $n_2\left(j\right)$, the expression for the effective free energy $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$ depends only on $N_1+N_2+1$ unconstrained variables, namely $\lambda \left(i\right)$, $\mu \left(j\right)$, and x. Below, we demonstrate how identifying the extremum of this function enables us to address the multi-assignment problem.

2.2. Optimizing the Effective Free Energy

Let $\overline{G}(i,j)$, $\overline{n_1}\left(i\right)$, and $\overline{n_2}\left(j\right)$ represent the expected values of $G(i,j)$, $n_1\left(i\right)$, and $n_2\left(j\right)$, respectively, in accordance with the Gibbs distribution specified in Equation (4). Computing these expected values directly is unfortunately not feasible as the partition function defined in Equation (8) lacks an analytical expression. Instead, we derive a saddle point approximation (SPA) by seeking extrema of the effective free energy with respect to the variables $\lambda$, $\mu$, and x:

$$\begin{array}{ccc}& & \frac{\partial F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)}{\partial {\lambda }_i}=\frac{\partial F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)}{\partial {\mu }_j}=\frac{\partial F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)}{\partial x}=0.\hfill \end{array}$$

(9)

After some rearrangements, these two equations can be written as

$$\begin{array}{cc}\hfill & \forall i,\sum _{j}X(i,j)=d_1\left(i\right),\hfill \end{array}$$

(10a)

$$\begin{array}{cc}\hfill & \forall j,\sum _{i}X(i,j)=d_2\left(j\right),\hfill \end{array}$$

(10b)

$$\begin{array}{cc}\hfill & \sum _{i,j}X(i,j)=k,\hfill \end{array}$$

(10c)

where

$$\begin{array}{ccc}\hfill X(i,j)& =& h\left[\beta \left(C(k,l)+ß\lambda \left(i\right)+ß\mu \left(j\right)+ßx\right),\right]\hfill \\ \hfill d_1\left(i\right)& =& g\left(ß\beta \lambda \left(i\right),p_{min}\left(i\right),p_{max}\left(i\right)\right),\hfill \\ \hfill d_2\left(j\right)& =& g\left(ß\beta \mu \left(j\right),q_{min}\left(j\right),q_{max}\left(j\right)\right),\hfill \end{array}$$

(11)

and

$$\begin{array}{ccc}\hfill h\left(x\right)=& & \frac{1}{e^x+1},\hfill \\ \hfill g(x,a,b)=& & \frac{(b-a+1)}{e^{(b-a+1)x}-1}+\frac{1}{1-e^x}+b.\hfill \end{array}$$

Note that

(i)

$g(x,a,a)=a$. If $a=1$ and $N_1=N_2=1$, we recover Equation (11) from [16] corresponding to the balanced assignment problem.

(ii)

$g(x,0,1)=h(-x)$. In this case, Equation (11) refers to the k-cardinality problem.

As frequently encountered, the saddle point may be purely imaginary. In this instance, it is evident from Equations (10) and (11) that the variables $ß\lambda \left(i\right)$, $ß\mu \left(j\right)$, and $ßx$ must be real. Consequently, we will replace $\left\{ß\lambda \right(i),ß\mu (j),ßx\}$ by $\left\{\lambda \right(i),\mu (j),x\}$ below.

In order to analyze the saddle point approximation, SPA, it is necessary to verify the existence and evaluate the uniqueness of the extrema of the effective free energy. The following theorem proves that $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$ is strictly concave.

Theorem 1.

The Hessian of the effective free energy $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$ is negative definite if $p_{min}\left(i\right)<p_{max}\left(i\right)$ for all i and $q_{min}\left(j\right)<q_{max}\left(j\right)$, for all j, and negative semidefinite otherwise. The free energy function $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$ is then strictly concave in the first case and concave in the second case.

Proof. 

See Appendix B.    □

The following property establishes a relationship between the solutions of the SPA system of equations and the expected values for the assignment matrix and for the indicator functions:

Proposition 1.

Let ${\overline{S}}_{\beta }$ be the expected state of the system at the temperature β with respect to the Gibbs distribution given in Equation (4). ${\overline{S}}_{\beta }$ is associated with an expected transportation plan ${\overline{G}}_{\beta }$ and expected indicator functions ${\overline{n_1}}_{\beta }$ and ${\overline{n_2}}_{\beta }$. Let ${\lambda }^{MF}\left(i\right)$, ${\mu }^{MF}\left(j\right)$, and $x^{MF}$ be the solutions of the system of Equation (10). Then the following identities hold:

$$\begin{array}{ccc}\hfill {\overline{G}}_{\beta }(i,j)& =& h\left(\beta (C(i,j)+{\lambda }^{MF}\left(i\right)+{\mu }^{MF}\left(j\right)+x^{MF})\right)\hfill \\ & =& X^{MF}(k,l)\hfill \\ \hfill {\overline{n}}_{1\beta }\left(i\right)& =& g(\beta {\lambda }^{MF}\left(i\right),p_{min}\left(i\right),p_{max}\left(i\right))=d_1^{MF}\left(i\right)\hfill \\ \hfill {\overline{n}}_{2\beta }\left(j\right)& =& g(\beta {\mu }^{MF}\left(j\right),q_{min}\left(j\right),q_{max}\left(j\right))=d_2^{MF}\left(j\right).\hfill \end{array}$$

(12)

To indicate that the solutions are mean field solutions, we use the superscript $MF$.

Proof. 

The proof of this proposition is virtually identical to the proof of the same results for the assignment problem, found in Appendix B of [15].    □

For a given value of the parameter $\beta$, ${\overline{n}}_{1\beta }\left(i\right)$ and ${\overline{n}}_{2\beta }\left(j\right)$ are the numbers of elements of $S_2$ and the number of elements in $S_1$ that are in correspondence with i and j, respectively, and ${\overline{G}}_{\beta }$ forms a transportation plan between $S_1$ and $S_2$ that is optimal with respect to the free energy defined in Equation (8). Note that these values are mean values and fractional. The matrix ${\overline{G}}_{\beta }$ belongs to a polytope, which we define as ${\mathcal{U}}^k(P_{min}^{max},Q_{min}^{max})$ (see Appendix A for a full characterization of this polytope). We can link the mean field assignment matrix ${\overline{G}}_{\beta }$ to an optimal free energy $F_{\beta }^{MF}$ and to an optimal internal energy $U_{\beta }^{MF}={\sum }_{i,j}{\overline{G}}_{\beta }(i,j)C(i,j)$. These values serve as mean field approximations of the exact free energy and internal energy of the system, respectively. Here are the key properties of $U_{\beta }^{MF}$ and $F_{\beta }^{MF}$:

Proposition 2.

$F_{\beta }^{MF}$ and $U_{\beta }^{MF}$ are monotonic increasing and monotonic decreasing functions, respectively, of the parameter β.

Proof. 

See Appendix C for $F_{\beta }^{MF}$ and Appendix D for $U_{\beta }^{MF}$.    □

Theorem 1 and Proposition 2 outlined above underscore several advantages of the proposed framework, which reframes the multi-assignment problem as a temperature-dependent process. Firstly, at each temperature, the multi-assignment problem with a generic cost matrix is transformed into a concave problem with a unique solution. This problem exhibits linear complexity in the number of variables, contrasting with the quadratic complexity of the original problem. The concavity facilitates the utilization of straightforward algorithms for identifying a minimum of the effective free energy function (Equation (8)). Additionally, Equation (12) offers robust numerical stability for computing the transportation plan and the functions $n_1$ and $n_2$, owing to the characteristics of the functions $h\left(x\right)$ and $g\left(x\right)$. Lastly, the convergence with respect to temperature is monotonic.

3. Solving Generic Assignment Problems

In the previous section, we proposed an effective free energy, $F_{\beta }\left(\mathit{\lambda },\mathit{\mu },x\right)$, that depends on $N_1+N_2+1$ unconstrained variables and on the inverse of the temperature, $\beta$, whose extremum identifies the solution of a possibly multi-job, multi-agent assignment problem. We have shown that the trajectory of this extremum is monotonic, increasing with respect to $\beta$. We now relate these values to the optimal assignment energy $U^{*}$:

Theorem 2.

Let $F_{\beta }^{MF}$ and $U_{\beta }^{MF}$ be the mean field approximations of the exact free energy and internal energy of the system at the inverse temperature β. The optimal assignment energy $U^{*}$ is then given by,

$$\begin{array}{ccc}\hfill U^{*}& =& \underset{\beta \to +\infty }{lim}{F}_{\beta }^{MF},\hfill \\ \hfill U^{*}& =& \underset{\beta \to +\infty }{lim}{U}_{\beta }^{MF},\hfill \end{array}$$

(13)

namely, the optimal assignment energy, $U^{*}$, is equal to the infinite limit with respect to the inverse temperature of both the mean field free energy and the internal energy.

Proof. 

See Appendix E.    □

This theorem illustrates that as the inverse temperature approaches infinity (or equivalently, as the temperature tends towards zero), the internal energy and free energy of the system converge to the optimal assignment energy. This confirms the validity of our statistical physics approach, particularly emphasizing the effectiveness of the saddle-point approximation. Note, however, that it does not define the behavior of the coupling matrix ${\overline{G}}_{\beta }=X^{MF}$. As ${\overline{G}}_{\beta }(i,j)=h\left(\beta (C(i,j)+{\lambda }^{MF}\left(i\right)+{\mu }^{MF}\left(j\right)+x^{MF})\right)$ and $0<h\left(x\right)<1$, the coupling matrix at any finite temperature is fractional. We need to show that as $\beta \to +\infty$, the corresponding matrix ${\overline{G}}_{\infty }$ does converge to the solution matrix $G^{*}$ and not to a fractional matrix that would lead to the same low energy $U^{*}$.

We first establish bounds on the internal energy and free energy at the SPA. Let us define

$$\begin{array}{ccc}\hfill A(N_1,N_2)=& & N_1{N}_{2}ln\left(2\right)+\sum _{i=1}^{N_1}ln(p_{max}\left(i\right)-p_{min}\left(i\right)+1)\hfill \\ & & +\sum _{j=1}^{N_2}ln(q_{max}\left(j\right)-q_{min}\left(j\right)+1).\hfill \end{array}$$

(14)

Theorem 3.

$F_{\beta }^{MF}$ and $U_{\beta }^{MF}$, the mean field approximations of the exact free energy and internal energy of the system at the inverse temperature β, satisfy the following inequalities:

$$\begin{array}{ccc}\hfill U^{*}-\frac{A(N_1,N_2)}{\beta }& \le F_{\beta }^{MF}\le & U^{*},\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill U^{*}& \le U_{\beta }^{MF}\le & U^{*}+\frac{A(N_1,N_2)}{\beta },\hfill \end{array}$$

(16)

where $U^{*}$ is the optimal assignment energy and $A(N_1,N_2)$ is defined in Equation (14).

Proof. 

See Appendix F.    □

Now, let us assume that the multi-assignment problem linked with the $N_1\times N_2$ cost matrix C possesses a unique optimal assignment matrix, denoted as $G^{*}$. We have the following theorem:

Theorem 4.

Let ${\overline{G}}_{\beta }$ be the coupling matrix solution of the SPA equations at the inverse temperature β. We assume that the multi-assignment problem has a unique solution, with $G^{*}$ being the associated coupling matrix. If Δ is the difference in total cost between the optimal solution and the second-best solution, then

$$\begin{array}{c}\hfill \underset{i,j}{max}\left|{\overline{G}}_{\beta }(i,j)-G^{*}(i,j)\right|\le \frac{A(N_1,N_2)}{\beta \Delta }.\end{array}$$

(17)

Proof. 

See Appendix G.    □

This theorem validates that, in the generic case in which the solution to the assignment problem is unique, the converged solution matrix ${\overline{G}}_{\infty }$ is this unique solution.

4. Solving Degenerate Assignment Problems

The method we propose is basically a relaxation approach to the general assignment problem. Indeed, we build a collection of transportation matrices ${\overline{G}}_{\beta }$ that belong to ${\mathcal{U}}^k(P_{min}^{max},Q_{min}^{max})\setminus {\mathcal{P}}^k(P_{min}^{max},Q_{min}^{max})$ (see Appendix A). Entries of these matrices are non-integer, strictly in the interval $(0,1)$. If the general assignment problem is known to have a unique integer solution, we have shown that these matrices converge to an element of ${\mathcal{P}}^k(P_{min}^{max},Q_{min}^{max})$ when $\beta \to +\infty$. The question remains as to what happens when the problem is degenerate, i.e., when it may have multiple integer solutions.

The general assignment problem is a linear programming problem. Checking if such a problem is degenerate is unfortunately often $NP$-complete ([22,23]). The degeneracies occur due to the presence of cycles in the linear constraints, i.e., in the cost matrix for the assignment problem. If this is the case, we propose randomly perturbing that matrix to bring it back to the generic problem. Megiddo and colleagues [24] have shown that an $\epsilon$-perturbation of a degenerate linear programming problem reduces this problem to a non-degenerate one.

5. Matching: A Program for Solving Assignment Problems

We have implemented the multi-assignment framework described here in a C++ program, Matching, that is succinctly described in Algorithm 1.

Matching relies on an iterative process in which the parameter $\beta$ is gradually increased. At each value of $\beta$, the nonlinear system of equations defined by Equation (10) is solved. We write this system as

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathbf{A}}_{\lambda }=0,\hfill \\ {\mathbf{A}}_{\mu }=0,\hfill \\ A_x=0,\hfill \end{array}\right.\end{array}$$

where $\mathbf{A}=({\mathbf{A}}_{\lambda },{\mathbf{A}}_{\mu },A_x)$ is a vector of predicates defined as

$$\begin{array}{ccc}\hfill A_{\lambda }\left(i\right)=& & \sum _{j}h\left(\beta (C(i,j)+\lambda \left(i\right)+\mu \left(j\right)+x)\right)\hfill \\ & & -g(\beta \lambda \left(i\right),p_{min}\left(i\right),p_{max}\left(i\right)),\hfill \\ \hfill A_{\mu }\left(j\right)=& & \sum _{i}h\left(\beta (C(i,j)+\lambda \left(i\right)+\mu \left(j\right)+x)\right)\hfill \\ & & -g(\beta \mu \left(j\right),q_{min}\left(j\right),q_{max}\left(j\right)),\hfill \\ \hfill A_x=& & \sum _{i,j}h\left(\beta (C(i,j)+\lambda \left(i\right)+\mu \left(j\right)+x)\right)-k.\hfill \end{array}$$

This system has $N_1+N_2+1$ equations, with the same number of variables. It is solved using an iterative Newton–Raphson method (for details, see, for example [16,25]). Once the SPA system of equations is solved, the assignment matrix ${\overline{G}}_{\beta }$, the functions ${\overline{n}}_{1\beta }$, ${\overline{n}}_{2\beta }$, and the corresponding transportation energy $U^{MF}\left(\beta \right)$ are computed. The iterations over $\beta$ are stopped if the mean field energy $U_{\beta }^{MF}$ does not change anymore (within a tolerance TOL usually set to ${10}^{-6}$). Otherwise, $\beta$ is increased, and the current values of $\mathit{\lambda }$, $\mathit{\mu }$ and x are used as input for the following iteration. At convergence, the values of the assignment matrix are rounded to the nearest integer (indicated as in the output of Algorithm 1). The minimal energy is then computed using the corresponding integer matrix.

The primary computational expense of this algorithm arises from solving the nonlinear set of equations corresponding to the SPA at every $\beta$ value. We use for this purpose an iterative Newton–Raphson method (see [15]).

Algorithm 1 Matching: a temperature-dependent framework for solving the multi-assignment problem

Input:$N_1$ and $N_2$, the number of agents and tasks; $p_{min}\left(i\right)$ and $p_{max}\left(i\right)$, the minimum and maximum number of agents needed to perform task i, and $q_{min}\left(j\right)$ and $q_{max}\left(j\right)$, the minimum and maximum number of tasks that can be performed by an agent j; k, the expected number of assignments; the cost matrix C. Initial value ${\beta }_0$ for $\beta$, the inverse of the temperature Initialize: Initialize arrays $\mathit{\lambda }$ and $\mathit{\mu }$ to 0 and initialize $x=0$. Set $STEP=\sqrt{10}$. for$i=1,\dots$ until convergence do     (1) Initialize ${\beta }^i=STEP\ast {\beta }^{i-1}$.     (2) Solve non linear Equation (10) for $\mathit{\lambda }$, $\mathit{\mu }$ and x at saddle point     (3) Compute corresponding ${\overline{G}}_{\beta }^i$, ${\overline{n}}_{1\beta }^i$, ${\overline{n}}_{2\beta }^i$, and $U_{\beta }^{i,MF}$     (4) Check for convergence: if $|U_{\beta }^{i,MF}-U_{\beta }^{i-1,MF}|<TOL$, stop end for Output: The converged assignment matrix $⌊{\overline{G}}_{\beta }⌉$, the functions $⌊{\overline{n}}_{1\beta }⌉$, $⌊{\overline{n}}_{2\beta }⌉$ over the agents and tasks, and the minimal associated cost $U_{\beta }$.

As for any annealing scheme, the initial temperature, or, in our case, the initial value ${\beta }_0$, is a parameter that significantly impacts the efficiency of the algorithm. Setting ${\beta }_0$ to be too small (i.e., a high initial temperature) will lead to inefficiency as the system will spend a significant amount of time at high temperatures, while setting ${\beta }_0$ too high will require many steps to converge at the corresponding low temperature, thereby decreasing the efficiency brought about by annealing. The value of $\beta$ scales the cost matrix C and as such is related to the range of this matrix, more specifically to its largest value, $C_{max}$. We found that setting ${\beta }_0{C}_{max}=1$ provides satisfactory annealing efficiency. The value for $STEP$ was chosen empirically.

6. Examples

The framework we propose is general: it allows us to solve balanced and unbalanced assignment problems, including those that allow for multi-agent or multi-task assignments. We illustrate our method on the latter types of problems, as, to our knowledge, there are currently no solutions to those that are guaranteed to reach the optimal assignment.

As a first illustration of our framework, we ran Matching on the cost matrix $C_1$ given in

Table 1. The cost matrix $C_1$.

	Tasks
Agents	${\mathit{T}}_{\mathbf{1}}$	${\mathit{T}}_{\mathbf{2}}$	${\mathit{T}}_{\mathbf{3}}$	${\mathit{T}}_{\mathbf{4}}$	${\mathit{T}}_{\mathbf{5}}$	${\mathit{T}}_{\mathbf{6}}$	${\mathit{T}}_{\mathbf{7}}$	${\mathit{T}}_{\mathbf{8}}$
$A_1$	300	250	180	320	270	190	220	260
$A_2$	290	310	190	180	210	200	300	190
$A_3$	280	290	300	190	190	220	230	260
$A_4$	290	300	190	240	250	190	180	210
$A_5$	210	200	180	170	160	140	160	180

. This matrix has been used in previous studies of multi-agent assignment problems [10,11,13,14,26,27]. Matching was run with the following parameters: $p_{min}=0$ for all agents (i.e., some agents may stay idle), $p_{max}=4$ for all agents (i.e., an agent can perform up to 4 tasks), $q_{min}=q_{max}=1$ (i.e., a task is only performed by one agent), and $k=8$ (all tasks are performed). In Figure 1, we show the corresponding trajectories of the internal energy $U_{\beta }^{MF}$ and free energy $F_{\beta }^{MF}$.

As expected, the internal energy is monotonically decreasing while the free energy is monotonically increasing, and both converge to the same value, 1440. Theorem 3 provides bounds on those energy values. Note that it can be rewritten as

$$\begin{array}{c}\hfill U_{\beta }^{MF}-\frac{A(N_1,N_2)}{\beta }\le U^{*}\le U_{\beta }^{MF},\\ \hfill F_{\beta }^{MF}\le U^{*}\le U_{\beta }^{MF}+\frac{A(N_1,N_2)}{\beta },\end{array}$$

i.e., at each value of $\beta$, we have bounds based on internal energy and free energy for the actual optimal cost of the assignment problem. These bounds are shown as shaded areas in Figure 1. Note that the widths of the intervals defined by the bounds are inversely proportional to $\beta$ and therefore decrease monotonically as $\beta$ increases.

In

Table 2. Assignments for $C_1$ with $p_{max}=4$.

	${\mathit{p}}_{\mathbf{min}}=0$	${\mathit{p}}_{\mathbf{min}}=1$	Wang a
	$A_1\to T_3$	$A_1\to T_3$	$A_1\to T_3$
	$A_2\to T_4,T_8$	$A_2\to T_8$	$A_2\to T_8$
	$A_3$	$A_3\to T_4$	$A_3\to T_4$
	$A_4\to T_7$	$A_4\to T_7$	$A_4\to T_7$
	$A_5\to T_1,T_2,T_5,T_6$	$A_5\to T_1,T_2,T_5,T_6$	$A_5\to T_1,T_2,T_5,T_6$
Total cost	1440	1450	1450

^a Assignment based on an ant colony algorithm [13].

, we compare the assignment found by Matching on $C_1$ with $p_{max}$ set to 4 (i.e., up to four tasks per agent), and $p_{min}=0$ (agents can be idle) or $p_{min}=1$ (each agent must execute at least one task). The removal of the latter constraint leads to a better optimal assignment in which agent 3 remains idle. When we maintain this constraint, our results are the same as those found by Wang et al. [13].

As a second more general test of our method, we compared the results of the framework we propose with the results of our own version of the Hungarian algorithm, as well as with those found in previous studies. We used both the matrix C1 and the matrix C2 given in

Table 3. The cost matrix $C_2$.

	Tasks
Agents	${\mathit{T}}_{\mathbf{1}}$	${\mathit{T}}_{\mathbf{2}}$	${\mathit{T}}_{\mathbf{3}}$	${\mathit{T}}_{\mathbf{4}}$	${\mathit{T}}_{\mathbf{5}}$	${\mathit{T}}_{\mathbf{6}}$	${\mathit{T}}_{\mathbf{7}}$	${\mathit{T}}_{\mathbf{8}}$	${\mathit{T}}_{\mathbf{9}}$	${\mathit{T}}_{\mathbf{10}}$
$A_1$	10	2	14	9	6	7	21	32	18	11
$A_2$	7	12	9	3	5	6	9	16	54	12
$A_3$	4	8	6	12	21	9	21	14	45	13
$A_4$	21	9	12	9	32	10	19	25	16	10
$A_5$	10	12	30	15	12	17	30	12	12	9
$A_6$	15	7	34	17	7	16	14	17	9	5

in those computing experiments.

We ran Matching as follows. For all tasks j, we set $q_{min}\left(j\right)=q_{max}\left(j\right)=1$, i.e., a task is only performed by one agent, and all tasks are expected to be executed. The latter is further enforced by setting k to be the total number of tasks (8 for $C_1$ and 10 for $C_2$). For an agent i, we either set all $p_{min}\left(i\right)$ to be 1 (in which case all agents must perform at least one task) or 0 (in which case some agents may not be assigned). We considered all $p_{max}\left(i\right)$ to be equal to a given value p and ran Matching with p varying from 1 to k. Note that both matrices $C_1$ and $C_2$ contain cycles. As such, optimal assignments based on those matrices are not unique. To remove the degeneracy, we added random uniform noise between $[0,ϵ]$ with $ϵ=0.001$ to all values of the cost matrices (see Section 4 for details).

The Hungarian algorithm remains a standard for solving balanced as well as unbalanced assignment problems. It needs to be adapted, however, when solving multi-task assignment problems. The corresponding textbook solution that was used, for example, in Ref. [14] is to make copies or clones of the agents, solve the augmented unbalanced assignment problem using the Hungarian algorithm, and then regroup all clones of an agent to define the tasks that it is assigned to. We consequently created multiple versions of the cost matrices $C_1$ and $C_2$, with all agents copied p times, where p defines the maximum number of tasks that an agent can perform. We ran our own version of the Hungarian algorithm on those matrices. This version is adapted from the serial code written by C. Ma and is publicly available at https://github.com/mcximing/hungarian-algorithm-cpp, (accessed on 1 May 2021).

Comparisons of the results of Matching, of the Hungarian algorithm, and of previous studies are provided in

Table 4. Solving the multi-task assignment problems defined by the cost matrix $C_1$.

Method $\setminus \mathit{p}$ a	2	3	4	5	6	7	8
Matching, $p_{min}=1$	1520	1470	1450	1450	1450	1450	1450
Matching, $p_{min}=0$	1520	1470	1440	1420	1410	1400	1400
Hungarian	1520	1470	1440	1420	1410	1400	1400
Majumdar GA b	1520	1470	1450	NA c	NA	NA	NA
Wang AC d	1520	1470	1450	NA	NA	NA	NA

^a Each agent is allowed to execute up to p tasks. Note that as there are 8 tasks and 5 agents only, the minimum value of p is 2 if all the tasks must be executed. ^b Genetic algorithm [26]. ^c Not available in either [26] or [13]. ^d Ant colony algorithm [13].

for the cost matrix C1 and in

Table 5. Solving the multi-task assignment problems defined by the cost matrix $C_2$.

Method $\setminus \mathit{p}$ a	2	3	4	5	6	7	8
Matching, $p_{min}=1$	66	65	65	65	65	65	65
Matching, $p_{min}=0$	66	62	61	61	61	61	61
Hungarian	66	62	61	61	61	61	61
Majumdar GA b	66	67	66	74	NA c	NA	NA
Wang AC d	66	65	65	NA	NA	NA	NA

^a Each agent is allowed to execute up to p tasks. Note that as there are 10 tasks and 6 agents only, and the minimum value of p is 2 if all the tasks must be executed. ^b Genetic algorithm [26]. ^c Not available in either [26] or [13]. ^d Ant colony algorithm [13].

for the cost matrix C2.

The results of Matching have been proven to be optimal (see Section 3). As expected, those results are strongly dependent on the constraints imposed on the problem: we can find assignments with a lower total cost if we allow agents to remain idle by setting $p_{min}$ to 0. For example, for the cost matrix $C_1$, in the extreme case with $p_{min}=0$ and $p_{max}=8$ for all agents, we find that all eight tasks have been assigned to agent 5, while the other agents have no task to perform, with a total cost of 1400. If instead we set $p_{min}=1$ and keep $p_{max}=8$, we find that agent 5 is assigned tasks 1, 2, 5, and 6; agent 1 is assigned task 3; agent 2 is assigned task 8; agent 3 is assigned task 4; and agent 4 is assigned task 7, for a total cost of 1450. If we instead run the Hungarian algorithm with agents that have been cloned to allow for multi-task assignments, we find optimal assignments that match those found by Matching with $p_{min}$ set to 0 for both cost matrices. For matrix $C_1$, for example, if each agent is represented with 8 clones, we find that agent 5 is assigned tasks 1, 2, 4, 5, 6, 7, and 8 while agent 1 is assigned task 3, for a total cost of 1400. This illustrates two points. First, as expected, the optimal assignment is not unique, as we find two different assignments with the same cost, 1400. Second, and more importantly, this shows the difficulty of applying the Hungarian algorithm for such problems. It works fine if we want to find the optimal assignment without consideration of constraints, such as the constraint each agent needs to perform at least one task, but does not fit if such a constraint is necessary.

The optimal solutions found by Matching when $p_{min}$ is set to 1 match those found by Wang et al. [13] for both matrices and are better than those found by Majumdar et al. [26] for matrix $C_2$. Note that the latter are obtained using either an ant colony algorithm or a genetic algorithm. Both are probabilistic algorithms that do not guarantee that the true minimum is found.

7. Conclusions

We have developed a general framework for solving balanced and unbalanced and constrained and unconstrained assignment problems. Given two sets of points $S_1$ and $S_2$ with the possibly different cardinalities $N_1$ and $N_2$, constraints on the number of possible assignments for each element of $S_1$ and of $S_2$, and a cost matrix defining the penalties of on assignment, we have constructed a concave free energy parameterized by temperature that captures those constraints. The definition of this free energy is general enough that it includes balanced and unbalanced cases. Its extremum establishes an optimal assignment between the two sets of points. We have demonstrated that this free energy consistently decreases as a function of $\beta$ (the inverse of temperature) towards the optimal assignment cost. Moreover, we have established that for sufficiently large $\beta$ values, the precise solution to the generic multi-assignment problem can be directly derived through straightforward rounding to the nearest integer of the elements of the assignment matrix linked to this extremum. Additionally, we have developed a method that guarantees convergence for addressing degenerate assignment problems.

The formalism introduced in this paper was designed to generalize the Hungarian algorithm for a large range of assignment problems. We expect it to be useful for a much larger set of problems, especially those associated with graphs. Graphs capture data relationships and, as such, are essential to numerous applications. They are particularly useful in domains such as Web search [28], neural [29] and social network analysis [30], gene networks [31], etc. More generally, they are designed to represent complex knowledge. The scale of modern graphs leads to a need to develop more efficient methods to process very large graphs. The formalism we propose is well adapted to tackle this problem for applications such as bipartite or simple graph matching. We will also consider extensions to higher-order matching problems, such as k-matching problems [32] (for example the three-assignment problem [33]) that are known to be $NP$-complete [34]. These problems have their own applications for graph analyses.