A Novel Method for Inference of Chemical Compounds of Cycle Index Two with Desired Properties Based on Artificial Neural Networks and Integer Programming

Abstract

Inference of chemical compounds with desired properties is important for drug design, chemo-informatics, and bioinformatics, to which various algorithmic and machine learning techniques have been applied. Recently, a novel method has been proposed for this inference problem using both artificial neural networks (ANN) and mixed integer linear programming (MILP). This method consists of the training phase and the inverse prediction phase. In the training phase, an ANN is trained so that the output of the ANN takes a value nearly equal to a given chemical property for each sample. In the inverse prediction phase, a chemical structure is inferred using MILP and enumeration so that the structure can have a desired output value for the trained ANN. However, the framework has been applied only to the case of acyclic and monocyclic chemical compounds so far. In this paper, we significantly extend the framework and present a new method for the inference problem for rank-2 chemical compounds (chemical graphs with cycle index 2). The results of computational experiments using such chemical properties as octanol/water partition coefficient, melting point, and boiling point suggest that the proposed method is much more useful than the previous method.

1. Introduction

Inference of chemical compounds with desired properties is important for computer-aided drug design. Since drug design is one of the major targets of chemo-informatics and bioinformatics, it is also important in these areas. Indeed, this problem has been extensively studied in chemo-informatics under the name of inverse QSAR/QSPR [1,2], where QSAR/QSPR denotes Quantitative Structure Activity/Property Relationships. Since chemical compounds are usually represented as undirected graphs, this problem is important also from graph theoretic and algorithmic viewpoints.

Inverse QSAR/QSPR is often formulated as an optimization problem to find a chemical graph maximizing (or minimizing) an objective function under various constraints, where objective functions reflect certain chemical activities or properties. In many cases, objective functions are derived from a set of training data consisting of known molecules and their activities/properties using statistical and machine learning methods.

In both forward and inverse QSAR/QSPR, chemical graphs are often represented as vectors of real or integer numbers because it is difficult to directly handle graphs using statistical and machine learning methods. Elements of these vectors are called descriptors in QSAR/QSPR studies, and these vectors correspond to feature vectors in machine learning. Using these chemical descriptors, various heuristic and statistical methods have been developed for finding optimal or nearly optimal graph structures under given objective functions [1,3,4]. In many cases, inference or enumeration of graph structures from a given feature vector is a crucial subtask in these methods. Various methods have been developed for this enumeration problem [5,6,7,8] and the computational complexity of the inference problem has been analyzed [9,10,11]. On the other hand, enumeration in itself is a challenging task, since the number of molecules (i.e., chemical graphs) with up to 30 atoms (vertices) C, N, O, and S, may exceed ${10}^{60}$ [12].

As in many other fields, Artificial Neural Network (ANN) and deep learning technologies have recently been applied to inverse QSAR/QSPR. For example, variational autoencoders [13], recurrent neural networks [14,15], and grammar variational autoencoders [16] have been applied. In these approaches, new chemical graphs are generated by solving a kind of inverse problems on neural networks, where neural networks are trained using known chemical compound/activity pairs. However, the optimality of the solution is not necessarily guaranteed in these approaches. In order to guarantee the optimality, a novel approach has been proposed [17] for ANNs with ReLU activation functions and sigmoid activation functions, using mixed integer linear programming (MILP). In their approach, activation functions on neurons are efficiently encoded as piece-wise linear functions so as to represent ReLU functions exactly and sigmoid functions approximately.

Recently, a new framework has been proposed [18,19,20] by combining two previous approaches; efficient enumeration of tree-like graphs [5], and MILP-based formulation of the inverse problem on ANNs [17]. This combined framework for inverse QSAR/QSPR mainly consists of two phases, one for constructing a prediction function to a chemical property, and the other for constructing graphs based on the inverse of the prediction function. The first phase solves (I) Prediction Problem, where a prediction function ${\psi }_{\mathcal{N}}$ on a chemical property $\pi$ is constructed with an ANN $\mathcal{N}$ using a data set of chemical compounds G and their values $a\left(G\right)$ of $\pi$. The second phase solves (II) Inverse Problem, where (II-a) given a target value $y^{*}$ of the chemical property $\pi$, a feature vector $x^{*}$ is inferred from the trained ANN $\mathcal{N}$ so that ${\psi }_{\mathcal{N}}\left(x^{*}\right)$ is close to $y^{*}$ and (II-b) then a set of chemical structures $G^{*}$ such that $f\left(G^{*}\right)=x^{*}$ is enumerated. In (II-b) of the above-mentioned previous methods [18,19,20], an MILP is formulated for acyclic chemical compounds. Their methods were applicable only to acyclic chemical graphs (i.e., tree-structured chemical graphs), where the ratio of acyclic chemical graphs in a major chemical database (PubChem) is 2.91%. Afterward, Ito et al. [21] designed a method of inferring monocyclic chemical graphs (chemical graphs with cycle index or rank 1) by formulating a new MILP and using an efficient algorithm for enumerating monocyclic chemical graphs [22]. This still leaves a big limitation because the ratio of acyclic and monocyclic chemical graphs in the chemical database PubChem is only 16.26%.

To break this limitation, we significantly extend the MILP-based approach for inverse QSAR/QSPR so that “rank-2 chemical compounds” (chemical graphs with cycle index or rank 2) can be efficiently handled, where the ratio of chemical graphs with rank at most 2 in the database PubChem is 44.5%. Note that there are three different topological structures, called polymer-topologies over all rank-2 chemical compounds. In particular, we propose a novel MILP formulation for (II-a) along with a new set of descriptors. One big advantage of this new formulation is that an MILP instance has a solution if and only if there exists a rank-2 chemical graph satisfying given constraints, which is useful to significantly reduce redundant search in (II-b). We conducted computational experiments to infer rank-2 chemical compounds on several chemical properties.

The paper is organized as follows. Section 2.1 introduces some notions on graphs, a modeling of chemical compounds, and a choice of descriptors. Section 2.2 reviews the framework for inferring chemical compounds based on ANNs and MILPs. Section 2.3 introduces a method of modeling rank-2 chemical graphs with different cyclic structures in a unified way and proposes an MILP formulation that represents a rank-2 chemical graph G of n vertices, where our MILP requires only $O\left(n\right)$ variables and constraints when the maximum height of subtrees in G is constant. Section 3 reports the results on some computational experiments conducted for chemical properties such as octanol/water partition coefficient, melting point, and boiling point. Section 4 makes some concluding remarks. Appendix A provides the detail of all variables and constraints in our MILP formulation.

2. Materials and Methods

2.1. Preliminary

This section introduces some notions and terminology on graphs, a modeling of chemical compounds, and our choice of descriptors.

2.1.1. Multigraphs and Graphs

Let $\mathbb{R}$ and $\mathbb{Z}$ denote the sets of reals and non-negative integers, respectively. For two integers a and b, let $[a,b]$ denote the set of integers i with $a\le i\le b$.

Multigraphs

A multigraph is defined to be a pair $(V,E)$ of a vertex set V and an edge set E such that each edge $e\in E$ joins two vertices $u,v\in V$ (possibly $u=v$) and the vertices u and v are called the end-vertices of the edge e, and let $V\left(e\right)$ denote the set of the end-vertices of an edge $e\in E$, where an edge e with $\left|V\right(e\left)\right|=1$ is called a loop. We denote the vertex and edge sets of a multigraph M by $V\left(M\right)$ and $E\left(M\right)$, respectively. A path with end-vertices u and v is called a $u,v$-path, and the length of a path is defined to be the number of edges in the path.

Let M be a multigraph. An edge $e\in E\left(M\right)$ is called multiple (to an edge $e^{\prime }\in E\left(M\right)$) if there is another edge $e^{\prime }\in E\left(M\right)$ with $V\left(e\right)=V\left(e^{\prime }\right)$. For a vertex $v\in V\left(M\right)$, the set of neighbors of v in M is denoted by $N_M\left(v\right)$, and the degree ${\mathrm{deg}}_M\left(v\right)$ of v is defined to be the number of times an edge in $E\left(M\right)$ is incident to v; i.e., ${\mathrm{deg}}_M\left(v\right)=\left|\right\{e\in E\left(M\right)\mid v\in V\left(e\right),\left|V\left(e\right)\right|=2\left\}\right|+2\left|\right\{e\in E\left(M\right)\mid v\in V\left(e\right),\left|V\left(e\right)\right|=1\left\}\right|$. A multigraph is called simple if it has no loop and there is at most one edge between any two vertices. We observe that the sum of the degrees over all vertices is twice the number of edges in any multigraph M; i.e.,

$$2\left|E\left(M\right)\right|=\sum _{v\in V\left(M\right)}{\mathrm{deg}}_M\left(v\right).$$

For a subset X of vertices in M, let $M-X$ denote the multigraph obtained from M by removing the vertices in X and any edge incident to a vertex in X. An operation of subdividing a non-loop edge (resp., loop) $e\in E\left(M\right)$ with $V\left(e\right)=\{v_1,v_2\}$ (resp., $V\left(e\right)=\{v_1=v_2\}$) is to replace e with two new edges $e_1$ and $e_2$ incident to a new vertex $v_e$ such that each $e_i$ is incident to $v_i$. An operation of contracting a vertex u of degree 2 in M is to replace the two edges $uv$ and $u{v}^{\prime }$ incident to u with a single edge $v{v}^{\prime }$ removing vertex u, where the resulting edge is a loop when $v=v^{\prime }$. The rank $\mathrm{r}\left(M\right)$ of a multigraph M is defined to be the minimum number of edges to be removed to make the multigraph acyclic. We call a multigraph M with $\mathrm{r}\left(M\right)=k$ a rank-k graph. Let $V_{\mathrm{deg},i}\left(M\right)$ denote the set of vertices of degree i in M. The core $\mathrm{Cr}\left(M\right)$ of M is defined to be an induced subgraph $M^{*}$ that is obtained from $M^{\prime }:=M$ by setting $M^{\prime }:=M^{\prime }-V_{\mathrm{deg}},1\left(M^{\prime }\right)$ repeatedly until $M^{*}$ contains at most two vertices or consists of vertices of degree at least 2. The core $M^{*}$ of a connected multigraph M consists of a single vertex (resp., two vertices) if and only if M is a tree with an even (resp., odd) diameter. A vertex (resp., an edge) in M is called a core vertex (resp., core edge) if it is contained in the core of M and is called a non-core vertex (resp., non-core edge) otherwise. The core size $\mathrm{cs}\left(M\right)$ is defined to be the number of core vertices of M, and the core height $\mathrm{ch}\left(M\right)$ is defined to be the maximum length of a path between a vertex $v\in V\left(M^{*}\right)$ to a leaf of M without passing through any core edge. The set of non-core edges induces a collection of subtrees, each of which we call a non-core component of M, where each non-core component C contains exactly one core vertex $v_C$ and we regard C as a tree rooted at $v_C$. Let C be a non-core component of M. The height $\mathrm{height}\left(v\right)$ of a vertex v in C is defined to be the maximum length of a path from v to a leaf u in the descendants of v.

A multigraph is called a polymer topology if it is connected and the degree of every vertex is at least 3. Tezuka and Oike [23] pointed out that a classification of polymer topologies will lay a foundation for elucidation of structural relationships between different macro-chemical molecules and their synthetic pathways. For integers $r\ge 0$ and $d\ge 3$, let $\mathcal{PT}(r,d)$ denote the set of all rank-r polymer topologies with maximum degree at most d. Figure 1 illustrates the three rank-2 polymer topologies in $\mathcal{PT}(2,4)$.

For a polymer topology M, the least simple graph $S\left(M\right)$ of M is defined to be a simple graph obtained from M by subdividing each loop in M with two new vertices of degree 2 and subdividing all multiple edges (except for one) between every two adjacent vertices in M. Note that $\left|V\right(S\left(M\right)\left)\right|=\left|V\right(M\left)\right|+r+s$ for the rank r of M and the number s of loops in M.

The polymer topology $\mathrm{Pt}\left(M\right)$ of a multigraph M with $\mathrm{r}\left(M\right)\ge 2$ is defined to be a multigraph $M^{\prime }$ of degree at least 3 that is obtained from the core $\mathrm{Cr}\left(M\right)$ by contracting all vertices of degree 2. Note that $\mathrm{r}\left(\mathrm{Pt}\right(M\left)\right)=\mathrm{r}\left(M\right)$. Figure 2a–c illustrate the least simple graph $S\left(M\right)$ of each polymer topology $M\in \mathcal{PT}(2,4)$, where Figure 2d illustrates a graph that contains all least simple graphs.

Graphs

Let $H=(V,E)$ be a graph with a set V of vertices and a set E of edges. Define the 1-path connectivity ${\kappa }_1\left(H\right)$ of H to be ${\sum }_{uv\in E}1/\sqrt{{\mathrm{deg}}_H\left(u\right){\mathrm{deg}}_H\left(v\right)}$.

Let H be a rank-2 connected graph such that the maximum degree is at most 4. We see that H contains two vertices $v_a$ and $v_b$ such that either there are three disjoint paths between $v_a$ and $v_b$ or H contains two edge disjoint cycles C and $C^{\prime }$, which are joined with a path between $v_a$ and $v_b$ (possibly $v_a=v_b$). We introduce the topological parameter $\theta \left(H\right)$ of rank-2 connected graph H as follows. When H has three disjoint paths between $v_a$ and $v_b$, define $\theta \left(H\right)$ to be the minimum number of edges along a path between $v_a$ and $v_b$. When H contains two edge disjoint cycles C and $C^{\prime }$, which are joined with a path P between $v_a$ and $v_b$ (possibly $v_a=v_b$), define $\theta \left(H\right)$ to be $-\left|E\right(P\left)\right|$.

For positive integers $a,b$ and c with $b\ge 2$, let $T(a,b,c)$ denote the rooted tree such that the number of children of the root is a, the number of children of each non-root internal vertex is b and the distance from the root to each leaf is c. In the rooted tree $T(a,b,c)$, we denote the vertices by $v_1,v_2,\dots ,v_n$ ($n=a(b^c-1)/(b-1)+1$) with a breadth-first-search order, and denote the edge between a vertex $v_i$ with $i\in [2,n]$ and its parent by $e_i$. For each vertex $v_i$ in $T(a,b,c)$, let $\mathrm{Cld}\left(i\right)$ denote the set of indices j such that $v_j$ is a child of $v_i$, and $\mathrm{prt}\left(i\right)$ denote the index j such that $v_j$ is the parent of $v_i$ when $i\in [2,n]$.

2.1.2. Modeling of Chemical Compounds

Chemical Graphs

We represent the graph structure of a chemical compound as a graph with labels on vertices and multiplicity on edges in a hydrogen-suppressed model. Nearly 68.5% (resp., 99%) of the rank-2 chemical graphs with at most 200 non-hydrogen atoms registered in chemical database PubChem have a maximum degree at most 3 (resp., 4) for all non-core vertices in the hydrogen-suppressed model.

Let $\Lambda$ be a set of labels, each of which represents a chemical element such as C (carbon), O (oxygen), N (nitrogen) and so on, where we assume that $\Lambda$ does not contain H (hydrogen). Let $\mathrm{mass}\left(\mathtt{a}\right)$ and $\mathrm{val}\left(\mathtt{a}\right)$ denote the mass and valence of a chemical element $\mathtt{a}\in \Lambda$, respectively. In our model, we use integers ${\mathrm{mass}}^{*}\left(\mathtt{a}\right)=\lfloor 10·\mathrm{mass}\left(\mathtt{a}\right)\rfloor$, $\mathtt{a}\in \Lambda$ and assume that each chemical element $\mathtt{a}\in \Lambda$ has a unique valence $\mathrm{val}\left(\mathtt{a}\right)\in [1,4]$.

We introduce a total order < over the elements in $\Lambda$ according to their mass values; i.e., we write $\mathtt{a}<\mathtt{b}$ for chemical elements $\mathtt{a},\mathtt{b}\in \Lambda$ with $\mathrm{mass}\left(\mathtt{a}\right)<\mathrm{mass}\left(\mathtt{b}\right)$. Choose a set ${\Gamma }_{<}$ of tuples $\gamma =(\mathtt{a},\mathtt{b},k)\in \Lambda \times \Lambda \times [1,3]$ such that $\mathtt{a}<\mathtt{b}$. For a tuple $\gamma =(\mathtt{a},\mathtt{b},k)\in \Lambda \times \Lambda \times [1,3]$, let $\overline{\gamma }$ denote the tuple $(\mathtt{b},\mathtt{a},k)$. Set ${\Gamma }_{>}=\{\overline{\gamma }\mid \gamma \in {\Gamma }_{<}\}$, ${\Gamma }_{=}=\{(\mathtt{a},\mathtt{a},k)\mid \mathtt{a}\in \Lambda ,k\in [1,3]\}$ and $\Gamma ={\Gamma }_{<}\cup {\Gamma }_{=}$. a pair of two atoms $\mathtt{a}$ and $\mathtt{b}$ joined with a bond of multiplicity k is denoted by a tuple $\gamma =(\mathtt{a},\mathtt{b},k)\in \Gamma$, called  the adjacency-configuration of the atom pair.

We use a hydrogen-suppressed model because hydrogen atoms can be added at the final stage. a chemical graph over $\Lambda$ and $\Gamma$ is defined to be a tuple $G=(H,\alpha ,\beta )$ of a graph $H=(V,E)$, a function $\alpha :V\to \Lambda$ and a function $\beta :E\to [1,3]$ such that

(i)

H is connected;

(ii)

${\sum }_{uv\in E}\beta \left(uv\right)\le \mathrm{val}\left(\alpha \left(u\right)\right)$ for each vertex $u\in V$; and

(iii)

$\left(\alpha \right(u),\alpha (v),\beta (uv\left)\right)\in \Gamma$ for each edge $uv\in E$.

Let $\mathcal{G}(\Lambda ,\Gamma )$ denote the set of chemical graphs over $\Lambda$ and $\Gamma$.

Descriptors

In our method, we use only graph-theoretical descriptors for defining a feature vector, which facilitates our designing an algorithm for constructing graphs. Given a chemical graph $G=(H,\alpha ,\beta )$, we define a feature vector $f\left(G\right)$ that consists of the following 14 kinds of descriptors:

-

$n\left(G\right)$: the number of vertices in G;

-

$\mathrm{cs}\left(G\right)$: the core size of G;

-

$\mathrm{ch}\left(G\right)$: the core height of G;

-

${\kappa }_1\left(G\right)$: the 1-path connectivity of G;

-

${\mathrm{dg}}_i\left(G\right)$ ($i\in [1,4]$): the number of vertices of degree i in G;

-

${\mathrm{ce}}_{\mathtt{a}}^{\mathrm{co}}\left(G\right)$ ($\mathtt{a}\in \Lambda )$: the number of core vertices with chemical element $\mathtt{a}\in \Lambda$;

-

${\mathrm{ce}}_{\mathtt{a}}^{\mathrm{nc}}\left(G\right)$ ($\mathtt{a}\in \Lambda )$: the number of non-core vertices with chemical element $\mathtt{a}\in \Lambda$;

-

$\overline{\mathrm{ms}}\left(G\right)$: the average of ${\mathrm{mass}}^{*}$ of atoms in G;

-

$b_k^{\mathrm{co}}\left(G\right)$ ($k\in [2,3]$): the number of double and triple bonds in core edges;

-

$b_k^{\mathrm{nc}}\left(G\right)$ ($k\in [2,3]$): the number of double and triple bonds in non-core edges;

-

${\mathrm{ac}}_{\gamma }^{\mathrm{co}}\left(G\right)$ ($\gamma =(\mathtt{a},\mathtt{b},k)\in \Gamma$): the number of adjacency-configurations $(\mathtt{a},\mathtt{b},k)$ of core edges;

-

${\mathrm{ac}}_{\gamma }^{\mathrm{nc}}\left(G\right)$ ($\gamma =(\mathtt{a},\mathtt{b},k)\in \Gamma$): the number of adjacency-configurations $(\mathtt{a},\mathtt{b},k)$ of non-core edges;

-

$\theta \left(H\right)$: the topological parameter of H; and

-

$n_{\mathtt{H}}\left(G\right)$: the number of hydrogen atoms to be included in G; i.e.,

$n_{\mathtt{H}}\left(G\right)={\sum }_{\mathtt{a}\in \Lambda }\mathrm{val}\left(\mathtt{a}\right)({\mathrm{ce}}_{\mathtt{a}}^{\mathrm{co}}\left(G\right)+{\mathrm{ce}}_{\mathtt{a}}^{\mathrm{nc}}\left(G\right))-2(n\left(G\right)+1+b_2^{\mathrm{co}}\left(G\right)+b_2^{\mathrm{nc}}\left(G\right)+2b_3^{\mathrm{co}}\left(G\right)+2b_3^{\mathrm{nc}}\left(G\right))$.

The number k of descriptors in our feature vector $x=f\left(G\right)$ is $k=2|\Lambda |+2|\Gamma |+15$.

2.2. A Method for Inferring Chemical Graphs

This section reviews the framework that solves the inverse QSAR/QSPR by using MILPs [18]. For a specified chemical property $\pi$ such as boiling point, we denote by $a\left(G\right)$ the observed value of the property $\pi$ for a chemical compound G. As the Phase 1, we solve (I) Prediction Problem with the following three steps.

Phase 1.

Step 1: Let $\mathrm{DB}$ be a set of chemical graphs. For a specified chemical property $\pi$, choose a class $\mathcal{G}$ of graphs such as acyclic graphs or monocyclic graphs. Prepare a data set $D_{\pi }=\{G_i\mid i=1,2,\dots ,m\}\subseteq \mathcal{G}\cap \mathrm{DB}$ such that the value $a\left(G_i\right)$ of each chemical graph $G_i$, $i=1,2,\dots ,m$ is available. Set reals $\underline{a},\overline{a}\in \mathbb{R}$ so that $\underline{a}\le a\left(G_i\right)\le \overline{a}$, $i=1,2,\dots ,m$. See Figure 3 for an illustration of Step 1.

Step 2: Introduce a feature function $f:\mathcal{G}\to {\mathbb{R}}^k$ for a positive integer k. We call $f\left(G\right)$ the feature vector of $G\in \mathcal{G}$, and call each entry of a vector $f\left(G\right)$ a descriptor of G. See Figure 4 for an illustration of Step 2.

Step 3: Construct a prediction function ${\psi }_{\mathcal{N}}$ with an ANN $\mathcal{N}$ that, given a vector in ${\mathbb{R}}^k$, returns a real in the range $[\underline{a},\overline{a}]$ so that ${\psi }_{\mathcal{N}}\left(f\left(G\right)\right)$ takes a value nearly equal to $a\left(G\right)$ for many chemical graphs in D. See Figure 5 for an illustration of Step 3.

Next we explain how to solve the inverse problem to the prediction in Phase 1 using an MILP formulation. A vector $x\in {\mathbb{R}}^k$ is called admissible if there is a graph $G\in \mathcal{G}$ such that $f\left(G\right)=x$ [18]. Let $\mathcal{A}$ denote the set of admissible vectors $x\in {\mathbb{R}}^k$. In this paper, we use the range-based method to define an applicability domain (AD) [24] to our inverse QSAR/QSPR. Set $\underline{x_j}$ and $\overline{x_j}$ to be the minimum and maximum values of the j-th descriptor $x_j$ in $f\left(G_i\right)$ over all graphs $G_i$, $i=1,2,\dots ,m$ (where we possibly normalize some descriptors such as ${\mathrm{ce}}_{\mathtt{a}}^{\mathrm{co}}\left(G\right)$, which is normalized with ${\mathrm{ce}}_{\mathtt{a}}^{\mathrm{co}}\left(G\right)/n\left(H\right)$). Define our AD $\mathcal{D}$ to be the set of vectors $x\in {\mathbb{R}}^k$ such that $\underline{x_j}\le x_j\le \overline{x_j}$ for the variable $x_j$ of each j-th descriptor, $j=1,2,\dots ,k$. As the second phase, we solve (II) Inverse Problem for the inverse QSAR/QSPR by treating the following inference problems.

(II-a) Inference of Vectors

Input: A real $y^{*}\in [\underline{a},\overline{a}]$.

Output: Vectors $x^{*}\in \mathcal{A}\cap \mathcal{D}$ and $g^{*}\in {\mathbb{R}}^h$ such that ${\psi }_{\mathcal{N}}\left(x^{*}\right)=y^{*}$ and $g^{*}$ forms a chemical graph $G^{*}\in \mathcal{G}$ with $f\left(G^{*}\right)=x^{*}$.

(II-b) Inference of Graphs

Input: A vector $x^{*}\in \mathcal{A}\cap \mathcal{D}$.

Output: All graphs $G^{*}\in \mathcal{G}$ such that $f\left(G^{*}\right)=x^{*}$.

To treat Problem (II-a), we use MILPs for inferring vectors in ANNs [17]. In MILPs, we can easily impose additional linear constraints or fix some variables to specified constants. We include into the MILP a linear constraint such that $x\in \mathcal{D}$ to obtain the next result.

Theorem 1.

Let $\mathcal{N}$ be an ANN with a piecewise-linear activation function for an input vector $x\in {\mathbb{R}}^k$, $n_A$ denote the number of nodes in the architecture and $n_B$ denote the total number of break-points over all activation functions. Then there is an MILP $\mathcal{M}(x,y;{\mathcal{C}}_1)$ that consists of variable vectors $x\in \mathcal{D}(\subseteq {\mathbb{R}}^k)$, $y\in \mathbb{R}$, and an auxiliary variable vector $z\in {\mathbb{R}}^p$ for some integer $p=O(n_A+n_B)$ and a set ${\mathcal{C}}_1$ of $O(n_A+n_B)$ constraints on these variables such that: ${\psi }_{\mathcal{N}}\left(x^{*}\right)=y^{*}$ if and only if there is a vector $(x^{*},y^{*})$ feasible to $\mathcal{M}(x,y;{\mathcal{C}}_1)$.

See Appendix A for the set of constraints to define our AD $\mathcal{D}$ in the MILP $\mathcal{M}(x,y;{\mathcal{C}}_1)$ in Theorem 1.

To attain the admissibility of inferred vector $x^{*}$, we also introduce a variable vector $g\in {\mathbb{R}}^q$ for some integer q and a set ${\mathcal{C}}_2$ of constraints on x and g such that $x^{*}\in \mathcal{A}$ holds in the following sense: $(x^{*},g^{*})$ is feasible to the MILP $\mathcal{M}(x,g;{\mathcal{C}}_2)$ if and only if $g^{*}$ forms a chemical graph $G^{*}\in \mathcal{G}$ with $f\left(G^{*}\right)=x^{*}$. The Phase 2 consists of the next two steps.

Phase 2.

Step 4: Formulate Problem (II-a) as the above MILP $\mathcal{M}(x,y,g;{\mathcal{C}}_1,{\mathcal{C}}_2)$ based on $\mathcal{G}$ and $\mathcal{N}$. Find a set $F^{*}$ of vectors $x^{*}\in \mathcal{A}\cap \mathcal{D}$ such that $(1-\epsilon )y^{*}\le {\psi }_{\mathcal{N}}\left(x^{*}\right)\le (1+\epsilon )y^{*}$ for a tolerance $\epsilon$ set to be a small positive real. See Figure 6 for an illustration of Step 4.

Step 5: To solve Problem (II-b), enumerate all graphs $G^{*}\in \mathcal{G}$ such that $f\left(G^{*}\right)=x^{*}$ for each vector $x^{*}\in F^{*}$. See Figure 7 for an illustration of Step 5.

In this paper, we set a graph class $\mathcal{G}$ to be the set of rank-2 graphs. In Step 4, we solve an MILP $\mathcal{M}(x,g;{\mathcal{C}}_2)$ that is formulated on a novel idea of representing rank-2 chemical graphs, as will be discussed in Section 2.3.2. In Step 5, we use branch-and-bound algorithms for enumerating rank-2 chemical compounds [25,26].

2.3. Representing Rank-2 Chemical Graphs

This section introduces a method of modeling rank-2 chemical graphs with different cyclic structures in a unified way and proposes an MILP formulation that represents a rank-2 chemical graph G of n vertices.

2.3.1. Scheme Graphs and Tree-Extensions

Given positive integers $n^{*}$ and p, a graph with $n^{*}$ vertices and p edges can be represented as a subgraph of a complete graph $K_{n^{*}}$ with $n^{*}(n^{*}-1)/2$ edges. However, formulating this as an MILP may require to prepare $\Omega \left({\left(n^{*}\right)}^2\right)$ variables and constraints. To reduce the number of variables and constraints in an MILP that represents a rank-2 graph, we decompose a rank-2 graph G into the core and non-core of G so that the core is represented by one of the three rank-2 polymer topologies and the non-core is a collection of trees in which the height is bounded by the core height of G. We do not specify how many subtrees will be attached to each edge in the polymer topology in advance, since otherwise we would need a different MILP for a distinct combination of such assignments of subtrees. Instead we allow each edge in a polymer topology to collect a necessary number of subtrees in our MILP (see the next section for more detail). In this section, we introduce a “scheme graph” to represent three possible rank-2 polymer topologies, an “extension” of the scheme graph to represent the core of a rank-2 graph and a “tree-extension” to represent a combination of the core and non-core of a rank-2 graph, so that any of the three kinds of rank-2 polymer topologies can be selected in a single MILp formulation.

Scheme Graphs

Formally, we define the scheme graph for rank 2 to be a pair $(K,\mathcal{E})$ of a multigraph K and an ordered partition $\mathcal{E}=(E_1,E_2,E_3)$ of the edge set $E\left(K\right)$. Figure 2d illustrates the scheme graph $(K=(\{u_1,u_2,u_3,u_4\},E),\mathcal{E}=(E_1,E_2,E_3))$. An edge in $E_1$ is called a semi-edge, an edge in $E_2$ is called a virtual edge and an edge in $E_3$ is called a real edge.

Extensions of Scheme Graphs

Based on the scheme graph $(K,\mathcal{E})$, we construct the core of a rank-2 graph H as an “extension,” which is defined as follows (see also Figure 8). The extension $H_{\mathrm{core}}$ in Figure 9a An extension of the scheme graph $(K,\mathcal{E})$ is defined to be a simple graph obtained from K by using each real edge $e=uv\in E_3$, by eliminating or replacing each virtual edge $e=uv\in E_2$ (resp., semi-edge $e=uv\in E_1$) with a $u,v$-path of length at least two (resp., 1) in the core of H, where a $u,v$-path of length 1 means an edge $uv$. Figure 9a illustrates an extension $H_{\mathrm{core}}$ of the scheme graph $(K,\mathcal{E})$ which is obtained by removing virtual edges $a_4,a_5\in E_2$ and by replacing semi-edge $a_1\in E_1$ with a path $(u_{1,1},v_{1,1},v_{2,1},u_{4,1})$, semi-edge $a_2\in E_1$ with a path $(u_{2,1},v_{3,1},v_{4,1},v_{5,1},u_{3,1}$) and by using semi-edge $a_3\in E_1$ and real edges $a_6,a_7\in E_3$. The extension $H_{\mathrm{core}}$ in Figure 9a is isomorphic to the core of the rank-2 graph H in Figure 9b. Observe that each of the least simple graphs $S\left(M_i\right)$, $i=1,2,3$ in Figure 2 is obtained as an extension of the scheme graph $(K,\mathcal{E})$ in Figure 2d.

Tree-Extensions

Let $s^{*}=\left|V\left(K\right)\right|=4$ denote the number of vertices in the scheme graph. For non-negative integers a, b and c, we consider a rank-2 graph H such that $\mathrm{cs}\left(H\right)=s^{*}+a=4+a$, $\mathrm{ch}\left(H\right)=b$ and the maximum degree of a core vertex is at most c. We define an “$(a,b,c)$-tree-extension” as a minimal supergraph of all such rank-2 graphs H. Formally, the $(a,b,c)$-tree-extension (or a tree-extension) is defined to be the graph obtained by augmenting the graph K as follows:

(i)

For each vertex $u_s\in V\left(K\right)$, $s\in [1,s^{*}]$, create a copy $S_s$ of the rooted tree $T(c-2,c-1,b)$. For each $s\in [1,s^{*}]$, let the root of rooted tree $S_s$ be equal to the vertex $u_s$ and denote by $u_{s,i}$ the copy of the i-th vertex of $T(c-2,c-1,b)$ in $S_s$ (see Figure 8a).

(ii)

Create a new path $(v_{1,1},v_{2,1},\dots ,v_{a,1})$ with a vertices, where the edge between $v_{t,1}$ and $v_{t+1,1}$ is denoted by $e_{t+1}$ (see Figure 8c). For each $t\in [1,a]$, create a copy $T_t$ of the rooted tree $T(c-2,c-1,b)$, let the root of rooted tree $T_t$ be equal to the vertex $v_{1,1}$ and denote by $v_{t,i}$ the copy of the i-th vertex of $T(c-2,c-1,b)$ in $T_t$ (see Figure 8b).

(iii)

For every pair $(s,t)$ with $s\in [1,s^{*}]$ and $t\in [1,a]$, join vertices $u_{s,1}$ and $v_{t,1}$ with an edge $u_{s,1}{v}_{t,1}$ (see Figure 8c).

Figure 8 illustrates the $(3,2,4)$-tree-extension of the scheme graph. We show how a rank-2 graph can be constructed as a subgraph of a tree-extension with some example. Figure 9b illustrates a rank-2 graph H with $n\left(H\right)=21$, $\mathrm{cs}\left(H\right)=9$, $\mathrm{ch}\left(H\right)=2$ and $\theta \left(H\right)=1$, where the maximum degree of a non-core vertex is 3. To prepare a tree-extension so that the graph H can be a subgraph of the tree-extension, we set ${\mathrm{cs}}^{*}:=\mathrm{cs}\left(H\right)$, $a:=t^{*}:={\mathrm{cs}}^{*}-s^{*}=5$, $b:={\mathrm{ch}}^{*}:=\mathrm{ch}\left(H\right)=2$ and $c:=d_{max}:=3$. Figure 9c illustrates a subgraph $H^{\prime }$ of the $(t^{*}=5,{\mathrm{ch}}^{*}=2,d_{max}=3)$-tree-extension such that $H^{\prime }$ is isomorphic to the rank-2 graph H.

2.3.2. MILPs for Rank-2 Chemical Graphs

We present an outline of our MILP $\mathcal{M}(x,g;{\mathcal{C}}_2)$ in Step 4 of the framework. For integers $d_{max},n^{*},{\mathrm{cs}}^{*}$, ${\mathrm{ch}}^{*},{\theta }^{*}\in \mathbb{Z}$, let $\mathcal{H}(d_{max},n^{*},{\mathrm{cs}}^{*},{\mathrm{ch}}^{*},{\theta }^{*})$ denote the set of rank-2 graphs H such that the degree of each core vertex is at most 4, the degree of each non-core vertex is at most $d_{max}$, $n\left(H\right)=n^{*}$, $\mathrm{cs}\left(H\right)={\mathrm{cs}}^{*}$, $\mathrm{ch}\left(H\right)={\mathrm{ch}}^{*}$ and $\theta \left(H\right)={\theta }^{*}$. In this paper, we obtain the following result.

Theorem 2.

Let Λ be a set of chemical elements, Γ be a set of adjacency-configurations, where $|\Lambda |\le |\Gamma |$, and $k=2|\Lambda |+2|\Gamma |+15$. Given integers $d_{max}\in \{3,4\}$, $n^{*}\ge 3$, ${\mathrm{cs}}^{*}\ge 3$ ${\mathrm{ch}}^{*}\ge 0$ and ${\theta }^{*}$, there is an MILP $\mathcal{M}(x,g;{\mathcal{C}}_2)$ that consists of variable vectors $x\in {\mathbb{R}}^k$ and $g\in {\mathbb{R}}^q$ for some integer $q=O\left(\right|\Gamma |·{\mathrm{cs}}^{*}·{(d_{max}-1)}^{{\mathrm{ch}}^{*}})$ and a set ${\mathcal{C}}_2$ of $O\left(\right|\Gamma |+{\mathrm{cs}}^{*}·{(d_{max}-1)}^{{\mathrm{ch}}^{*}})$ constraints on these variables such that: $(x^{*},g^{*})$ is feasible to $\mathcal{M}(x,g;{\mathcal{C}}_2)$ if and only if $g^{*}$ forms a rank-2 chemical graph $G^{*}=(H,\alpha ,\beta )\in \mathcal{G}(\Lambda ,\Gamma )$ such that $H\in \mathcal{H}(d_{max},n^{*},{\mathrm{cs}}^{*},{\mathrm{ch}}^{*},{\theta }^{*})$ and $f\left(G^{*}\right)=x^{*}$.

Note that our MILP requires only $O\left(n^{*}\right)$ variables and constraints when the maximum core height of a subtree in the non-core of $G^{*}$ and $|\Gamma |$ are constant. We formulate an MILP in Theorem 2 so that such a graph H is selected as a subgraph of the scheme graph.

We explain the basic idea of our MILP. Define

$$t^{*}\triangleq {\mathrm{cs}}^{*}-s^{*},$$

$$c^{*}\triangleq |E_1\cup E_2|for(K,\mathcal{E}=(E_1,E_2,E_3)),$$

$$n_{\mathrm{tree}}\triangleq 1+2({(d_{max}-1)}^{{\mathrm{ch}}^{*}}-1)/(d_{max}-2)and{n}_{\mathrm{in}}\triangleq 1+2({(d_{max}-1)}^{{\mathrm{ch}}^{*}-1}-1)/(d_{max}-2),$$

where $n_{\mathrm{tree}}$ and $n_{\mathrm{in}}$ are the numbers of vertices and non-leaf vertices in the rooted tree $T(d_{max}-2,d_{max}-1,{\mathrm{ch}}^{*})$, respectively. The MILP mainly consists of the following three types of constraints.

• Constraints for selecting a rank-2 graph H as a subgraph of the $(t^{*},{\mathrm{ch}}^{*},d_{max})$-tree-extension of the scheme graph $(K,\mathcal{E})$;
• Constraints for assigning chemical elements to vertices and multiplicity to edges to determine a chemical graph $G=(H,\alpha ,\beta )$;
• Constraints for computing descriptors from the selected rank-2 chemical graph G; and
• Constraints for reducing the number of rank-2 chemical graphs that are isomorphic to each other but can be represented by the above constraints.

In the constraints of 1, we treat each edge in the tree-extension as a directed edge because describing some condition for H to belong to $\mathcal{H}(d_{max},n^{*},{\mathrm{cs}}^{*},{\mathrm{ch}}^{*},{\theta }^{*})$ becomes slightly easier than the case of undirected graphs. More formally we prepare the following.

(i)

In the scheme graph $(K,\mathcal{E})$, denote the edges in $E_1\cup E_2\cup E_3$ by $E_1=\{a_1,a_2,\dots ,a_{|E_1|}\}$, $E_2=\{a_{|E_1|+1},\dots ,a_{c^{*}}\}$ and $E_3=\{a_{c^{*}+1},\dots ,a_m\}$ (where $c^{*}=|E_1\cup E_2|$), and regard each edge $a_i=u_{s,1}{u}_{s^{\prime },1}\in E_1\cup E_2\cup E_3$ as a directed edge from one end-vertex $u_{s,1}$ to the other end-vertex $u_{s^{\prime },1}$ with $s<s^{\prime }$. Let $a\left(i\right)$ be a binary variable for each edge $a_i$, $i\in [1,m]$.

(ii)

In each tree $S_s$ (resp., $T_t$) in the tree-extension, we regard each edge $e_{s,i}^{\prime }$, $i\ge 2$ in the rooted tree $S_s$, $s\in [1,s^{*}]$ (resp., $e_{t,i}$, $i\ge 2$ in the rooted tree $T_t$, $t\in [1,t^{*}]$) as a directed edge from vertex $u_{s,\mathrm{prt}\left(i\right)}$ to vertex $u_{s,i}$ (resp., from vertex $v_{t,\mathrm{prt}\left(i\right)}$ to vertex $v_{t,i}$). Let $u(s,i)$ (resp., $v(t,i)$) be a binary variable for vertex $u_{s,i}$, $s\in [1,s^{*}]$ (resp., $t\in [1,t^{*}]$) and $i\in [1,n_{\mathrm{tree}}]$;

(iii)

In the path $P_{t^{*}}$ consisting of the roots of trees $T_t$, $[t\in 1,t^{*}]$, we regard each edge $e_t$, $t\in [2,t^{*}]$ as a directed edge from vertex $v_{t-1,1}$ to vertex $v_{t,1}$; and

(iv)

We regard each edge $u_{s,1}{v}_{t,1}$ for $s\in [1,s^{*}]$ and $t\in [1,t^{*}]$ as two directed edges, one directed from vertex $u_{s,1}$ to vertex $v_{t,1}$ and the other directed oppositely. Let $e(s,t)$ (resp., $e(t,s)$) be a binary variable of directed edge $(u_{s,1},v_{t,1})$ (resp., $(v_{t,1},u_{s,1})$).

Based on these, we include constraints with some more additional variables so that a selected subgraph H is a connected rank-2 graph. See constraints Equations (A10) to (A42) in Appendix A for the details.

In the constraints of 2, we prepare an integer variable $\tilde{\alpha }\left(u\right)$ for each vertex u in the tree-extension that represents the chemical element $\alpha \left(u\right)\in \Lambda$ if u is in a selected graph H (or $\tilde{\alpha }\left(u\right)=0$ otherwise) and an integer variable $\tilde{\beta }\left(e\right)\in [0,3]$ (resp., $\widehat{\beta }\left(e\right)\in [0,3]$) for each edge e (resp., $e=e(s,t)$ or $e(t,s)$, $s\in [1,s^{*}]$, $t\in [1,t^{*}]$) in the tree-extension that represents the multiplicity $\beta \left(e\right)\in [1,3]$ if e is in a selected graph H (or $\tilde{\beta }\left(e\right)$ or $\widehat{\beta }\left(e\right)$ takes 0 otherwise). This determines a chemical graph $G=(H,\alpha ,\beta )$. Also we include constraints for a selected chemical graph G to satisfy the valence condition $\left(\alpha \right(u),\alpha (v),\beta (uv\left)\right)\in \Gamma$ for each edge $uv\in E$. See constraints Equations (A43) to (A61) in Appendix A for the details.

In the constraints of 3, we introduce a variable for each descriptor and constraints with some more variables to compute the value of each descriptor in $f\left(G\right)$ for a selected chemical graph G. See constraints Equations (A62) to (A113) in Appendix A for the details.

With constraints 1 to 3, our MILP formulation already represents a rank-2 chemical graph G and a feature vector $x\in {\mathbb{R}}^k$ so that $x=f\left(G\right)$ holds. In the constraints of 4, we include some additional constraints so that the search space required for an MILP solver to solve an instance of our MILP problem is reduced. For this, we consider a graph-isomorphism of rooted subtrees of each tree $S_s$ or $T_s$ and define a canonical form among subtrees that are isomorphic to each other. We try to eliminate a chemical graph G that has a subtree in $S_s$ or $T_s$ that is not a canonical form. See constraints Equations (A114) to (A119) in Appendix A for the details.

3. Results

We implemented our method of Steps 1 to 5 for inferring rank-2 chemical graphs and conducted experiments to evaluate the computational efficiency for three chemical properties $\pi$: octanol/water partition coefficient (Kow), melting point (Mp), and boiling point (Bp). We executed the experiments on a PC with Intel Core i5 1.6 GHz CPU and 8GB of RAM running under the Mac OS operating system version 10.14.6. We show 2D drawings of some of the inferred chemical graphs, where ChemDoodle version 10.2.0 is used for constructing the drawings.

Results on Phase 1.

Step 1. We set a graph class $\mathcal{G}$ to be the set of all rank-2 chemical graphs. For each property $\pi \in$ {Kow, Mp, Bp}, we select a set $\Lambda$ of chemical elements and collected a data set $D_{\pi }$ on rank-2 chemical graphs over $\Lambda$ provided by HSDB from PubChem. To construct the data set, we eliminated chemical compounds that have at most three carbon atoms or contain a charged element such as ${\mathtt{N}}^{+}$ or an element $\mathtt{a}\in \Lambda$ in which the valence is different from our setting of valence function $\mathrm{val}$.

Table 1. Results of Step 1 in Phase 1.

$\mathit{\pi }$	$|{\mathit{D}}_{\mathit{\pi }}|$	$\mathit{\Lambda }$	$\left|\mathsf{\Gamma }\right|$	$[\underline{\mathit{n}},\overline{\mathit{n}}]$	$[\underline{\mathbf{cs}},\overline{\mathbf{cs}}]$	$[\underline{\mathbf{ch}},\overline{\mathbf{ch}}]$	$[\underline{\mathit{\theta }},\overline{\mathit{\theta }}]$	$[\underline{\mathit{a}},\overline{\mathit{a}}]$
Kow	93	C,N,O	9	[9, 31]	[7, 16]	[0, 13]	[−5, 3]	[$-3.7$, 12.2]
Mp	63	C,N,O	7	[9, 31]	[7, 17]	[0, 4]	[−6, 3]	[−80, 300]
Bp	45	C,N,O,S,P,Cl	9	[9, 25]	[7, 15]	[0, 7]	[−4, 3]	[155, 420]

shows the size and range of data sets that we prepared for each chemical property in Step 1, where we denote the following:

-

$\pi$: one of the chemical properties Kow, Mp and Bp;

-

$|D_{\pi }|$: the size of data set $D_{\pi }$ for property $\pi$;

-

$\Lambda$: the set of chemical elements over data set $D_{\pi }$ (hydrogen atoms are added at the final stage);

-

$|\Gamma |$: the number of tuples in $\Gamma$;

-

$[\underline{n},\overline{n}]$: the minimum and maximum number $n\left(G\right)$ of non-hydrogen atoms over data set $D_{\pi }$;

-

$[\underline{\mathrm{cs}},\overline{\mathrm{cs}}]$, $[\underline{\mathrm{ch}},\overline{\mathrm{ch}}]$: the minimum and maximum core size and core height over chemical compounds in $D_{\pi }$, respectively;

-

$[\underline{\theta },\overline{\theta }]$: the minimum and maximum values of the topological parameter $\theta \left(G\right)$ over data set $D_{\pi }$; and

-

$[\underline{a},\overline{a}]$: the minimum and maximum values of $a\left(G\right)$ in $\pi$ over data set $D_{\pi }$.

Step 2. We used a feature function f that consists of the descriptors defined in Section 2.1.

Step 3. We used scikit-learn version 0.21.6 with Python 3.7.4 to construct ANNs $\mathcal{N}$ where the tool and activation function are set to be MLPRegressor and ReLU, respectively. We tested several different architectures of ANNs for each chemical property. To evaluate the performance of the resulting prediction function ${\psi }_{\mathcal{N}}$ with cross-validation, we partition a given data set $D_{\pi }$ into five subsets $D_{\pi }^{\left(i\right)}$, $i\in [1,5]$ randomly, where $D_{\pi }\setminus D_{\pi }^{\left(i\right)}$ is used for a training set and $D_{\pi }^{\left(i\right)}$ is used for a test set in five trials $i\in [1,5]$. For a set $\{y_1,y_2,\dots ,y_N\}$ of observed values and a set $\{{\psi }_1,{\psi }_2,\dots ,{\psi }_N\}$ of predicted values, we define the coefficient of determination to be ${\mathrm{R}}^2\triangleq 1-\frac{{\sum }_{j\in [1,N]}{(y_j-{\psi }_j)}^2}{{\sum }_{j\in [1,N]}{(y_j-\overline{y})}^2}$, where $\overline{y}=\frac{1}{N}{\sum }_{j\in [1,N]}{y}_j$.

Table 2. Results of Steps 2 and 3 in Phase 1.

$\mathit{\pi }$	k	Activation	Architecture	L-Time	Test ${\mathbf{R}}^2$ (ave.)	(Best)
Kow	37	relu	(37,10,1)	3.92	0.866	0.964
Mp	33	relu	(33,10,1)	21.68	0.805	0.916
Bp	43	relu	(43,10,1)	11.88	0.802	0.947

shows the results on Steps 2 and 3, where

-

k: the number of descriptors for the chemical compounds in data set $D_{\pi }$ for property $\pi$;

-

Activation: the choice of activation function;

-

Architecture: $(a,b,1)$ consists of an input layer with a nodes, a hidden layer with b nodes, and an output layer with a single node, where a is equal to the number of descriptors;

-

L-time: the average time (sec.) to construct ANNs for each trial;

-

test ${\mathrm{R}}^2$ (ave.): the average of coefficient of determination over the five test sets; and

-

test ${\mathrm{R}}^2$ (best): the largest value of coefficient of determination over the five test sets.

For each chemical property $\pi$, we selected the ANN $\mathcal{N}$ that attained the best test ${\mathrm{R}}^2$ score among the five ANNs to formulate an MILP $\mathcal{M}(x,y,z;{\mathcal{C}}_1)$ in the second phase.

Results on Phase 2.

We implemented Steps 4 and 5 in Phase 2 as follows.

Step 4. In this step, we solve the MILP $\mathcal{M}(x,y,g;{\mathcal{C}}_1,{\mathcal{C}}_2)$ formulated based on the ANN $\mathcal{N}$ obtained in Phase 1. To solve an MILP in Step 4, we use CPLEX version 12.10. In our experiment, we choose a target value $y^{*}\in [\underline{a},\overline{a}]$ and fix or bound some descriptors in our feature vector as follows:

-

Fix variable $\theta$ that represents the polymer parameter $\theta \left(H\right)$ to be each integer in $\{-2,0,2\}$;

-

Set $d_{\max }$ to be each of 3 and 4;

-

Fix $n^{*}$ to be some four integers in $\{15,19,20,25,30\}$ for $\theta \in \{-2,0\}$ and $\{15,19,20,22,25\}$ for $\theta =2$;

-

Choose three integers from $[7,16]$ and fix ${\mathrm{cs}}^{*}$ to be each of the three integers;

-

Fix ${\mathrm{ch}}^{*}$ to be each of the four integers in $[2,5]$.

Based on the above setting, we generated 12 instances for each $n^{*}$. We set $\epsilon =0.02$ in Step 4.

Table 3. Results of Steps 4 and 5 with $d_{\max }=3$ and $\theta =-2$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	12/12	9.96	100	2236.0
Kow	5	20	12/12	30.38	12	$>$1 h
Kow	5	25	12/12	47.57	12	$>$1 h
Kow	5	30	12/12	69.38	12	$>$1 h
Mp	150	15	12/12	9.52	100	2069.0
Mp	150	20	12/12	22.79	12	$>$1 h
Mp	150	25	12/12	47.20	12	$>$1 h
Mp	150	30	12/12	66.90	12	$>$1 h
Bp	250	15	11/12	9.50	100	103.5
Bp	250	19	12/12	19.08	12	$>$1 h
Bp	250	22	12/12	25.78	12	$>$1 h
Bp	250	25	12/12	67.64	12	$>$1 h

,

Table 4. Results of Steps 4 and 5 with $d_{\max }=4$ and $\theta =-2$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	11/12	31.84	100	413.8
Kow	5	20	12/12	69.65	12	$>$1 h
Kow	5	25	12/12	144.20	11	$>$1 h
Kow	5	30	12/12	352.01	12	$>$1 h
Mp	150	15	9/12	20.68	100	947.4
Mp	150	20	11/12	73.73	11	$>$1 h
Mp	150	25	9/12	140.09	9	$>$1 h
Mp	150	30	12/12	304.04	12	$>$1 h
Bp	250	15	7/12	28.51	100	232.7
Bp	250	19	11/12	82.01	11	$>$1 h
Bp	250	22	12/12	150.55	12	$>$1 h
Bp	250	25	12/12	239.84	12	$>$1 h

,

Table 5. Results of Steps 4 and 5 with $d_{\max }=3$ and $\theta =0$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	12/12	11.00	100	121.1
Kow	5	20	12/12	25.64	12	$>$1 h
Kow	5	25	12/12	38.79	12	$>$1 h
Kow	5	30	12/12	49.65	12	$>$1 h
Mp	150	15	12/12	8.45	100	373.4
Mp	150	20	12/12	18.94	12	$>$1 h
Mp	150	25	12/12	37.13	12	$>$1 h
Mp	150	30	12/12	44.745	4	$>$1 h
Bp	250	15	9/12	8.450	100	74.2
Bp	250	19	11/12	16.31	11	$>$1 h
Bp	250	22	12/12	21.71	12	$>$1 h
Bp	250	25	12/12	45.80	12	$>$1 h

,

Table 6. Results of Steps 4 and 5 with $d_{\max }=4$ and $\theta =0$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	9/12	36.33	100	23.2
Kow	5	20	12/12	82.01	12	$>$1 h
Kow	5	25	12/12	138.96	12	$>$1 h
Kow	5	30	12/12	292.79	12	$>$1 h
Mp	150	15	9/12	19.89	100	557.6
Mp	150	20	11/12	63.62	11	$>$1 h
Mp	150	25	12/12	112.49	12	$>$1 h
Mp	150	30	12/12	171.11	12	$>$1 h
Bp	250	15	3/12	34.60	100	11.2
Bp	250	19	6/12	203.65	6	$>$1 h
Bp	250	22	9/12	218.07	9	$>$1 h
Bp	250	25	11/12	783.80	11	$>$1 h

,

Table 7. Results of Steps 4 and 5 with $d_{\max }=3$ and $\theta =2$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	12/12	11.64	100	1386.7
Kow	5	20	12/12	23.84	12	>1 h
Kow	5	25	12/12	33.71	12	>1 h
Kow	5	30	12/12	61.85	12	>1 h
Mp	150	15	12/12	9.80	100	1614.3
Mp	150	20	12/12	20.15	12	>1 h
Mp	150	25	12/12	36.42	12	>1 h
Mp	150	30	12/12	40.58	12	>1 h
Bp	250	15	11/12	10.25	100	1756.1
Bp	250	19	12/12	16.02	12	>1 h
Bp	250	22	12/12	23.63	12	>1 h
Bp	250	25	12/12	63.84	12	>1 h

and

Table 8. Results of Steps 4 and 5 with $d_{\max }=4$ and $\theta =2$.

$\mathit{\pi }$	${\mathit{y}}^{*}$	${\mathit{n}}^{*}$	$|{\mathit{F}}^{*}|/\#$I	IP-Time	$\#{\mathit{G}}^{*}$	G-Time
Kow	5	15	11/12	28.15	100	20.3
Kow	5	20	12/12	71.90	12	$>$1 h
Kow	5	25	12/12	112.71	12	$>$1 h
Kow	5	30	12/12	267.21	12	$>$1 h
Mp	150	15	9/12	22.53	100	2748.1
Mp	150	20	11/12	53.44	11	>1 h
Mp	150	25	12/12	143.33	12	>1 h
Mp	150	30	12/12	220.63	12	>1 h
Bp	250	15	6/12	27.33	100	254.2
Bp	250	19	9/12	75.50	9	>1 h
Bp	250	22	11/12	133.01	11	>1 h
Bp	250	25	12/12	228.75	12	>1 h

show the results of Step 4 for $d_{max}=3$ and 4, respectively, where we denote the following:

-

$y_{\pi }^{*}$: a target value in $[\underline{a},\overline{a}]$ for a property $\pi$;

-

$n^{*}$: a specified number of vertices in $[\underline{n},\overline{n}]$;

-

$|F^{*}|/\#$I: #I means the number of MILP instances in Step 4 (where #I = 12), and $|F^{*}|$ means the size of set $F^{*}$ of vectors $x^{*}$ generated from all feasible instances among the #I MILP instances in Step 4;

-

IP-time: the average time (sec.) to solve one of the #I MILP instances to find a set $F^{*}$ of vectors $x^{*}$.

Figure 10a–c illustrate some rank-2 chemical graphs $G^{*}$ with $\theta \left(G^{*}\right)=-2$ constructed from the vector $g^{*}$ obtained by solving the MILP in Step 4.

Figure 11a–c illustrate some rank-2 chemical graphs $G^{*}$ with $\theta \left(G^{*}\right)=0$ constructed from the vector $g^{*}$ obtained by solving the MILP in Step 4.

Figure 12a–c illustrate some rank-2 chemical graphs $G^{*}$ with $\theta \left(G^{*}\right)=2$ constructed from the vector $g^{*}$ obtained by solving the MILP in Step 4.

Step 5. In this step, we modified the algorithms proposed by Tamura et al. [25] and Yamashita et al. [26] to enumerate all rank-2 graphs $G^{*}\in \mathcal{G}$ such that $f\left(G^{*}\right)=x^{*}$ for each $x^{*}\in F^{*}$. We stop the execution when either the total number of graphs inferred over all vectors $x^{*}\in F^{*}$ exceeds 100 or the execution time exceeds one hour.

Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8 show the results on Step 5 for $d_{max}=3$ and 4, respectively,

-

$\#G^{*}$: the number of all (or up to 100) rank-2 chemical graphs $G^{*}$ that are computed under 1 h time limit in Step 5, where $f\left(G^{*}\right)=x^{*}$ for some $x^{*}\in F^{*}$. (Note that $|F^{*}|$ such graphs $G^{*}$ have been found in Step 4, and Figure 10, Figure 11 and Figure 12 illustrate some of such graphs $G^{*}$.);

-

G-time: the running time (sec.) to execute Step 5, where “>1 h” means that the execution time exceeds the limit.

We also conducted some additional experiments to demonstrate that our MILP-based method is flexible to control conditions on the inference of chemical graphs. In Step 3, we constructed an ANN ${\mathcal{N}}_{\pi }$ for each of the three chemical properties $\pi \in \{$Kow, Mp, Bp}, and formulated the inverse problem of each ANN ${\mathcal{N}}_{\pi }$ as an MILP ${\mathcal{M}}_{\pi }$. Since the set of descriptors is common to all three properties Kow, Mp, and Bp, it is possible to infer a rank-2 chemical graph $G^{*}$ that satisfies a target value $y_{\pi }^{*}$ for each of the three properties at the same time (if one exists). We specify the size of graph so that $n:=22$, core size := 14, core height := 3, $\theta :=-2$ and $d_{max}:=3$, and set target values with $y_{\mathrm{K}\mathrm{ow}}^{*}:=5$, $y_{\mathrm{Mp}}^{*}:=150$ and $y_{\mathrm{Bp}}^{*}:=250$ in an MILP that consists of the three MILPs ${\mathcal{M}}_{\mathrm{K}\mathrm{ow}}$, ${\mathcal{M}}_{\mathrm{Mp}}$ and ${\mathcal{M}}_{\mathrm{Bp}}$. The MILP was solved in 268.11 (sec) and we obtained a rank-2 chemical graph $G^{*}$ illustrated in Figure 10d.

4. Discussion

In this paper, we proposed a new method for the inverse QSAR/QSPR to rank-2 chemical graphs by significantly enhancing the framework due to Azam et al. [18], Zhang et al. [20], and Ito et al. [21], and implemented it for inferring rank-2 chemical graphs using the algorithms for enumerating rank-2 chemical graphs due to Tamura et al. [25] and Yamashita et al. [26]. From the results on some computational experiments, we observe that the proposed method runs efficiently for an instance with $n^{*}\le 30$ non-hydrogen atoms up to Step 4 and an instance with $n^{*}\le 15$ non-hydrogen atoms up to Step 5. Due to this development, the ratio of chemical compounds covered in the PubChem database increased from 16.26% to 44.5%. It is left as future work to apply our new method for the inverse QSAR/QSPR to a wider class of graphs. The ratio of the number of chemical graphs with rank at most 3 (resp., 4) to the number of all chemical graphs in database PubChem is 68.8% (resp., 84.7%). Among rank-4 chemical compounds, Remdesivir ${\mathtt{C}}_{27}{\mathtt{H}}_{35}{\mathtt{N}}_6{\mathtt{O}}_8\mathtt{P}$, an antiviral medication, which is being studied as a possible post-infection treatment for COVID-19, has a chemical graph G with $\mathrm{r}\left(G\right)=4$, $n\left(G\right)=42$, $\mathrm{cs}\left(G\right)=24$, and $\mathrm{ch}\left(G\right)=8$. The number of polymer topologies with rank 3 (resp., 4) such that the maximum degree is at most 4 is 12 (resp., 73). Our MILP formulation can be easily extended to the case of rank 3 or 4 by replacing the current set of constraints for the scheme graph with a set of those for a new scheme graph that is designed for rank-3 or -4 polymer topologies.