Cognitive Hybrid Intelligent Diagnostic System: Typical Architecture

Abstract

The research refers to the modeling of the meaningful and relatively stable visual-figurative and verbal-sign representation of real problems in medical diagnostics of the human organs and systems. The core results of the research are presented. Here, a new visual metalanguage is proposed. It describes the solution of a diagnostic problem by combining several interconnected processes of reasoning in different languages defining “a state of human organs and systems”, “a diagnostic problem” and elements of its decomposition. In the paper, a subject-figurative model of the cognitive hybrid intelligent diagnostic system, its typical architecture, and a synthesis algorithm are provided. Due to the integration of imitation of an internal subject-figurative vision of medical diagnostic problems and the corresponding communication statements of private diagnoses with imitation of the behavior inherent for councils in problem situations, the future implementation of such system prototypes will reduce the number of medical errors. The further stage of this research is the approbation of all solutions for the problem of diagnosing diseases of the pancreas on the materials of the Kaliningrad Regional Clinical Hospital and experimental study of the system. The research is limited by the subject area of medicine but can be generalized to the other areas.

1. Introduction

Nowadays, a significant variety of processed information and an unprecedented rate of increase in its volume, as well as the ambiguity of the assessment of diagnostic situations, errors in the choice of priorities, and the inconsistency of requirements, contribute to the ever more frequent occurrence of situations with insufficiently identified or detected conditions and with an uncertain purpose. A diagnostic problem (DP) is a complex (heterogeneous) problem of recognizing a disease and assessing individual biological characteristics and social status of the object of diagnosis (patient) by interpreting the results of a targeted medical examination and summarizing them in the form of an established diagnosis. The nature of decision-making regarding the DP is multilingual. Its composition and structure are variables. At the same time, the clinical thinking cannot be identified with scientific (formal-logical), philosophical, or figurative-artistic thinking, since it is a combination of all these types of thinking. The main difficulty lies in the fact that in each specific case the proportion of different types of thinking is always different, which predetermines the uniqueness and complexity of medical thinking.

Hence, the study of cognitive structures and mechanisms of a specialist in medicine is relevant, as well as the development of functional hybrid intelligent systems (HIS) or HIS with cognitive modeling of a diagnostic problem based on its results. It will make the processes of overcoming a diagnostic problem visible and contrasting.

The purpose of the study is to improve the quality of diagnoses by solving the following tasks:

• To model the image of a diagnostic medical problem;
• To imitate mixed, integrated visual-figurative and verbal-sign representations of the problem by experts;
• To imitate collective restructuring of reduced representation of the problem image according to the principle of consultation using the methods of HISs as decision support diagnostic systems with cognitive visualization of problems arising in medicine.

Within the research, theories, methods, models, and information technologies of representation of mental objects “diagnostic problem” and its decomposition elements, as well as methods of reduction of complex problems including diagnostic problems, have been analyzed.

The core results of the research are:

• The visual metalanguage;
• Representations of the mental image “an integrated method for solving a diagnostic problem” and a heuristic mechanism for solving problems by dynamically restructuring the whole into a decomposition of related problems as well as a language for their description;
• A subject-figurative model of cognitive hybrid intelligent diagnostic systems (CHIDS), their typical architecture, and synthesis algorithm.

Method of reduction of the diagnostic problem in medicine and formation of the representation of methods of solving functional and technological subproblems from the decomposition of the diagnostic problem, as well as verbal-sign languages describing individual and collective reasoning, have been developed.

In the paper, only the core results are presented. This document is structured as follows. Section 2 presents readers with the literature review of approaches and methods of patients’ state representation. Section 3 introduces readers to Materials and Methods. Section 4 explains the core results of the research. Section 5 includes the discussion of the results and future research work. Finally, Section 6 shows the assumptions.

2. Literature Review

The founders of cognitive computer graphics are D.A. Pospelov and A.A. Zenkin. I.B. Fominykh [1] is the founder of image engineering. They laid down the basic terminology and defined tasks. Their ideas were developed by B.A. Kobrinsky, V.B. Tarasov, O.P. Kuznetsov, Yu.R. Valkman, V.M. Khachumov, A.A. Bashlykov, and A.E. Yampolskaya.

Within the studies of V.M. Khachumov directed at cognitive visualization of controlled objects [2], multidimensional data are correlated to a cognitive graphic image in the form of integral functional profiles or scenes with the help of a computer. Here, a continuous monitoring of the state of patients with bronchial asthma is provided. States and reference values of the process are visualized with three-dimensional images (stars). The star increases and becomes more solid with respiratory distress; if a temperature or a pulse rate increases then points of a star elongate; with a growth in the parameters of blood gases, a star increases with a possible change in the common structure without a pronounced effect of smoothing or separation.

Works by B.A. Kobrinsky [3,4] are aimed at problems of figurative representations of specialists in medicine and the inclusion of visual images in the knowledge bases (KB) of intelligent systems. He emphasizes the relevance of creating hybrid visualized knowledge bases with logical-intuitive-figurative representations. Here, the latter can be in the form of complexly described situations-precedents, including metaphorically presented (diagnostically significant) integral manifestations of the pathological process. Two other forms of it are specific manifestations of the disease as associated signs and visualized manifestations of the disease (drawings, photos, etc.). They amplify verbalized concepts and facilitate diagnostic decision-making based on similarity with images of patients having certain diseases.

E. Vardell and C. Bou-Crick [5] propose a system of visual diagnostic decisions support after a preliminary diagnosis has been made by a doctor. The system contains 1212 uniquely defined diagnoses and 24,115 images of manifestations of diseases. It is aimed at identifying a number of diseases including dermatological, infectious, genetic ones, etc.

F. Rassom [6] identifies the main trends in the field of visualization—the development of complex types of diagrams, an increase in the level of interaction with user visualization, and an increase in the size and complexity of data structures represented by visualization. These trends are also typical for medical information systems. For example, C. Plaisant [7] and T. Wang [8] describe the information systems LifeLines and LifeLines2, in which medical records are divided according to their inherent aspects, such as problems, symptoms, examinations/results, diagnoses, treatments, and medications, etc. The color is used to indicate the severity or type of disease. The level of detail of the mechanism allows scaling of the patient’s medical history. The group of authors of [9] presented the AnamneVis system where all information about a patient is presented in the form of a diagram that shows all human diseases in the past and present. The human body in AnamneVis is stylized as a map of organs and systems easily enlarged to detail down to the smallest anatomical structures. Another frequently used method for visualizing patient data is topological and temporal plan execution diagrams in the Asgaard project [10]. The first one displays the links between the planned activities, and the second one is based on the time dimension in the planning and various conditions.

There are also several works [11,12,13,14] which investigate medical artificial intelligence with visualization of information, but they have no representation of patients’ state or processes of diagnostics.

Thus, there is no common understanding of the phenomenon of the image of the state of the human body, its individual systems, and elements of the diagnostic problem (plan and various “homogeneous subproblems”). The way to reduce the informational and cognitive load on the doctor diagnosing the patient is the modeling of an integrated and mixed visual-figurative and verbal-sign representation of a diagnostic problem. This will also improve the quality and the safety of medical intelligent diagnostic decision support systems.

3. Materials and Methods

Under the study, methods of systems theory and system analysis, decision making, theory and technology for the development of hybrid intelligent systems, medical technological processes, psychology of thinking, cognitive graphics, image engineering, as well as a systematic approach to the analysis of body, were used.

The approach of P.K. Anokhin was used to remove the uncertainty of diagnostic situations. According to the approach, there are three stages of research: the stage of preparation (“pre-decision”), the actual procedure of decision-making, and the stage of its implementation.

Within the informative preparation (P.K. Anokhin), a “diagnostic problem” and its elements in medicine were considered from a new point of view—a two-level representation and solution of them (M.V. Samsonova, V.V. Efimova) [15] were supplemented by the solution mechanisms according to V.F. Spiridonov [16] based on psychological studies of beginners and experts. These are “self-determination of the solver” (G.P. Shchedrovitsky) [16] and “changing the presentation of the problem (its processing) in the course of solving”. The latter presents a solved problem as a hierarchically built system of subproblems and serving them tasks that directly imply more or less clear methods of solution. To develop a new method for reducing the diagnostic problem, taking into account the cognitive visualization of its elements, the results of research by V.F. Spiridonov [10] on the mechanisms for solving problems and tasks as well as the results of an investigation in the field of system analysis aimed at the features of reduction process and presentation of its result. Reduction methods based on knowledge were used [17].

To develop languages of schemes of role-playing visual relations, the following theories and methods were used:

• The developed and tested in practice axiomatic theory of role conceptual models using the tetrad of concepts “resource-property-action-relationship” [18] (A.V. Kolesnikov) which follows from the triad of A.V. Uyomov “thing-property-relationship” [19];
• The Nazarenko–Osipov theory of medical technological processes [20,21];
• The approach of M.A. Gaides to the system analysis of the human body [22];
• The principle of virtual consultation by S.B. Rumovskaya [17] according to which the virtual team model was developed as a stratified (multilevel) model of a council of highly specialized experts following the works of M. Mesarovich and I. Takahara on the theory of hierarchical multilevel systems [23].

Next comes the second stage named “decision-making” according to P.K. Anokhin. Here, the axiomatic theory of role conceptual models (A.V. Kolesnikov) [18], the concept of thinking by D.A. Pospelov and V.N. Pushkin [24], the principle of consultation by S.B. Rumovskaya [17], and studies of N.L. Morina, D.A. Oshanina [25], and L.D. Lindenbrathan [26] were used. According to the latter, the diagnosis is made in the process of comparing the image of current state of the object with a set of sample images that are images of various pathological abnormalities and the norm. The more experienced the doctor, the greater the number of sample images he has. Respectively, their quality is much higher, and the efficiency of formation of the image underlying the diagnosis is higher, and the probability of error is lower. Thinking, according to D.A. Pospelov and V.N. Pushkin [24], is the establishment of relations between objects. By special mental mechanisms, the object of diagnosis is recreated in the head of a person (doctor) with a reflection of detected properties and relations between resources. In contrast to the language of perception which fixes those properties of resources that are manifested in the effects on the sense organs, thinking is characterized by a special language of relations and links, through which the subject (doctor) gets the opportunity for internal work with those objects (subsystems of the body) and their properties (that are not given in perception) which are outside the scope of its direct contacts with the object of diagnosis. The results of the second stage are: languages of verbal-sign statements of individual activity aimed at solving diagnostic problems and subtasks, determining subject solutions, searching for diagnostic links, and eliminating compensated difficulties; a description of collective activity to overcome diagnostic problems and the language of subject-figurative representation of mental image “an integrated method for solving a diagnostic problem”.

The method of forming subject-figurative models of concepts of methods for solving homogeneous diagnostic problems, which are elements of the decomposition of a diagnostic problem and the heterogeneous diagnostic problem, were developed on the basis of the research by V.F. Spiridonov [16], the principle of consultation by S.B. Rumovskaya [17], and the research by B.A. Kobrinsky on the inclusion of visual-figurative representations in medical knowledge bases and at different stages of decision-making [3,4].

At the last stage of implementation of solutions to develop the typical architecture and the algorithm for the synthesis of cognitive HIS, the following methods and approaches were used: problem-structural methodology of functional HIS (by A.V. Kolesnikov and I.A. Kirikov) [27]; an engineering technique for automated solving of a complex medical diagnostic problem using the Virtual Council tool environment (S.B. Rumovskaya); and experience in applying a systematic approach and analytical methods of solving problems in medicine including the approach of M.A. Gaides to the analysis of the human body [22].

4. The Core Results

4.1. Metalanguage for Description and Representation of Patient’s States

The visual metalanguage of an axiomatic theory of role-based visual relations of a problem diagnostic situation and its elements within the categorical core “resource-property-action–relation” is proposed at the Figure 1.

It determines the relevant mapping of the verbal-symbolic and visual-spatial aspects of intellectual activity into the architecture and algorithms of artificial intelligence. The metalanguage includes nine levels, each of which is represented by a subject-figurative core and a verbal-sign shell. The last one is the synthesis of names of resources, their properties, actions of objects, subjects, cause-and-effect and other relations, which are taken from everything that can be expressed by concepts, definitions, sentences, and texts in the language of professional activity.

The axiomatic theory of role visual diagnostic models is based on the understanding of visual language as a semiotic system:

$$S_d=\langle Y_d,G_d,M_d,V_d,{\delta }_d\rangle ,$$

(1)

$$Y_d=\langle K{Y}_d,V{Y}_d,\Psi \rangle , K{Y}_d=\langle {}^{1}X,{}^{2}X,{}^{3}X,R\rangle , V{Y}_d=\langle {}_d{}^{1}V,{}_d{}^{2}V,{}_d{}^{3}V,{}_d{}^{}R{}_v\rangle ,$$

(2)

$$\Psi =〈{}^1\Psi ,{}^2\Psi ,{}^3\Psi ,{}^R\Psi 〉, {}^1\Psi :{}^{1}X\leftrightarrow {}_d^{1}V, {}^2\Psi :{}^{2}X\leftrightarrow {}_d^{2}V, {}^3\Psi :{}^{3}X\leftrightarrow {}_d^{3}V,{}^2\Psi :R\leftrightarrow {}_{d}R,$$

(3)

$${}_v{G}_d=〈V{Y}_{d^{\prime }}N,{}^{p}P〉, {}_v{M}_d=〈K{Y}_d,G_d〉, V_v{V}_d=\left\{〈E,a_d,Mod,C_d〉\right\},$$

(4)

where $Y_d$—conceptual-visual categorical core; $G_d,M_d,V_d$—sets of syntactic, semantic (knowledge about the subject area of medical diagnostics), and pragmatic rules (a set of rules for inferring diagnostic solutions), respectively; ${\delta }_d$—transformation rules for the sets $Y_d,$$G_d,M_d,V_d$; $K{Y}_d$—conceptual categorical core; $V{Y}_d$—subject-figurative core; ${}^{1}X,{}^{2}X,{}^{3}X$—sets of resources, their properties, and actions, which reflects the process of diagnostics in medicine; $R$—a set of relations defined on ${}^{1}X,{}^{2}X,{}^{3}X$; ${}_d{}^{1}V,{}_d{}^{2}V,{}_d{}^{3}V$—sets of visual signs of resources, properties, and actions; ${}_{d}R_v$—a set of role-visual relations defined on one or more of ${}_d{}^{1}V,{}_d{}^{2}V,{}_d{}^{3}V$, subject-figurative schematic statements; ${}_{v}N$—a dictionary of nonterminal symbols;a dictionary of nonterminal symbols; ${}^{p}P$—a set of product rules; E—qualified doctors whose actions are limited by the external environment; $a_d$—an action defined on the set of actions $A_d$ that is an object of norm regulation; $Mod$—a set of modality systems; $C_d$—a set of situations in which the action has not to be applied.

Each concept in the language describing the relations and links (LDRL) of the verbal-sign shell is denoted as $x^n\in X^n$ where $n=0$ for basic concepts and $n \in \left\{1,2,\dots \right\}$ for derivative concepts. A set of concepts of the LDRL is $X^0=X_{\alpha }^0\cup X_{\beta }^0$ where $X_{\alpha }^0$ and $X_{\beta }^0$ are major and auxiliary subsets, respectively. The major subset is defined by three categories: resources, properties, and actions. The auxiliary subset is defined by categories of measures, values, characteristics, parameters, names, states, and evaluations.

Hence, ${}^{1}X^0$—resources, ${}^{2}X^0$—properties, ${}^{3}X^0$—actions, ${}^{4}X^0$—measures, ${}^{5}X^0$—values, ${}^{6}X^0$—states, ${}^{7}X^0$—evaluations and ${}^{8}X^0$—unusual concepts. $X^0={\displaystyle \underset{i}{\overset{}{\cup }}{}_{}{}^{i}X{}^0}$ for $i\in$$\overline{1;8}$. $\varnothing$ = ${\displaystyle \underset{i}{\overset{}{\cup }}\left({}_{}{}^{i}X{}^0{\displaystyle \underset{}{\overset{}{\cap }}{}_{}{}^{j}X{}^0}\right)}$ for $i\ne j$ and $i,j\in$$\overline{1;8}$ (double concepts and homonymity can exist).

Each relation in the LDRL is denoted as $r_{}^0$$\in$${}_{}{}^{}R{}^0$ or $r_{}^n$$\in$${}_{}{}^{}R{}^n$ depending on its notation by a word or a phrase in the language of the professional activity (LPA). $R^0={\displaystyle \underset{}{\overset{}{\cup }}{}_{}{}^{i}R{}^0}$ for $i\in$$\overline{1;7}$: ${}_{}{}^{1}R{}^0$—definitional relations, ${}_{}{}^{2}R{}^0$—comparison, ${}_{}{}^{3}R{}^0$—dimensional, ${}_{}{}^{4}R{}^0$—temporary, ${}_{}{}^{5}R{}^0$—inclusion, ${}_{}{}^{6}R{}^0$—causality, ${}_{}{}^{7}R{}^0$—preference. Further, denote to be by $r_1^0$$\in$${}_{}{}^{2}R{}^0$, ${}_{}{}^{3}R{}^0$, ${}_{}{}^{4}R{}^0$ and to have by $r_2^0$$\in$${}^{1}R^0$, ${}_{}{}^{5}R{}^0$, ${}_{}{}^{6}R{}^0$, ${}_{}{}^{7}R{}^0,$ then we have $\varnothing$ = ${\displaystyle \underset{}{\overset{}{\cup }}\left({}_{}{}^{i}R{}^0{\displaystyle \underset{}{\overset{}{\cap }}{}_{}{}^{j}R{}^0}\right)}$ for $i\ne j$ and $i,j\in$$\overline{1;7}$ in the general case. This makes it possible to single out seven classes of relations in the LPA and to fill in appropriate dictionaries. To specify the structure of signs in LPA, the separation of $R^0$ into subsets $R_{11}^0$, $R_{22}^0,$ …, $R_{23}^0$, $R_{24}^0$, $R_{25}^0$, $R_{45}^0$, $R_{16}^0$, $R_{37}^0$, $R_{66}^0$, $R_{77}^0$ was obtained such that $R^0={\displaystyle \underset{ij}{\overset{}{\cup }}{R}_{ij}^0}$ for $i,j\in \overline{1,7}$ and $\varnothing$ = ${\displaystyle \underset{}{\overset{}{\cup }}\left(R_{ij}^0{\displaystyle \underset{}{\overset{}{\cap }}{R}_{km}^0}\right)}$ for $i\ne k,j\ne m$, $i,k\in$$\overline{1;7}$ and $j,m$$\in$$\overline{1;7}$ in the general case. For example, $R_{11}^0$ = {to be in, near, a part, to have a composition}, $R_{22}^0$ = {to be larger, smaller, equal}, $R_{33}^0$ = {to be simultaneously, earlier}, $R_{31}^0$ = {to have a subject, an object, a tool}, $R_{13}^0$ = {to be a condition, an object, a tool, a location}, $R_{32}^0$ = {to have a duration, a result}. Relations from classes $R_{12}^0$, $R_{21}^0$, $R_{23}^0$, $R_{24}^0$, $R_{25}^0$, $R_{45}^0$, $R_{16}^0$, $R_{37}^0$ include words to be, to have, as well as concepts a parameter, a characteristic, a name, a measure, etc.

The derivative relations $R_{ij}^n$ were also considered. The language of the first level $L_1$($X^0,R_{ij}^0$, ${\mathcal{P}}^1$) = $\left\{r_{i{j}_l}^{\Pi }\right\}$ is a set of signs of derivative relations (DR). It is equal to $R_{ij}^n$. DR are deduced from purpositive relations $r_1^0$ (to have) and $r_2^0$ (to be).

The definition of simple and complex resources, properties, and actions is executed at the second and the third levels, respectively.

The language of the second level $L_2\left(X^0,{}_{}{}^{1}R{}_{ij}^{\Pi },(,),{\mathcal{P}}^2\right)=\left\{{}_{}{}^{i}x{}_k^{}\right\}$ is a set of signs of derivative concepts (DC). It is equal to $X$. DCs are deduced from $X^0,{}_{}{}^{1}R{}_{ij}^n,(,)$.

The language of the third level $L_3\left(X^0,X^{},{}_{}{}^{5}R{}_{ij}^{\Pi },(,),{\mathcal{P}}^3\right)=\left\{{}_{}{}^{i}x{}_k^n\right\}$ is a set of signs of DCs. It is equal to $X^n$. DCs are deduced from $X^0,X^{},{}_{}{}^{5}R{}_{ij}^{\Pi },(,)$ and arranged by the inclusion relation (to be a part of or to include) in a hierarchical structure.

Modeling of spatial and production structures, d-situations (simultaneous execution of a set of actions on the object of diagnosis—diagnostic operations as well as medical operations), and r-situations (simultaneous execution of actions by resources, which include reactions of some systems of the body, its subsystems and their elements to the activity of other systems of the body, its subsystems and their elements, as well as reactions to external influences on the body) is executed at the fourth level of LDRC $L_4\left(X^0,X^n,\left\{{}_{}{}^{f}R{}_{ij}^{\Pi }\right\},(,),{\mathcal{P}}^4\right)=\left\{{}_{}{}^i\pi {}_k^{}\right\}$—is a set of signs of DCs. It is equal to Π. DCs are deduced from $X^0,X^n,{}_{}{}^{3}R{}_{ij}^{\Pi },{}_{}{}^{4}R{}_{ij}^{\Pi },{}_{}{}^{6}R{}_{ij}^{\Pi },(,)$.

Formalisms of languages of the fifth–ninth levels are considered in [28,29]. The CHIDS, its subject-figurative model, architecture, and algorithm of synthesize are presented below.

4.2. Subject-Figurative Model of the Functional Hybrid Intelligent Diagnostic System

The cognitive hybrid intelligent diagnostic system is considered as a functional hybrid intelligent system with modeling a diagnostic problem and is defined on the basis of (1.5) from [17] as a hybrid intelligent system that perceives input diagnostic data, issues output signals (diagnosis) and is in a certain state relevant to the state of the collective diagnostic process:

$$\begin{array}{c}{\alpha }^u(t)=R_{19}^n\left({}^{1}X^0,ME{T}^i\right)\wedge R_{12}^n\left({}^{1}X^0,{}^2\stackrel{⌢}{x}_1^n\right)\wedge R_{12}^n\left({}^{1}X^0,{}^2\stackrel{⌢}{x}_2^n\right)\wedge R_{12}^n\left({}^{1}X^0,{}^2\stackrel{⌢}{s}^n\right)\wedge \\ \wedge {}^{6}R_{22}\left({}^2\stackrel{⌢}{s}^n(t),{}^2\stackrel{⌢}{s}^n(t+1)\right)\wedge {}^{7}R_{22}\left({}^2\stackrel{⌢}{x}_1^n(t),{}^2\stackrel{⌢}{s}^n(t)\right)\wedge {}^{8}R_{22}\left({}_2\stackrel{⌢}{s}^2(t),{}^2\stackrel{⌢}{x}_2^n(t)\right)\wedge \\ \wedge {}^{\psi \phi }\ddot{R}_{11}^k\left({}^{1}X^0,{}^{1}X^n\right)\wedge {}^{9}R_{22}^n\left({}^2\stackrel{⌢}{x}_1^n,{}^{2}X_1^n\right)\wedge {}^{10}R_{22}^n\left({}^{2}X_2^n,{}^2\stackrel{⌢}{x}_2^n\right),\end{array}$$

(5)

where $ME{T}^i$—an integrated method; ${}^2\stackrel{⌢}{x}_1^n$—a vector of the initial data of the DP transmitted to the input of one or more elements of the hybrid ${\alpha }^u$ solving subproblems from the decomposition of the DP; ${}^2\stackrel{⌢}{x}_2^n$—a vector of the output of one or more ${\alpha }^u$—elements (a goal of the DP solving); ${}^2\stackrel{⌢}{s}^n$—a vector of states of the ${\alpha }^u$ formed from the states of “behavioral” elements with analytical, evolutionary, statistical calculations and logical reasoning as well as pseudo-states of elements with neuro-, fuzzy calculations and reasoning based on experience; ${}^{6}R_{22}$, ${}^{7}R_{22}$, ${}^{8}R_{22}$—the hybrid functioning relations specified at adjacent times on sets of state–state, input–state pairs and state–output pairs, respectively; ${}^{1}X_n$—a set of sings of elements; ${}^{\psi \phi }\ddot{R}_k^{11}$—hybrid element φ and ψ integration relations (knowledge of these elements involves in the integration of the type k); ${}^{2}X_1^n$, ${}^{2}X_2^n$—a set of elements values “input” and “output” from ${}^{1}X^n$, respectively; ${}^{9}R_{22}^n$, ${}^{10}R_{22}^n$—relations on the set of pairs of “HIS input–inputs of elements” and on the set of pairs of “outputs of elements–output of HIS”, respectively.

A heterogeneous model field (HMF) of a diagnostic problem in Equation (5) is a set of heterogeneous models reflecting theoretical, professional knowledge and experience of experts (doctors) and technological models that forms subject-figurative representations of experts, a decision-maker, and the object of diagnosis. Each model is implemented as an element of a hybrid. An integrated method in Equation (5) is the result of the process of synthesis of a method of solving the diagnostic problem.

The subject-figurative model of CHIDS is presented at the Figure 2. It is relevant to the conceptual model at Equation (5). In the center of the CHIDS model (oval), there is an arrow forming a role-visual relation “aggregate-action” which emphasizes the activity approach to the representation of the system. Above the resource part of the arrow, there is CHIDS’s property—a system identifier. The triangular part of the arrow displays the content of the role-visual relation “action-property”: “action-time” (to have a start time, to have an end time); “action-name” (to have a name); “action-characteristic” (to have a characteristic); “action-assessment” (to have an assessment). The characteristic pictogram here reveals the methodological aspect of the design of the CHIDS element.

The resource (rectangular) component of the action (arrow) to form the system includes the following role-visual relations:

• “action-object” where the object is HMF;
• “action-subject” where the right role is the resource “team of experts”; “action-object” where the right role of the resource is “a set of programs”;
• “action-result” where the right role is the architecture of the CHIDS that is relevant to the structure of the DP;
• “action-property” performed after a connection between the input of the CHIDS and the input of one of its elements is established. This connection is symbolized by the closed bases of triangles (properties). Each property consists of a role-visual relation “property-resource”: on the left, the “input” property and the resource is CHIDS (rectangle);
• “action-property” (shown at the Figure 2 by ellipsis which number is equal to the HMF dimension) which is performed after a connection between the output of one element of the CHIDS and the input of another is established. Each property is in a role-visual relation “property-resource”: on the left-the “output” property and the resource-element. The similar right one visualizes the statement “the input of the CHIDS element”;
• “action-property” which is performed after the connection between the output of the CHIDS element and its output is established. Each property consists of a role-based visual relation “property-resource”: on the left, the “output” property and the resource (an element). The similar right one visualizes the statement “CHIDS exit”.

Below the resource part of the arrow, there are the properties of the CHIDS which schematize the role-visual relations “aggregate-input”, “aggregate-state”, and “aggregate-output”. The triangular part of the arrow displays the contents of the role-visual relation “action-property”. The characteristic pictogram reveals the methodological aspect of the design of the CHIDS.

4.3. Typical Architecture of the Cognitive Hybrid Intelligent Diagnostic System

The CHIDS architecture is relevant to the structure of the diagnostic problem from the conceptual model at Equation (5) for which it is intended. The typical architecture of the CHIDS (Figure 3) includes:

• user interface;
• functional elements (FE) that solve a set of subproblems of accounting and control as well as diagnostic subproblems solved by experts and the decision-maker;
• technological elements that solve the subproblem of information preprocessing and the problem of forming cognitive images of experts, decision-maker and the diagnostic object;
• the storage of subject-figurative models.

The minimum power of the FE sets that solve the problems of accounting and control is equal to 1. The elements of the CHIDS interact and exchange information including visual information. By transferring private solutions to each other in a graphical, verbal-sign or combined form, the elements of the CHIDS modify the initial version of the problem decomposition, design an integrated solution by running the algorithm for synthesizing the CHIDS in the “Coordinating unit”. The result of its work modifies the composition and connections of the elements of the CHIDS and, if necessary, requests the addition and/or updating of the values of the set of health indicators of the object entered through the user interface. The algorithm can be launched when additional information about the state of health of the object appears entered by the user through the interface. This takes into account the dynamic nature of the DP and the synthesis of the CHIDS which is relevant to the problem at the certain time of its solution. Technological elements that solve the problems of forming subject-figurative models of experts, decision-maker and the diagnostic object are implemented as static production expert systems with the inclusion of pictograms and basic visual symbols in the right parts of the knowledge bases designed similarly to the expert system at [30] (Table 3.1, (3.19)).

All subject-figurative models can be displayed at the user interface. Functional elements that model the solution of diagnostic subproblems by experts and decision-maker can be complex and have the architecture shown at the Figure 4.

4.4. Cognitive Hybrid Intelligent Diagnostic System Synthesis: The Algorithm

The method of solving a diagnostic problem is constructed at the current time of its solution. Input information:

• A decomposition ${\stackrel{⌢}{p}}_D=\langle P^h,{}^{3}R_{88}^n\rangle$ of the DP in the view of oriented graph. Nodes are $p_i^h$, $i=\left[1,N_h\right]$—homogeneous subproblems (functional and technological) and edges are ${}^{3}r_{88}^n(p_i^h,p_j^h)\in {\stackrel{⌢}{p}}_D$ for $i,j=\left[1,N_h\right],$$i\ne j$—relations over HMF;
• HMF M^a and a set of one-to-one correspondence $\left\{{}^{3}r_{88}^n\left(p_{\psi }^h,p_{\phi }^h\right)\leftrightarrow {}^{\psi \phi }\ddot{R}_{11}^k\right\}$ where ${}^{4}R_{88}^n$ is a set of correspondence relations between goals and input data of subproblems from the decomposition of the DP and goals and input data of the DP;
• A set of interpreters I^a of autonomous models $mo{d}_x^a$ and inter-model interfaces ${}^{xy}\zeta$;
• Correspondences of $mo{d}^a\in MO{D}^a$ and interpreters $I^{\eta }\in I^a|\eta =\left\{a,s,e,f,n,g,p\right\}$ where a—analytical methods, s—statistical, e—expert systems, f—fuzzy systems, n—neural networks, g—genetic algorithms, p—CBR-systems;
• A set $S^x\in S,x\in \left[1,N_{ms}\right]$ of valid states $S^x=\left\{s_{\delta }^x\right\},\delta \in \left[1,N_{sx}\right]$ for each model $mo{d}_x^a$ and the order over the $P^h$–$In=\left\{I{n}^1,\dots ,I{n}^{NT}\right\}$ where $I{n}^1=\left\{i{n}_1,\dots ,i{n}_{I{n}^1}\right\}$ and $I{n}^{NT}=\left\{i{n}_1,\dots ,i{n}_{I{n}^{NT}}\right\}$—inferior indexes of subproblems from $P^h$.

Output information is the list Ls^D of integration relations over the models from the HMF ${}^{1}R_{12}\left(mo{d}_x^a,s_{\delta }^x\right)\circ {}^{xy}\ddot{R}_{11}^k\left(mo{d}_x^a,mo{d}_y^a\right)$, $x,y\in \left[1,N_{ms}\right]$ and a set E of matrices “model–model”—$E_j\in E_j^l$, $l\in \left[1,NT\right]$ the strings of which are $mo{d}^a$ in the state $s_{\delta }^x$ solving subproblem $p_{\psi }^h$ and columns are $mo{d}^a$ solving subproblem $p_{\phi }^h$.

Steps of the algorithm:

• The beginning, l = 1;
• Choose from the decomposition $\beta =\left[1,N_h^l\right]$ of ${}^{3}r_{88}^n\left(p_{\psi }^h,p_{\phi }^h\right)$ in which each $p_{\phi }^h$ has the priority from In^l;
• j = 1, E^l =$\varnothing$;
• Choose the next j-th pare from $P_{\beta }^h$;
• Initialize the matrix $E_j^l$ for j-th pair: the columns are models $mo{d}_{\phi }^a$ solving subproblem $p_{\psi }^h$, and rows are models $mo{d}_{\psi }^a$ in the state $s_{\delta }^x$ solving subproblem $p_{\phi }^h$.
• Supplement the $E^l$ by the matrix $E_j^l$;
• $j=N_h+1$? If NO then j = j + 1 and go to the item 4;
• j = 1;
• Choose $E_j^l$ from E^l. Only those elements are activated for which integration relations are specified between models. Using an expert system, the models are evaluated. Estimations of $mo{d}_{\psi }^a$ are entered in the denominators, and estimations of $mo{d}_{\phi }^a$—in the numerators of the elements. For the initial states of the models, select a pair of models with the maximum value of the integrated estimation and add it to the Ls^D;
• $j=N_h+1$? If NO then j = j + 1 and go to the item 9;
• l = NT? If NO then l = l + 1 and go to the item 2, otherwise supplement Ls^D with interpreters and interfaces and form a knowledge base of a functional element that models the decision of the decision-maker subproblem from matrices E, so that it can rebuild the integrated model of the CHIDS depending on the situation and combine both symbolic reasoning and visual.

The coordinating unit (Figure 3) built the integrated model on the basis of the list Ls^D and a set E of matrices “model–model” which are output information of the algorithm proposed above and obtained depending on the current diagnostic situation. The algorithm is initiated every time the diagnostic problem is solved so the integrated model is always relevant to the specificities of the object and process of its diagnostics. Moreover, combining the results of logical-linguistic modeling of the collective decision-making with representations of the object and subjects (models of members of a council solving problem or subproblems) makes the situation, the state of the object and processes of reasoning visible and fixable. Representations reduce the pressure on the doctor because people analyze schemes (images) at a higher speed than verbal-sign information. Implementation of the obtained integrated method in the Cognitive Hybrid Intelligent Diagnostic System within the diagnostics helps its user (a doctor) to highlight only what is important for solving the current diagnostic problem from the whole content.

5. Discussion

Works in the field of theory, methods, models, and information technologies for representation of objects of thinking “diagnostic problem” and its elements are associated with the following names: B.A. Kobrinsky [3,4], L.D. Lendenbratan, N.L. Morina, D.A. Oshanin [25], F. Rassom [6], V.M. Khachumov [2], C. Bou-Crick [5], D. Heller [7], et al. The analysis showed that there is no common understanding of the phenomenon of the image of a state of a human body. At the same time, the safety of the decision depends on the efficiency of the doctor’s reaction to the emerging problematic diagnostic situation.

Works in the field of hybrid intelligent systems are associated with such names as: A.N. Borisov, A.V. Gavrilov, I.A. Kirikov, P.M. Klachek, A.V. Kolesnikov, S.V. Listopad, G.S. Osipov, D.A. Pospelov, S.B. Rumovskaya, G.V. Rybina, S.A. Soldatov, V.B. Tarasov, I.B. Fominykh, N.G. Yarushkina, S. Goonatilake, S. Khebbal, L. Medsker and others. A multi-model semiotic system has been developed [31] in 1983–1986. The concept of hybrid expert systems has then been introduced by D.A. Pospelov and A.N. Borisov [32], and G.V. Rybina developed the concept of integrated expert systems [33]. HIS were announced in 1994–1995 by L. Medsker [34] and, in essence, coincided with the intelligent hybrid systems of S. Goonatilake and S. Khebbal (1992) [35], as well as the hybrid intelligent adaptive systems of N. Kasabov and R. Kozma (1998) [36]. In 1997–1998, hybrid systems with a discrete part based on knowledge [37] and semiotic systems of distributed intelligence [38] were proposed. Within the framework of Kaliningrad School (2001–2007), the informal axiomatic theory of schemes of role conceptual models for the triadic model “resource-property-action” and a problem-structural methodology of functional HIS were developed. In 2009–2012, I.B. Fominykh [1] proposed the formalism of a hybrid intellectual system based on temporal logical models and image engineering. In Kaliningrad School (2011–2019) HISs [39,40,41,42] were used in combination with multi-agent systems to solve a complex transport and logistics problem, with coordination, and functional HIS was used as a support system for making complex diagnostic decisions based on the “heterogeneous problem” model relevant to the practice of consultations in clinical medicine (S.B. Rumovskaya).

In the field of hybrid intelligent systems, the currently working ones must be noted:

• Bauman Moscow State Technical University, Moscow (hybrid intelligent information systems for the analysis of judicial arbitration practice [43] and hybrid intelligent systems for processing big data [44]);
• Immanuel Kant Baltic Federal University, Kaliningrad (hybrid computational intelligence methods [45]);
• Rostov State Transport University, Rostov-on-Don (hybrid intelligent systems [46]);
• Lipetsk State Technical University, Lipetsk (hybrid intelligent information systems for natural language text processing [47]);
• Belarusian State University of Informatics and Radioelectronics, Minsk, Belarus (hybrid semantically compatible intelligent systems [48]);
• University of Electronic Science and Technology of China, Chengdu, China (hybrid intelligent systems for diagnosing cardiovascular diseases [49]), et al.

The advantages of modern research in the direction of hybrid intelligent systems: (1) combining within a single system of various methods of artificial intelligence which allows taking into account the instrumental heterogeneity of practical problems making it possible to compensate for the weaknesses of some methods with the advantages of the others; (2) modeling the knowledge of specialists in various fields allows us to take into account the heterogeneity of the structure of the problem. The problems of modern research in the direction of hybrid intelligent systems: (1) success in solving a problem is made dependent on the quality of the system of sequentially solved service tasks replacing it and the quality of the HMF functionally correlated with them for the synthesis of an integrated method; (2) HIS’s elements are rather rigidly interconnected which makes it difficult to further improve and maintain them up to date. Unlike existing research in the field of HIS, the one presented in this paper ensures the adaptation of the system to changing conditions and the doctor’s representation of the problem by synthesizing each time a new integrating model of the imitation of the natural mechanisms of the human (expert) thinking under medical diagnostic problems. The implementation of the components of the system in the form of separate models of a heterogeneous model field in the long term will allow improving the system by adding new methods and diagnostic steps and removing obsolete ones without significantly reworking the algorithm for synthesizing an integrated diagnostic solution over a heterogeneous model field. So, cognitive hybrid diagnostic systems are superior to world analogues in the field of building hybrid intelligent systems for solving practical problems, in particular in medicine.

The next and last stage of the research is the approbation of the architectural solutions of CHIDS for the problem of diagnosing diseases of the pancreas, in particular acute pancreatitis and cancer, on the materials of the Kaliningrad Regional Clinical Hospital (KRCH). The stage includes the development of a user interface, a heterogeneous model field and the filling of a repository of subject-figurative models. Their implementation is planned using MATLAB-Simulink, Dgraph database, and JavaScript.

The proposed Metalanguage is universal for all branches of medicine. It determines the relevant mapping of the verbal-symbolic and visual-spatial aspects of intellectual activity into the architecture and algorithms of artificial intelligence. There is a high validity in applying the Metalanguage for describing intellectual activity aimed at solving diagnostic problems in different subject areas because components of all its levels are filling according to the type of diagnostic object and the type of its diagnosis. The Metalanguage with minor revisions can even be generalized to the problems of the other type, not diagnostic, depending on the type of object. However, of course these two issues have to be verified at concrete practical tasks. The same applies to the proposed architecture which is based on psychological studies of beginners and experts [16], the results of which are universal for all types of subject areas and problems.

6. Conclusions

The safety of the decisions made depends on the responsiveness of the doctors to the emerging problematic diagnostic situation. According to various estimates [50], approximately 50–60% of Russians have encountered doctors’ mistakes at least once in their lives. On average, in 35% of cases there is a discrepancy between polyclinic and clinical diagnosis, and in 21.6% of cases the diagnosis established during life was incorrect.

In human thinking [51], a huge role is played by schemes of standard situations the use of which significantly speeds up reasoning. Visualization allows the translation of those contents (images, representations and, accordingly, processes with them) into an externally noticeable plan that occurs “in the specialist’s (doctor’s) head”. It makes them visible, fixable, and easily manageable both for himself and for those with whom he interacts. Schematization allows selection from the visualized, “external” content only what is important for solving a diagnostic problem by highlighting through “drawing” only what is needed and important in a specific problematic diagnostic situation, and the rest is omitted. These processes are clear, they are easier to manage and control the influence of some subjective preferences on them.

Cognitive hybrid diagnostic systems will eliminate contradictions between the properties of objective reality, diagnostic processes, and features of the diagnostic object (the patient’s body) and the scientific picture of the world that is a part of the world view of scientists and specialists working in computer science, design, and medicine. The CHIDS architecture is relevant to the composition and structure of the intellect of an expert doctor, reflecting not only the logical-mathematical (left-sided intellect) but also the visual-spatial (right-sided) mechanism of thinking. The metalanguage of the axiomatic theory of role-visual diagnostic models is universal. The study is being conducted with the participation of experts—doctors from the Kaliningrad Regional Clinical Hospital. Presented findings have been aligned with them for further work. The future work (the last stage) is the approbation of all solutions for the concrete diagnostic problem. KRCH has chosen the problem of examination of the pancreas, in particular diagnostics of chronic pancreatitis and cancer. This problem will be analyzed and identified. Its parameters and limitations will be specified. After that we will follow the next sequence of steps:

• Reduction of a problem according to the method and general scheme of the representation of the diagnostic problem decomposition proposed in the [29];
• Developing of the heterogeneous model field on the basis of the obtained decomposition. Every model of the field will be built by means of a certain method of artificial intelligence depending on the type of used calculations. The limit of the study regarding types of calculations: neuro-, fuzzy, reasoning based on experience, analytical, evolutionary, statistical, and logical reasoning;
• Synthesizing of the integrated model by the proposed in the manuscript algorithm. It would have to be rebuilt over the heterogeneous model field depending on the input information about the object every time the object is being diagnosed;
• Initialization of the Cognitive Hybrid Intelligent Diagnostic System for the pancreas;
• Laboratory research of the system and interpretation of the results according to which we could have to return back at the step of “Development of the heterogeneous model field” for revision of models.

Due to the integration of imitation of an internal subject-figurative vision of medical diagnostic problems and the corresponding communication statements of private diagnoses with imitation of the behavior inherent for councils in problem situations, an industrial prototype of the hybrid functional intellectual system with cognitive modeling of a diagnostic problem will allow us to approach the smoothing of conflicting indications of equipment, reduce the number of medical errors, improve the quality and, accordingly, the safety of medical intelligent diagnostic decision support systems.