Temporal Extraction of Complex Medicine by Combining Probabilistic Soft Logic and Textual Feature Feedback

Abstract

In medical texts, temporal information describes events and changes in status, such as medical visits and discharges. According to the semantic features, it is classified into simple time and complex time. The current research on time recognition usually focuses on coarse-grained simple time recognition while ignoring fine-grained complex time. To address this problem, based on the semantic concept of complex time in Clinical Time Ontology, we define seven basic features and eleven extraction rules and propose a complex medical time-extraction method. It combines probabilistic soft logic and textual feature feedback. The framework consists of two parts: (a) text feature recognition based on probabilistic soft logic, which is based on probabilistic soft logic for negative feedback adjustment; (b) complex medical time entity recognition based on text feature feedback, which is based on the text feature recognition model in (a) for positive feedback adjustment. Finally, the effectiveness of our approach is verified in text feature recognition and complex temporal entity recognition experimentally. In the text feature recognition task, our method shows the best F1 improvement of 18.09% on the Irregular Instant Collection type corresponding to utterance $l_{17}$. In the complex medical temporal entity recognition task, the F1 metric improves the most significantly, by 10.42%, on the Irregular Instant Collection type.

1. Introduction

In the medical field, multiple types of temporal information describe the occurrence of and state change in medical events, and it plays an essential role in conducting disease risk prediction [1,2], early detecton of diseases [3], disease trajectory tracking, adverse reaction detection, etc. [4,5]. Most of these temporal information representations are complex times with characteristics such as fuzziness and irregularity [6]. To realize the representation and reasoning of complex time in clinical texts, Hu et al. [7] proposed Clinical Time Ontology (CTO) and defined the semantic concepts of each type of complex time.

Temporal entity recognition originated in the sixth and seventh Message Understanding Conferences (MUC-6 [8] and MUC-7 [9]), mainly focusing on formal texts such as news articles [10,11] and later evolving to other domains [12,13]. The survey shows the importance of medical time entity identification in constructing timelines and clinical decision support systems, etc. [14]. However, medical texts contain many proper names and different habits of medical recorders [15], which results in the inability to directly apply time recognition methods from other fields to the medical area. With the release of shared tasks in medical Natural Language Processing (NLP), researchers have begun to focus on the study of medical temporal recognition [16,17]. The most commonly used temporal recognition methods are rule-based methods [18,19], statistical learning-based methods [20], and deep learning-based methods [21,22]. Rule-based methods perform recognition by manually generalizing the expression rules of temporal entities. Although they have high accuracy, they require much time and effort and have poor portability. Statistical learning-based methods perform recognition by selecting feature templates and sequence annotation sets, but the recognition effect depends more on the annotated corpus. Deep-learning-based methods can automatically learn features from the corpus to achieve entity recognition, which makes it more advantageous than the above two methods. However, these methods mainly target coarse-grained simple moments or periods, such as Date (e.g., 6 April 1994), Duration (e.g., two weeks), and Time (e.g., Thursday evening), with less attention to fine-grained complex time. If they are applied directly to fine-grained complex medical time recognition, their accuracy is low, which limits the utilization of medical information, such as aiding medical decision-making systems.

After analyzing the temporal text in electronic medical record data, we found the reason for the lower effectiveness of the existing models in identifying fine-grained complex temporal. The main reason is that there are more overlapping text features between different temporal types, and the text descriptions of some temporal types face the problem of a large span. The above can make it more challenging to identify complex temporal entities correctly. Specific examples are shown in Figure 1 and Figure 2.

In Figure 1, “2007年至2008年 (from 2007 to 2008)” is a text description of Absolute Interval (AIST), and “2007年至2008年初 (from 2007 to early 2008)” is a time description of a Fuzzy Interval (FITL). Since there are only weak differences in text composition between the above two time types, and the overlapping features are relatively large, the existing models often face the problem of type recognition errors.

In Figure 2, “2000年8月，2005年1月19日，2009年12月23日 (in August 2000, 19 January 2005, and 23 December 2009)” is a time description of Irregular Instant Collection (IISTC), which has a large span. Therefore, the existing models often identify four entities of Absolute Instants (AISTs).

To address the above issues, we develop specific rules for complex medical time and introduce them into the deep learning model through probabilistic soft logic (PSL) and text feedback mechanisms to extract complex medical time. After analyzing time text in the electronic medical record and the concept of complex time semantics in the CTO, we define seven basic features and eleven extraction rules and propose a complex medical time extraction method combining probabilistic soft logic and textual feature feedback (PSLF-CTR). The extraction method consists of two parts: (a) PSL-based Text Feature Recognition (PSL-TFR), which is based on probabilistic soft logic for negative feedback adjustment; (b) entity recognition, which is based on the trained text features in (a) for positive feedback adjustment.

In summary, our main contributions in this paper are as follows:

• We realize the atomization and regularization of complex medical time texts. We split the features of medical time and propose seven basic features after analyzing the text of medical time. Then, exploring the textual concept of complex medical time in the CTO, we combine the basic features and first-order predicates and propose eleven extraction rules.
• We apply the PSL to text feature recognition. The PSL, a first-order logical predicate probabilistic inference framework, is combined with rules defining the correspondence of each time type to obtain the satisfaction distance corresponding to eleven types of time. The distance is used as a feedback factor in the model to improve the effectiveness of the text-feature-recognition model.
• We add a text feedback mechanism to the entity recognition model and validate the results of the entity recognition model on the trained text feature model. A positive feedback factor is added based on the validation results, improving the complex time entity recognition model.

2. Related Work

2.1. CTO

The CTO defines eleven types of time in the medical domain, including simple time and complex time [7]. Among them, time with precision and absoluteness is a simple time, such as Absolute Instant, Absolute Interval, and time with characteristics such as ambiguity and repetition is a complex time. Of the eleven types of time in CTO, the remaining nine time types other than the two simple times are complex times, and the specific classification is shown in

Table 1. Time types in CTO and their examples.

Syntactic Forms	Semantic Concepts	Acronyms	Examples	Eglish Examples
Instant	Absolute	AIST	2021年12月4日	4 December 2021
	Relative	RIST	昨天	Yesterday
	Fuzzy	FIST	6点左右	Around 6:00 p.m.
Interval	Absolute	AITL	2021年11月至12月	November to December 2021
	Relative	RITL	昨天早6点到9点	Yesterday from 6:00 a.m. to 9:00 a.m.
	Fuzzy	FITL	大概下午5点开始8点结束	About 5:00 p.m. start 8:00 p.m. end
	Incomplete	IITL	昨天结束服药	Finished taking medication yesterday
Collection	Periodic Instant	PISTC	每天下午三点	Daily at 3:00 p.m.
	Irregular Instant	IISTC	10点、 11点、 20点分别有抽搐发作	Seizures at 10:00, 11:00 and 20:00 o’clock
	Periodic Interval Collection	PITLC	每年3至5月	March to May every year
	Irregular Interval	IITLC	在昨天下午9点至12点、今天早上5点至6点、下午4点至11点分别进行抢救	Resuscitation took place yesterday from 9:00 pm to 12:00 pm, this morning from 5:00 a.m. to 6:00 a.m., and from 4:00 p.m. to 11:00 p.m.

.

2.2. Time Entity Recognition

Temporal entity recognition is one of the essential tasks in medical named entity recognition [7], and scholars at home and abroad have conducted a series of studies on it. For English texts, Chang et al. [23] proposed SUTime, a time tagger for identifying and normalizing time expressions in English texts. It obtains time expressions using combining and filtering strings. Strotgen et al. [24] proposed HeidelTime, a processing system with multilingual features. This system introduces lexical annotation and manual setting rules for recognizing and normalizing temporal expressions. These approaches are temporal annotators implemented by designing rules using a fixed method that requires more labor, so subsequent researchers started to focus on ways to extend or automatically be able to generate rules. Zhong et al. [25] implement the lightweight time tagger SynTime. It is mainly based on boundary expansion and uses heuristics to design generic rules. Although it saves labor costs somewhat, it assumes that the words are all correctly labeled, which would otherwise significantly affect its results. And then, with the advent of machine learning, its models started to be applied in this field. Moharasar et al. [26] proposed a semi-supervised framework for detecting temporal expressions in clinical texts to enhance the accuracy of temporal entity recognition through a large number of unannotated texts. Ding et al. [27] propose a pattern-based approach PTime, which automatically generates and selects patterns for identifying temporal expressions. MacAvaney et al. [28] proposed conditional random field and decision tree integration, which uses lexical, syntactic, semantic, distributional, and rule-based feature approaches to implement a cross-domain clinical temporal-information-extraction framework. With the development of deep learning, researchers have started to apply it to the field. Hossain et al. [29] proposed a temporal information extraction system using long short-term memory (LSTM) recurrent neural networks (RNN) and word embedding, which achieved good results on the TempEval-2 dataset. Patra et al. [30] proposed a model that combines rules and neural models that identify relevant date-time entities and their associated negation constraints for a specific task. Li et al. [31] proposed a method for extracting clinical event expressions and corresponding temporal information based on the architecture of recurrent neural networks. These methods are relatively mature based on English text, but because of the grammatical differences between Chinese and English [32], they do not work well in Chinese text recognition.

For Chinese text, Lin et al. [33] proposed a recognition method based on regular expressions. Still, developing rules is time-consuming and labor-intensive and has poor domain adaptation. Later, Wu et al. [34] proposed combining conditional random fields with a temporal lexicon for temporal expression recognition. However, it relies on a temporal trigger lexicon and is less effective for time recognition without trigger words. With the emergence of deep learning methods, Liu et al. [35] used a BiLSTM-CRF approach to extract temporal information in geological texts. Kai et al. [36] used the BERT-BiLSTM-CRF model to extract temporal information from labeled media messages. These two extraction methods regarding geological time have significant strengths in their fields and strong domain specificity. Zhu et al. [37] proposed a Chinese temporal expression recognition method based on the BERT-FLAT-CRF model. It uses BERT to enhance word vector expressions, FLAT to fuse temporal lexical features, and CRF to extract optimal sequence annotation. However, these temporal entity recognition methods focus more on coarse-grained simple time and ignore fine-grained complex time. Fine-grained complex temporal descriptive text features are difficult to distinguish, so recognizing them is challenging. Therefore, we introduce a logical framework to improve the recognition of text features.

Probabilistic Soft Logic (PSL) [38] is a probabilistic reasoning framework that uses first-order logic to represent complex relationships succinctly and uses soft truth values between [0, 1] as operation values. Thus, it can capture the uncertain rows and incompletion inherent in real-world knowledge [39]. In recent years, it has been applied to various fields such as sentiment classification [39], collective classification [40], embedding uncertain knowledge graphs [41], and joint estimation of user and publisher feasibility in fake news detection [42].

Therefore, we introduce PSL, a first-order logic framework for probabilistic reasoning, into the deep learning approach. We embed the probability calculation results into a deep learning model through a text feedback mechanism, which is used to better discriminate text features of complex medical temporal and achieve complex temporal recognition.

3. Methodology

Based on the CTO, we define seven kinds of atomic statements and eleven kinds of extraction rules $r_1$∼$r_{11}$ for complex medical time, the detailed definitions of which are shown in Section 3.1. By combining them with PSL, we realize the integration of logic rules in entity recognition tasks to further improve the accuracy of complex time recognition. Therefore, we design a complex medical time extraction with PSL and text feature feedback, whose overall framework is shown in Figure 3. The model mainly consists of two parts: text feature recognition and entity recognition. Text feature recognition (PSL-TFR) adjusts the model with negative feedback by using the calculation results of the PSL program as a negative feedback factor. The entity recognition (PSLF-CTR) adjusts itself with positive feedback by using the validation results of the text feature recognition model as a positive feedback factor. The specific process is as follows.

(1) Training of the PSL-TFR model: The processed electronic medical record data are input into the text feature recognition model. The obtained results are fed into a probabilistic soft logic program for calculation, in which the PSL approach is introduced to calculate the corresponding satisfaction distance $d_r$ according to the rules $r_1$∼$r_{11}$. This $d_r$ is used as a feedback factor to train the PSL-TFR model with negative feedback, thus improving the model’s ability to recognize temporal text features.

(2) Training of the PSLF-CTR model: The electronic medical record information is input into the entity recognition model. Then, the obtained text information is input into the trained PSL-TFR model to obtain the prediction results of the complex time text feature. The data with labeling results are consistent with the prediction results identified for positive feedback training, improving the entity recognition capability.

The following section describes the details of the calculation of $d_r$ and the feedback regulation.

3.1. Basic Extraction Rules for Complex Medical Time

Textual descriptions of time usually have multiple textual features, so there are still some challenges in determining the time type corresponding to time text. Based on the definitions of each type of time in terms of semantics, we extract seven basic semantic features and design eighteen atomic statements, as shown in (1)–(18). Among them, (1)–(7) are used for the basic text features, and (8)–(18) describe the conclusion.

For the “relativity” of time, if there are modifiers with relative meaning in the sentence, such as “之前(before)” or “之后(after)”, then the time expressed in the sentence is “relative”. Let “is relative ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_1=isrelative\left(T\right)$$

(1)

For the “ fuzziness” of time, if there are modifiers with fuzzy meaning in the sentence, such as “大约(about)” or “大概(around)”, then the time expressed in the sentence is “fuzzy”. Let “is fuzzy ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_2=isfuzzy\left(T\right)$$

(2)

For the “periodicity” of time, if there are modifiers with periodic meaning in the sentence, such as “每天(every day)” or “每周(every week)”, then the time expressed in the sentence is “periodic”. Let “is periodic ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_3=isperiodic\left(T\right)$$

(3)

For the “incompleteness” of time, if there is a modifier with incomplete meaning in the sentence, such as “时间不详(time unknown)”, then the time expressed in the sentence is “incomplete”. Let “is incomplete ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_4=isincomplete\left(T\right)$$

(4)

The “continuity” of time often exists in the form of a period. If there is a modifier in the sentence that has the meaning of a period, such as “至(to)” or “持续(lasting)”, then the time expressed in the sentence is “interval”. Let “is an interval ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_5=isaninterval\left(T\right)$$

(5)

The “non-continuity” of time often exists in the form of moments. Since its textual features are not obvious and there is mutual exclusivity between moments and periods, it can be expressed as the negation of the predicate “is an interval ()”. Let “is instant()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_6=isinstant\left(T\right)=\neg l_5$$

(6)

For the “discrete” nature of time, if there are modifiers with multiple meanings in the sentence, such as “第一次(first time)” or “多次(many times)”, then the time expressed in the sentence is “multiple”. Let “is multiple ()” be the predicate and T be the time text; then, the atomic statement can be expressed as follows:

$$l_7=ismultiple\left(T\right)$$

(7)

Since logic rules often consist of $head$ and $body$, the part $head$ is the conclusion that determines the type of time. It needs to be expressed using atomic statements. The atomic statements for describing the conclusion are defined as follows:

$$l_8=AbsoluteInstant\left(T\right)$$

(8)

$$l_9=RelativeInstant\left(T\right)$$

(9)

$$l_{10}=FuzzyInstant\left(T\right)$$

(10)

$$l_{11}=AbsoluteInterval\left(T\right)$$

(11)

$$l_{12}=ReleInterval\left(T\right)$$

(12)

$$l_{13}=FuzzyInterval\left(T\right)$$

(13)

$$l_{14}=IncompleteInterval\left(T\right)$$

(14)

$$l_{15}=PeriodicInstantCollection\left(T\right)$$

(15)

$$l_{16}=PeriodicIntervalCollection\left(T\right)$$

(16)

$$l_{17}=IrregularInstantCollection\left(T\right)$$

(17)

$$l_{18}=IrregularIntervalCollection\left(T\right)$$

(18)

Let $R_l$ denote the set of original statements. According to the definition of time types, different time types can be defined logically using the atomic statement $l_i$ and the analytic (∨), combined (∧), and negation (¬) symbols. The definitions are shown below.

Regarding the judgment of AIST, the temporal text should satisfy “is instant(T)”, “is not relative(T)”, “is not fuzzy(T)”, and “is not multiple(T)”. Thus, the logical rule can be defined as follows:

$$r_1:l_6\wedge \left(\neg l_1\right)\wedge \left(\neg l_2\right)\wedge \left(\neg l_7\right)\to l_8$$

(19)

Regarding the judgment of RIST, the temporal text should satisfy “is instant(T)”, “is relative(T)”, “is not fuzzy(T)”, and “is not multiple(T)”. Thus, the logic rule can be defined as follows:

$$r_2:l_6\wedge l_1\wedge \left(\neg l_2\right)\wedge \left(\neg l_7\right)\to l_9$$

(20)

Regarding the judgment of FIST, the temporal text should satisfy “is instant(T)”, “is fuzzy(T)”, and “is not multiple(T)”. Thus the logic rule can be defined as follows:

$$r_3:l_6\wedge l_2\wedge \left(\neg l_7\right)\to l_{10}$$

(21)

Regarding the judgment of AITL, the temporal text should satisfy “is an interval(T)”, “is not relative(T)”, “is not fuzzy(T)”, “is not incomplete(T)”, and “is not multiple(T)”. Thus the logic rule can be defined as follows:

$$r_4:l_5\wedge \left(\neg l_1\right)\wedge \left(\neg l_2\right)\wedge \left(\neg l_4\right)\wedge \left(\neg l_7\right)\to l_{11}$$

(22)

Regarding the judgment of RITL, the temporal text should satisfy “is an interval(T)”, “is relative(T)”, “is not fuzzy(T)”, and “is not incomplete(T)” and “is not multiple(T)”. Thus the logic rule can be defined as follows:

$$r_5:l_5\wedge l_1\wedge \left(\neg l_2\right)\wedge \left(\neg l_4\right)\wedge \left(\neg l_7\right)\to l_{12}$$

(23)

Regarding the judgment of FITL, the temporal text should satisfy “is an interval(T)”, “is relative(T)”, “is fuzzy(T)”, “is not incomplete(T)”, and “is not multiple(T)”. Thus the logic rule can be defined as follows:

$$r_6:l_5\wedge l_2\wedge \left(\neg l_4\right)\wedge \left(\neg l_7\right)\to l_{13}$$

(24)

Regarding the judgment of IITL, the temporal text should satisfy “is an interval(T)”, “is incomplete(T)”, and “is not multiple(T)”. Therefore the logic rule can be defined as follows:

$$r_7:l_5\wedge l_4\wedge \left(\neg l_7\right)\to l_{14}$$

(25)

Regarding the judgment of PISTC, the temporal text should satisfy “is instant(T)”, “is multiple(T)”, and “is periodicity(T)”. Therefore the logic rule can be defined as Equation (26), and by the same token, the logic rule of PITLC can be defined as shown in (27).

$$r_8:l_7\wedge l_6\wedge l_3\to l_{15}$$

(26)

$$r_9:l_7\wedge l_5\wedge l_3\to l_{16}$$

(27)

Regarding the judgment of IISTC, the temporal text should satisfy “is instant(T)”, “is multiple(T)”, and “is not periodicity(T)”. Thus, the logic rule is shown in Equation (28) and, by the same token, the IITLC are defined as shown in Equation (29).

$$r_{10}:l_7\wedge l_6\wedge \left(\neg l_3\right)\to l_{17}$$

(28)

$$r_{11}:l_7\wedge l_5\wedge \left(\neg l_3\right)\to l_{18}$$

(29)

3.2. Feedback Factor Calculation Based on Probabilistic Soft Logic

Probabilistic soft logic is a framework based on first-order logic rules for probabilistic reasoning. The difference is that it uses the [0, 1] interval of soft truth values of random variables, not only the extreme values 0 and 1 [39]. The following will introduce the relevant concepts of PSL:

Definition 1.

Given a set of atomic statements $\tilde{l}$, $I\left(l\right)$ denotes the soft truth value that satisfies the atomic statement l.

Definition 2.

A rule $\tilde{r}$ usually consists of a disjunctive clause for an atomic statement or a negative atomic statement, for example ${\eta }_r:B_1\wedge B_2\wedge \cdots \wedge B_m\to H_1\wedge H_2\wedge \cdots \wedge H_n$, where $B_1\wedge B_2\wedge \cdots \wedge B_m$ is the body $r_{body}$ of the rule, $H_1\wedge H_2\wedge \cdots \wedge H_n$ is the head $r_{head}$ of the rule, and ${\eta }_r\in [0,1]$ is the weight of the rule r, indicating the prior confidence of the rule.

Definition 3.

A specific atomic statement l and a specific rule r are instantiations of $\tilde{l}$ and $\tilde{r}$, such as $l_{10}$, $r_3:l_6\wedge l_2\wedge \left(\neg l_7\right)\to l_{10}$.

Definition 4.

Define the computation of the base logic operator in PSL according to the $Lukasiewiczt$ equation:

$$I(l_1\vee l_2)=min\left\{I\left(l_1\right)+I\left(l_2\right),1\right\}$$

(30)

$$I(l_1\wedge l_2)=max\left\{I\left(l_1\right)+I\left(l_2\right)-1,0\right\}$$

(31)

$$I\left(\neg l_1\right)=1-I\left(l_1\right)$$

(32)

$$I\left(r_{body}\to r_{head}\right)=I\left(\neg r_{body}\vee r_{head}\right)$$

(33)

Definition 5.

According to the interpretation I, $d_r\left(I\right)$ denotes the satisfaction distance of rule r. The calculation is as follows.

$$d_r\left(I\right)=max\left\{I\left(r_{body}\right)-I\left(r_{head}\right),0\right\}$$

(34)

Definitions 1–5 describe how PSL calculates the soft truth values and satisfiability distances of compound rules in detail. PSL provides theoretical support for calculating the satisfiability probabilities of compound logic rules.

Combine the probability that each atomic statement in $l_1$∼$l_{18}$ is satisfied and its corresponding rule r. After the logical operation defined in PSL, the distance of satisfaction of this time atomic statement, the formula is shown in (35). If $d_r$ > 0, the model predictions are contradictory, and the model needs to be adjusted by negative feedback, using $d_r$ as a feedback factor to amplify its loss.

$$d_r=max\left\{I\left(r_{c_{body}}\right)-I\left(r_{c_{head}}\right),0\right\}=max\left\{I\left(r_{c_{body}}\right)-I\left(r_{correct}\right),0\right\}$$

(35)

3.3. Text Feature Feedback Mechanism

3.3.1. Negative Feedback Mechanism

The PSLF-CTR model incorporates the logical rules related to temporal text representation into the machine learning model. Based on the BERT-TextCNN model, we designed the PSL-TFR model to improve the model’s accuracy in recognizing each type of temporal text feature. Figure 4 shows its structure.

The training process of PSL-TFR adds a negative feedback adjustment mechanism to the Bert-TextCNN model. The set of probabilities $P_{set}=\left\{P\left(l_j\right)\right\}$ can be obtained after the Softmax layer that the text feature satisfies for the atomic utterance $l_1$∼$l_{18}$. The formula is shown in (36).

$$P\left(l_j\right)=\frac{e^{l_j}}{{\sum }_i^m{e}^{l_j}}$$

(36)

where m is the number of atomic statements and takes the value of eighteen. The obtained $P_{set}$ and the corresponding $r_c$ to the actual type are input to the PSL calculation program, which in turn yields each atomic statement $d_r$. If $d_r$ > 0, the model prediction results are contradictory, and the model needs to be adjusted by negative feedback, using $d_r$ as a feedback factor to expand its loss. Since the text feature recognition method incorporates logic rules for negative feedback adjustment, the loss function of the new model is based on the Bert-TextCNN model with the addition of the feedback factor $d_r$. The calculation method is shown in (37):

$$\mathcal{L}=-\left[y\ast log\widehat{y}+\left(1-y\right)\ast log\left(1-\widehat{y}\right)\right]+\lambda \ast d_r$$

(37)

where $\lambda$ is the feedback coefficient, taking values in the range (0, 5], indicating the weight of the negative feedback factor.

3.3.2. Positive Feedback Mechanism

Conventional named-entity-recognition models usually encode the text first and then tag the text using CRF methods. Due to the excellent encoding capability of the BERT model, the vectorized representation of text is achieved using the BERT model. In addition, considering the bidirectional semantic features of character text, the BiLSTM model is used to learn the semantic information embedded in the text. Finally, the CRF method is used to label all characters. The structure of PSLF-CTR, an entity-recognition model based on text feature feedback, is shown in Figure 5.

The model training process is based on the Bert-BiLSTM-CRF model, adding a positive-feedback adjustment mechanism. Let the model have the temporal type $Typ{e}_i$ based on the text and decode the label information of the input sentence after the CRF model. The probability is obtained that the input character x is labeled $Typ{e}_i$, and its formula is shown in (38):

$$P\left(Type|x\right)=\frac{1}{Z\left(x\right)}exp\left(h\left(Typ{e}_1|x\right)+\sum _{k=1}^{n-1}\left[g\left(Typ{e}_k,Typ{e}_{k+1}\right)+h\left(Typ{e}_{k+1}|x\right)\right]\right)$$

(38)

where $Z\left(x\right)$ is the normalization factor, h is the emission matrix, and $g\left(Typ{e}_k,Typ{e}_{k+1}\right)$ denotes the transfer matrix of adjacent positions. The text is input to the PSL-TFR model, and then the satisfaction of the text for complex temporal features is obtained. If the labeling result agrees with the feature recognition result, the model is rewarded with feedback, and the loss is reduced with $\alpha$ as the feedback factor. The loss function of the BERT-BiLSTM-CRF model is shown in (39). After fusing the temporal text features, the model’s loss function is shown in (40).

$${\mathcal{L}}_{original}=-log\left(P\left(Typ{e}_1,Typ{e}_2,\dots ,Typ{e}_n\mid x\right)\right)$$

(39)

$$\mathcal{L}={\mathcal{L}}_{original}-\alpha$$

(40)

4. Experimental Results and Discussion

4.1. Datasets

In this paper, the model is labeled and trained on a private dataset and validated on a public dataset. The private dataset is derived from the clinical desensitized data of the partner hospital. The processed sample is given in the link https://github.com/wangdaven/EMR.git (accessed on 27 October 2022), because the hospital requires it not to be made public. The public dataset is derived from The China Conference on Knowledge Graph and Semantic Computing (CCKS) 2020 data. Since both datasets do not contain complex temporal-type annotations, we invited three annotators to manually evaluate the model-identification results. For the annotation, we annotated 600 private EHRs and 300 open-sourced EHRs in CCKS using the BIO annotation method. On average, each private EHR contains 604 characters, and each public EHR includes 441 characters. The number of each time-type label in the data in both datasets is shown in Figure 6.

The BIO annotation method is to add a label to each word in the sentence. This label consists of two parts: (a) the location of the entity to which the word belongs, where B indicates that the word is the first word of the entity, I indicates that the word is the middle word of the entity, and O indicates that it is not an entity; (b) the entity type corresponding to the word, such as the entity types AITL and PITLC.

4.2. Experimental Settings and Metrics

In this paper, we use the large-scale Chinese word vector data open-sourced by Tencent AI-LAb Lab in 2018 with a dimension of 200, so we set the Embedding-Dim of all models to 200 and the Optimizer to Adam. All experiments were run on a machine equipped with a Tesla P-100 PCIE graphics card and an Intel(R) Xeon(R) CPU E5-2600 v4 @ 2.00GHz model CPU. Through several experiments, the results achieved by the following parameter settings were found to be better under the available computing resources, as shown in

Table 2. Experimental parameter setting.

Parameter	Value
Max Length	256
Learning Rate	0.001
batch-size	32
dropout	0.1
$\lambda$	1
$\alpha$	0.15

, where the effects of the values of the negative feedback coefficient $\lambda$ in Equation (37) and the positive feedback factor $\alpha$ in Equation (40) on model F1 are shown in Figure 7.

Named entity identification includes both entity boundaries and entity types, so “Exact-match Evaluation” is used to evaluate the validity of the model [43]. This method recognizes an entity as correct only if both entity boundaries and types are true [44]. It involves three metrics: precision (P), recall (R), and F1 and is calculated as shown in (41)–(43).

$$P=TP/(TP+FP)$$

(41)

$$R=TP/(TP+FN)$$

(42)

$$F1=P\times R\times 2/(P+R)$$

(43)

True Positive (TP): The number of entities that are predicted to be positive samples and are actually positive samples.

True Negative (TN): The number of entities that are predicted to be negative samples and are actually negative samples.

False Positive (FP): The number of entities that are predicted to be positive samples but are actually negative samples.

False Negative (FN): The number of entities that are predicted to be negative samples but are actually positive samples.

Precision (P) represents the accuracy of the prediction in the positive sample results; a larger value indicates that more of the predicted samples are correct. Recall (R) represents the probability of predicting a positive sample among the actual positive samples; a larger value indicates that more of the actual positive samples are predicted. The F1 metric is the average of precision and recall, and a larger value indicates better model quality.

4.3. Results and Analyses

4.3.1. Comparison Experiments

PSLF-CTR performs positive feedback conditioning on itself, which is based on the recognition results of the PSL-TFR model, and uses the PSL calculation results as its feedback factor during the training of the PSL-TFR model. We designed comparison experiments between PSLF-CTR and BERT+FLAT+CRF and BERT+BiLSTM+CRF models in a private dataset to evaluate the competitiveness of the PSLF-CTR model in the temporal entity recognition task.

(1) BERT+BiLSTM+CRF model is the baseline model of this paper. It is a BiLSTM+CRF sequence annotation model using Google’s pre-trained BERT for word embedding.

(2) The BERT+FLAT+CRF model uses BERT to boost word vector expressions and FLAT to fuse temporal lexical features, and it finally extracts the optimal temporal expressions by CRF.

We explore the impact of complex temporal text features by comparing the three experiments on complex temporal entity recognition. Finally, we evaluate the ability that uses the PSLF-CTR model to recognize complex temporal entities on a publicly available dataset. The results are shown in

Table 3. The result of private dataset time recognition.

	BERT+BiLSTM+CRF			BERT+FLAT+CRF			PSLF-CTR
TYPE	P	R	F1	P	R	F1	P	R	F1
AIST	92.88	95.83	94.33	93.81	96.54	95.15	93.84 (0.96)	95.81 (−0.02)	94.82 (0.49)
RIST	85.13	85.67	85.40	85.53	86.90	86.21	89.14 (4.01)	89.14 (3.47)	89.14 (3.74)
FIST	81.86	77.67	79.71	80.65	79.91	80.28	81.17 (−0.69)	82.65 (4.98)	81.90 (2.19)
AITL	86.40	85.71	86.06	85.83	86.51	86.17	88.00 (1.60)	87.30 (1.59)	87.65 (1.59)
RITL	75.89	79.55	77.68	76.83	80.11	78.43	78.75 (2.86)	81.97 (2.42)	80.33 (2.62)
FITL	68.60	81.38	74.45	69.19	81.51	74.84	76.92 (8.32)	82.19 (0.81)	79.47 (5.02)
IITL	78.57	91.67	84.62	77.27	91.89	83.95	79.76 (1.19)	90.54 (−1.13)	84.81 (0.19)
PISTC	60.33	68.46	64.14	62.13	48.98	48.98	64.41 (4.08)	70.95 (2.49)	67.52 (3.38)
IISTC	47.37	69.23	56.25	52.94	69.23	60.00	64.29 (16.92)	69.23 (0.00)	66.67 (10.42)
PITLC	44.44	52.17	48.00	51.85	60.87	56.00	53.85 (9.41)	60.87 (8.70)	57.14 (9.14)
IITLC	71.43	71.43	71.43	77.78	77.78	77.78	77.78 (6.35)	77.78 (6.35)	77.78 (6.35)

and

Table 4. The results of manual evaluation of the CCKS dataset.

	BERT+BiLSTM+CRF			BERT+FLAT+CRF			PSLF-CTR
TYPE	P	R	F1	P	R	F1	P	R	F1
AIST	93.92	94.13	94.02	94.59	94.81	94.70	94.52 (0.60)	93.45 (−0.68)	93.98 (−0.04)
RIST	91.89	92.73	92.31	92.17	92.73	92.45	91.34 (−0.55)	92.73 (0.00)	92.03 (−0.28)
FIST	85.39	79.17	82.16	87.36	79.17	83.06	87.64 (2.25)	81.25 (2.08)	84.32 (2.16)
AITL	84.62	55.00	66.67	80.00	60.00	68.57	92.31 (7.69)	60.00 (5.00)	72.73 (6.06)
RITL	68.38	65.03	66.67	75.76	69.93	72.73	76.15 (7.77)	69.23 (4.20)	72.53 (5.86)
FITL	72.88	79.63	76.11	72.41	77.78	77.19	73.77 (0.89)	83.33 (3.70)	78.26 (2.15)
IITL	85.00	85.00	85.00	84.44	95.00	89.41	86.05 (1.05)	92.50 (7.5)	89.16 (4.16)
PISTC	45.71	52.17	48.73	46.15	52.17	48.98	46.60 (0.89)	52.17 (0.00)	49.23 (0.50)
IISTC	79.17	95.00	86.36	82.61	95.00	88.37	84.44 (5.27)	95.00 (0.00)	89.41 (3.05)
PITLC	50.00	50.00	50.00	50.00	50.00	50.00	50.00 (0.00)	50.00 (0.00)	50.00 (0.00)
IITLC	36.36	70.00	47.86	38.46	71.43	50.00	41.67 (5.31)	71.43 (1.43)	52.63 (4.77)

.

Table 3 shows that overall, the PSLF-CTR model had better recognition than the baseline and BERT+FLAT+CRF models. It had the highest F1 in each type of time recognition except for the AIST. For the recognition of AIST, the BERT+FLAT+CRF model performed the best, and the PSLF-CTR model had less than 1% improvement in F1 over the baseline model. The analysis of the samples shows that the main reasons are the large sample size, the obvious text features, and the short text.

For the recognition of complex times such as FIST, FITL, PISTC, IISTC, PITLC, and IITLC, the recognition effect of the PSLF-CTR model was improved more obviously, especially for the recognition of IISTC, the F1 improvement reached 10.42%. The baseline model was less effective in recognizing these six types of time, with F1 below 75% and even a PITLC below 50%. After analyzing their texts, we found the reason is that these six types of time all involve more complex temporal text features and fewer occurrences of IISTC and PITLC entities. Although the recognition effect of the BERT-FLAT-CRF model was better than that of the baseline model, there was a slight improvement compared to the baseline model for only individual lexical differences (e.g., FITL). The reason for the better performance of the PSLF-CTR model in combination with

Table 5. L-atom statements in private datasets predict the results of controlled experiments.

	BERT+textCNN			PSL-CTR
	P	R	F1	P	R	F1
$l_1$	99.81	96.89	98.33	99.34 (−0.47)	94.23 (−2.66)	96.72 (−1.61)
$l_2$	97.42	86.64	91.72	97.18 (−0.24)	87.22 (0.58)	91.93 (0.21)
$l_3$	96.97	91.10	94.65	97.75 (0.78)	89.21 (−1.89)	93.29 (−1.36)
$l_4$	78.79	55.32	65.00	80.25 (1.46)	61.41 (6.093)	69.58 (4.58)
$l_5$	96.64	93.98	95.29	98.73 (2.08)	92.38 (−1.60)	95.45 (0.16)
$l_6$	97.60	92.64	95.06	97.59 (−0.01)	92.87 (0.23)	95.17 (0.11)
$l_7$	96.84	64.04	77.09	92.97 (−3.87)	65.58 (1.54)	76.91 (−0.18)

is that the PSL-TFR model is sufficient to obtain a high accuracy rate in predicting the atomic statement $l{}_2$ (“$isfuzzy\left(T\right)$”) and the FIST, and FITL corresponds to the atomic statement $l{}_{10}$, $l{}_{13}$, whose logic rule is $r_3:l_6\wedge l_2\wedge \left(\neg l_7\right)\to l_{10}$, $r_6:l_5\wedge l_2\wedge \left(\neg l_4\right)\wedge \left(\neg l_7\right)\to l_{13}$, so the PSLF-CTR model can receive good feedback from PSL-TFR during the training process, thus improving the accuracy rate of identifying this type of entity. For PISTC, IISTC, PITLC, and IITLC, since the PSL-TFR model also has good accuracy in identifying atomic statements $l_7$ (“$ismultiple\left(T\right)$”) and $l_3$ (“$isperiodic\left(T\right)$”), the combination of the corresponding rules $r_8:l_7\wedge l_6\wedge l_3\to l_{15}$, $r_9:l_7\wedge l_5\wedge l_3\to l_{16}$, $r_{10}:l_7\wedge l_6\wedge \left(\neg l_3\right)\to l_{17}$ and $r_{11}:l_7\wedge l_5\wedge \left(\neg l_3\right)\to l_{18}$, etc. makes the PSLF-CTR model perform better than the original model in identifying complex time-type entities.

After adequately training PSLF-CTR models, we randomly selected 100 medical record data from the 300 in the public dataset for evaluation in this paper. The recognition results are shown in Table 4. For the AIST and RIST, the identification results of all three models were better, reaching more than 90%. Analyzing the data shows that the reason is a large number of samples and more obvious features. For other types of time, although the BERT-FLAT-CRF model was better than the baseline model for most of the time types, it even had the highest F1 value for RITL and IITL. However, it was less effective than the PSLF-CTR model in recognizing complex temporal entities with longer text lengths and larger feature spans (e.g., IITLC and IISTC). Among them, the PITLC had a special recognition effect, marked as 50% in all three models. The analysis shows that the reason is that there are fewer temporal entities of this type, so the experimental results obtained are subject to chance.

From Table 3 and Table 4, PSLF-CTR significantly improves recognition of complex temporal entities compared to the baseline model BERT+BiLSTM+CRF and performs worse in recognition of some types of entities compared to the BERT-FLAT-CRF model. However, compared to the overall, PSLF-CTR performs better in recognizing all types of temporal entities, especially complex ones.

4.3.2. Ablation Study

To explore the effect of logical rules on the model recognition of complex temporal text features, we design the ablation experiments on a private dataset containing PSL-TFR and BERT-TextCNN models. The experimental results for $l_1$∼$l_{18}$ recognition are shown in Table 5 and

Table 6. L-atom statements in private datasets predict the results of controlled experiments.

	BERT+textCNN			PSL-CTR
	P	R	F1	P	R	F1
$l_8$	90.39	93.99	92.15	97.93 (7.54)	96.08 (2.09)	97.00 (4.84)
$l_9$	81.63	77.16	79.33	92.20 (10.57)	91.58 (14.42)	91.89 (12.56)
$l_{10}$	73.08	77.98	75.45	85.93 (12.85)	82.96 (4.98)	84.42 (8.97)
$l_{11}$	90.53	82.13	86.13	93.88 (3.35)	94.64 (12.51)	94.26 (8.13)
$l_{12}$	68.56	71.60	70.05	82.47 (13.91)	79.74 (8.14)	81.08 (11.03)
$l_{13}$	83.24	70.86	76.55	92.35 (9.11)	80.10 (9.24)	85.79 (9.24)
$l_{14}$	66.67	76.00	71.03	86.68 (20.01)	80.91 (4.91)	83.70 (12.67)
$l_{15}$	82.13	83.40	82.76	93.11 (10.98)	86.77 (3.37)	89.83 (7.07)
$l_{16}$	60.86	58.23	59.52	83.93 (23.07)	67.14 (8.91)	74.60 (15.09)
$l_{17}$	47.23	47.84	47.53	63.64 (16.41)	67.74 (19.90)	65.63 (18.09)
$l_{18}$	45.00	44.00	44.49	58.62 (13.62)	56.67 (12.67)	57.63 (13.13)

.

In Table 5, it can be seen that the difference between the two models is not significant since $l_1$∼$l_7$ does not involve logic rules, and the negative feedback mechanism has limited influence on the model. The PSL-TFR scores the highest for the recognition of text features corresponding to $l_1$ atomic utterances and the lowest for the $l_4$ atomic utterances, in terms of precision, recall, and F1 metrics. Except for $l_4$, PSL-TFR obtained high scores in precision (P). However, in recall (R), PSL-TFR performed differently for different text feature recognition and performed poorly in recognizing $l_4$, $l_7$. By analyzing the experimental results, we found that the number of texts satisfying the features of “$isincomplete\left(T\right)$” and “$ismultiple\left(T\right)$” is small in the dataset, which is not conducive to the model capturing the text features.

The results of $l_8$∼$l_{18}$ atomic statements are shown in Table 6. The model without using any logical rules for constraints performs poorly overall in recognition of complex temporal text features and only performs well in recognizing the corresponding text features corresponding to $l_8$ and $l_{11}$. The F1 score of BERT-TextCNN does not exceed 50% in recognizing text features corresponding to $l_{17}$ and $l_{18}$. The PSL-TFR model improves in recognizing $l_8$∼$l_{18}$ all the corresponding text features and achieves a larger improvement in $l_{10}$, $l_{12}$, $l_{14}$∼$l_{18}$ feature recognition (complex time types such as RITL, FIST, and FITL). By analyzing the results of the two comparison experiments, we found that the BERT-TextCNN model often identifies FIST as AIST. At the same time, there are fewer such errors in the PSL-TFR model, which reduces the value of FP and thus improves the precision (P). Moreover, the precision (P) of $l_{16}$ is improved more, and the analysis of the results shows that the same value of FP is reduced, but the precision (P) index is improved more because it is less numerous. Overall, Table 6 verifies that using the text feature recognition model PSL-TFR with fused logic rules has better complex temporal text feature recognition capability.

5. Conclusions

In this paper, we propose a complex medical time recognition method combining PSL and text feedback mechanisms. First, we define seven basic features and eleven extraction rules based on the characteristics of complex medical time. Then these eleven rules are incorporated into the deep learning model through PSL and a text feedback mechanism to improve its ability to recognize complex medical time. Finally, the effectiveness of this method is verified by comparison experiments, and the PSL is proved to have an enhancing effect on text feature recognition using ablation experiments.

There are several limitations in this study, and in the future, we will continue our in-depth analysis to address the existing problems:

• The data used are not a publicly available normalized dataset. Therefore, we consider how to construct a Chinese medical time dataset and achieve normalized annotation of temporal entities and temporal relationships while protecting privacy.
• The model in this paper is only for a Chinese dataset. We will verify whether this method can be used for other languages and explore multilingual temporal recognition models in the future.
• This paper only uses feedback adjustment to adjust the model after the model obtains each training result. Another promising research direction is to explore embedding the rules in the model during the model training.