Chains-BERT: A High-Performance Semi-Supervised and Contrastive Learning-Based Automatic Question-and-Answering Model for Agricultural Scenarios

Abstract

Achieving automatic question-and-answering for agricultural scenarios based on machine reading comprehension can facilitate production staff to query information and process data efficiently. Nevertheless, when studying agricultural question-and-answer classification, there are barriers, such as small-scale corpus, narrow content range of corpus, or the need for manual annotation. In the context of such production needs, this paper proposed a text classification model based on text-relational chains and applied it to machine reading comprehension and open-ended question-and-answer tasks in agricultural scenarios. This paper modified the BERT network based on semi-supervised and contrastive learning to enhance the model’s performance. By incorporating the text-relational chains with the BERT network, the Chains-BERT model is constructed. Our efficient mode method outperformed other methods on the CAIL2018 dataset. Ultimately, we developed an automatic question-and-answering application to embed the contrastive-learning information aggregation model in this paper. The accuracy of the proposed model exceeded that of several contrasting mainstream models in many open-source datasets. In agricultural scenarios, the model has achieved state-of-the-art levels and is the best in efficiency.

1. Introduction

As the process of informatization in agriculture continues to advance, increasingly more intelligent applications and services have emerged, such as fruit-picking robots, automatic plant disease detection applications [1,2,3], remote sensing field image processing [4], etc. The way to answer the questions of agricultural workers in production has gradually shifted from offline consultation to online. Machine reading comprehension (MRC) is a text-based question-and-answer task (Text-QA), and is a promising research direction in the field of artificial intelligence and natural language processing. It enables computers to help humans find the desired answers in a large amount of text, providing efficient and convenient services to users, thus reducing the cost of people’s access to information. Based on machine reading comprehension to realize automatic question-answering of agricultural scenes, it can improve the processing efficiency of data and facilitate the subsequent research related to the question-answering (QA) system, which is of great significance to the informatization of the agricultural field.

Currently, machine learning is widely used in various fields. Making machines accomplish reading comprehension and question-answering is currently a rather hot topic at the forefront of the artificial intelligence community, mainly involving deep learning, natural language processing, and information retrieval. Previous researchers have optimized different aspects of machine reading comprehension techniques to achieve breakthroughs. Lifu Huang et al. [5] introduced COSMOS QA, a large-scale dataset containing 35,600 reading comprehension questions that require common sense. The dataset is expressed in a multiple-choice format. It focuses on various collections of people’s everyday narratives. They experimented with several state-of-the-art neural architectures and proposed a new architecture that can improve the competing baselines. Finally, the baseline performance of COSMOS QA was established. An Yang et al. [6] investigated the potential of using external knowledge bases (KBs) to further improve the BERT model of MRC. They introduced a self-adaptive KT-NET structure to select the desired knowledge from KBs, then fused it with BERT to achieve contextual and knowledge-aware prediction. The KT-NET provided consistent and significant improvements over BERT, outperforming competing benchmarks on the ReCoRD and squadron 1.1 benchmarks. Zhuosheng Zhang et al. [7] applied syntax to instruct the text modeling by blending syntactic constraints into the attention mechanism. They introduced syntactic dependency of interest into the self-attention network to establish an SDOI-SAN with syntax-guided self-attention. This design can possess better linguistically motivated word representations. Ample experiments demonstrated that the proposed SG-Net facilitates powerful performance.

These high-performance machine reading comprehension techniques have also been used in all kinds of domains to meet different needs. Shuyang Gao et al. [8] suggested using MEC in state tracking from datasets and model architectures. This innovation solved the challenge that existing methods require conversation data with state information and have limited ability to generalize to unknown domains. Ryota Tanaka et al. [9] introduced a new visual machine reading comprehension dataset called VisualMRC. In this dataset, the machine will read and understand the given text in the image to answer the question in natural language. VisualMRC focuses more on developing natural language comprehension and generation capabilities. Experiments show that VisualMRC outperforms the basic sequence-to-sequence model and the state-of-the-art VQA model.

However, there are problems, such as a small corpus, a narrow range of corpus contents, or the need for manual annotation, in the study of agricultural question classification. Agricultural practitioners prefer the web to the traditional way of consulting experts on agricultural production techniques. At the same time, web data are complex and confusing, making it challenging to get the appropriate results. In the context of such production requirements, it is crucial to construct a model for machine reading comprehension and automatic question-answering scenarios applicable to agricultural production. In this paper, we proposed a text classification model based on text-relational chains and applied it to machine reading comprehension and open-ended QA tasks in agricultural scenarios. In order to improve the model effect, we modified the BERT network and developed an application to embed the presented model. The main contributions of this paper are as follows:

• Proposed a text model based on textual relational chains.
• Modified the BERT network based on semi-supervised and contrastive techniques.
• Applied extensive experimental validation on open-source datasets.
• Developed an application for the model so that it can be applied in agricultural scenarios.

The subsequent parts of this article are organized as follows: Section 2 introduces the development, tasks, and methods of machine reading comprehension. Section 3 analyzes the datasets employed in this paper and presents applied and suggested methods. Section 4 displays the experiment platform and evaluation metrics. Section 5 gives experimental outcomes and discusses these results. Section 6 summarizes the whole work and provides ideas for future works.

2. Related Works

Machine reading comprehension (MRC) is a classical task in natural language processing (NLP). It requires the machine to fully understand the linguistic meaning of co-text and then answer the question. MRC is of great practical significance. From the industrial side, MRC is the core technology of many products. In a search engine, for example, a machine can read a document and answer a user’s query. In intelligent customer service, machines can answer questions from users. Academically, MRC measures how well a machine understands natural language. All in all, MRC is an auspicious task.

2.1. Machine Reading Comprehension Development

Early MRC systems can be traced back to the 1970s when Lehnert proposed the famous QUALM system [10]. However, due to its scale and field limitations, it has not been widely used. In the 1980s and 1990s, research on MRC fell into disuse. In 1999, Hirschman et al. presented a dataset of 60 stories [11], which brought renewed attention to MRC. At that time, approaches to MRC were limited to rule-based approaches [12,13]. Although these methods have strong interpretability, they cannot understand the semantic information of the text and primarily rely on the rules and features designed by experts, so they do not achieve good results.

From 2013 to 2018, researchers began formalizing MRC as a supervised learning task and using manual annotation methods to construct datasets. In 2013, MCTest [14] was released as a multiple-choice reading comprehension dataset, which was of high quality, whereas it was to small to train neural models. In 2015, CNN/Daily Mail [15] and CBT [16] were released. These two datasets are generated automatically from different domains and are much larger than previous datasets. In 2016, SQuAD [17] was launched as the first large-scale dataset with questions and answers written by humans. Then, in the same year, other datasets with high quality were constructed [18]. Within the next three years, various datasets from different domains sprung up [19,20,21,22,23], making training an end-to-end MRC model possible.

Since 2018, there has been tremendous change in solving MRC tasks. Along with the release of larger datasets, neural MRC, built by fully developed deep-learning techniques, shows its great superiority over traditional rule-based and machine-learning-based MRC and has gradually become the mainstream in the research community. Neural MRC methods based on deep learning can capture the semantic information of text to a certain extent, such as Math-LSTM [24], BiDAF [25], and QANet [26]. These models go far beyond traditional approaches. In recent years, with the advent of large-scale pre-trained language models such as BERT [27], MRC has made remarkable progress and even surpassed human performance on some datasets.

2.2. MRC Tasks

Machine reading comprehension is a primary task of the question-answering system. It provides relevant context to each question to infer the answer. Machine reading comprehension aims to extract the correct answer from a given context and even generate more complex answers based on the context. Analogous to the process of human reading comprehension, at present, machine reading comprehension can be divided into four types: cloze, multiple choice, span extraction, and free answer. In the following sections, each type of task is described in turn.

2.2.1. Cloze Tests

The cloze MRC usually takes a paragraph of text and removes a word or sentence from the text. The contents omitted from the text are usually nouns or entities, and the machine needs to pick the correct answer from the candidate words or sentences to fill the gap after fully understanding the context. This test method was proposed by linguists in 1953 [28]. The cloze MRC not only examines the machine’s language modeling ability but also the profound reasoning ability. A typical case of cloze-type MRC is shown in Figure 1.

Figure 1. Cloze-type MRC [29].

2.2.2. Multiple Choice

There are three core components in a multiple-choice MRC task: document, question, and candidate. The document is usually a given text containing several sentences. For each question, multiple candidates are provided, and usually, only one option is the correct answer. After reading a given document, the system is asked to choose the correct answer from a list of candidates for the question given. A typical example of a multiple-choice MRC is shown in Figure 2.

Figure 2. Multiple-choice MRC [14].

2.2.3. Span Extraction

Unlike the above two types of reading comprehension, this type of MRC has no candidate answer after a given question and needs to extract a piece of continuous text from the document as the answer. Specifically, given the context and the question, the machine needs to find a start position and an end position based on the question, with the text segment between the start position and the end position as the answer. In recent years, span extraction has become the most widely studied reading comprehension task. A typical example of span-extraction MRC is shown in Figure 3.

Figure 3. Span-extraction MRC [17]. Color fonts represent the answers of questions.

2.2.4. Free Answering

The above three types of MRC tasks are limited to a single context, which is unusual in real life. Freestyle question answering requires the machine to be able to reason and summarize the answer in multiple contexts for a given question. Therefore, the free-answer task is the most complex of the four tasks and is more suitable for realistic application scenarios. The free answer usually contains only the context and the question, and the machine needs to generate the answer to the question based on the context, which can be either a span in the context or text that does not appear in the context. A typical case of free-response MRC is shown in Figure 4.

Figure 4. Free-answer MRC [20].

2.3. Open Evaluation Datasets

Datasets are an essential driver to accelerate the development of the field of MRC. Some of these datasets, such as CNN & Daily Mail [15], SQuAD [17], and MSMARCO [18], can be seen as milestones in MRC, facilitating the emergence of the latest technologies. In this section, we present several representative datasets for each MRC task, as detailed in Table 1.

Table 1. Public datasets for machine reading comprehension.

Task	Dataset	Publication Year	Type	Training Set Size	Test Set Size	Source
Cloze tests	CNN/Daily Mail [15]	2015	Narrative writing	1.2 MB	56 KB	News report
	CBT [16]	2016	Narrative writing	669 KB	10 KB	Children’s books
	LAMBDA [29]	2016	Narrative writing	2 KB	5 KB	Fiction
	CliCR [30]	2018	Explanatory article	91 KB	7 KB	Clinical reports
Multiply choice	MCTest [14]	2013	Narrative writing	1480 KB	840 KB	Virtual stories
	RACE [22]	2017	Multiply types	88 KB	5 KB	English tests
Span extraction	SQuAD [17]	2016	Explanatory article	87 KB	10 KB	Encyclopedia
	NewsQA [19]	2016	Narrative writing	108 KB	6 KB	News
	Trivia [21]	2017	Multiply types	582 KB	73 KB	Encyclopedia and networks
Free answering	MSMARCO [18]	2016	Multiple types	100 KB	100 KB	Network texts
	SearchQA [31]	2017	Multiple types	100 KB	27 KB	Network texts
	Dureader [32]	2017	Multiple types	202 KB	10 KB	Network texts

3. Materials and Methods

3.1. Dataset Analysis

The dataset used in this paper was collected from the Agricultural Information Database of China Agricultural University and the Shandong Farm Manager App database. The dataset contains a total of 145,318 agricultural scenes, including topics such as animal husbandry, plant protection, and fine agriculture. The annotation of the database was accomplished by undergraduate and graduate students with relevant expertise from the College of Information and Electrical Engineering, College of Agronomy and Biotechnology, College of Plant Protection, and College of Biological Science of China Agricultural University.

3.2. Methods

In this paper, we proposed a contrastive learning approach based on text matching, which achieved higher accuracy while consuming the same amount of computing power as traditional methods.

3.2.1. Text Model Based on the Textual Relation Chain

In this paper, we proposed a method for the relation chain of different texts. Specifically, we considered labels as a sequence and constructed a chain relationship between labels based on this sequence. It is like the context in a sentence, as shown in Figure 5.

Figure 5. Textual relation chain.

In order to make this text structure based on chained representations processable by BERT networks, we then adapted the training process of BERT networks.

The model was first trained to learn the generic text representation by MLM (Mask Language Model) and NSP (Next Sentence Prediction) using a large-scale unlabeled corpus. Then the model parameters were fine-tuned on a specific downstream task dataset. In this training paradigm, the model parameters are no longer randomly initialized but have been integrated with generic knowledge. BERT has Base (12-layer Transformer as the encoder) and Large versions (24-layer Transformer as the encoder) according to the size of the model parameters, and cased (case sensitive) and uncased (case insensitive) versions according to whether they are case-sensitive or not, as shown in Figure 6.

Figure 6. Training process of the BERT network.

3.2.2. Optimization for BERT

To improve the accuracy of the model, this paper used the original BERT model as the fixed feature providing model $fF$ and the BERT model to be fine-tuned as the representation model to be trained $fT$. The pairing of the representation vector obtained by pooling the output of the same sentence at each Transformer layer of $fF$, and its representation vector at the CLS label obtained after $fF$, was used as a positive example. In contrast, the pairing between the representation vectors obtained from different sentences was used as a negative example for contrastive learning, as shown in Figure 7.

Figure 7. The structure of the BERT sentence encoder.

In the word embedding session, we added words that did not appear more than two times in BERT’s original word list to the word embedding by random initialization. In the encoding model of labels, we also used BERT to encode the text of labels or their descriptions. Then we add chain information to the encoding of labels. Finally, we encoded the label sequences by a Bi-LSTM. The overall flow of the model is shown in Figure 8.

Figure 8. The structure of the classification model.

3.2.3. Semi-Supervised

As unsupervised learning, semi-supervised learning aims to learn a generic feature representation for downstream tasks. In the classification task, we cannot determine the class of unlabeled samples. However, based on common sense, we can assume that the following basic assumptions hold for most tasks:

• Smoothness assumption: If two samples, $x_1$ and $x_2$, are similar, their corresponding output representations, $y_1$ and $y_2$, should also be close to each other. Conversely, if the two samples, $x_1$ and $x_2$, are different, their corresponding output representations, $y_1$ and $y_2$, should also be far apart.
• Clustering hypothesis and low-density hypothesis: The sample data should be clustered into clusters, and each cluster of samples corresponds to a subclass. In other words, similar samples are always densely clustered in one area, and the boundary line of classification is always located at the place where the sample density is low. Therefore, it can be assumed that if two samples, $x_1$ and $x_2$, are very similar, they are of the same kind, and the classification boundary will not be located between them.
• The stream shape assumption: The input control consists of multiple low-dimensional stream shapes, all samples are located in a stream shape, and the data points located on the same stream shape have the same classification labels.

In this paper, we use two semi-supervised learning methods:

• Agent labeling-based semi-supervised learning method.
  Proxy labels are high-confidence labels derived from the classification of the model itself, which are mixed into the labeled samples to provide additional information to assist in the subsequent training steps. However, these proxy labels are usually noisy and not fully accurate.
  Further, semi-supervised learning methods based on agent labels can be subdivided into self-supervised learning methods and multi-view learning methods. The self-supervised learning method first uses the labeled data to train the model, then uses the trained model to classify the unlabeled samples, removes the part of the classification confidence above a certain threshold, adds it to the labeled samples, then retrains the model and repeats this step until all the unlabeled samples have been added to the labeled samples.
  The multi-view learning approach involves training several models with different parameters, each with a higher confidence level of unlabeled samples classified by other models in addition to the labeled dataset. The classical methods are the Co-training and Tri-training.
  Co-training divides the labeled dataset into two and trains the two classification models separately. Each classification model adds its own unlabeled samples with confidence higher than a certain threshold to the training set of the other classification model and then repeats the training until all unlabeled samples have been added to both training sets.
  Tri-training is performed by taking three training sets from the originally labeled samples with put-back sampling, and training three models separately, and then each model adds the unlabeled samples judged by the other two models to its own training set, and then repeats the training until all the unlabeled samples have been added to the three training sets.
• Consistency-based semi-supervised learning method.
  The consistency-based semi-supervised learning method mainly utilizes the clustering assumption/low-density assumption, i.e., two extremely similar samples belong to the same class, regardless of the possibility that the classification boundary may cross the high-density region between them. Thus, for sample x, a certain perturbation can be made to produce $x^{\prime }$, and it is assumed that x and $x^{\prime }$ must belong to the same classification.
  We assume that a slight perturbation does not change the classification of a sample, so for the same sample x, $y_1$ and $y_2$ are obtained by applying different dropouts and random data augmentation, respectively, by the model operation, and the results of the two operations are $y_1$ and $y_2$ with a certain weight $w\left(t\right)$ added to the final loss function, as shown in Equation (1),

  $$L=w\frac{1}{|D_u|}\sum _{x_i\in D_u}{d}_{MSE}(z_i,\widehat{z_i})+\frac{1}{|D_l|}\sum _{x_i,y_i\in D_l}{d}_{MSE}H(y_i,f\left(x_i\right))$$

  (1)

  where w is the weight of the consistency loss, $D_u$ is the unlabeled dataset, $D_l$ is the labeled training set, f is the classification model, H is the classification loss function, and $d_{MSE}$ refers to the mean square error.

3.2.4. Contrastive Learning and Information Aggregation Method

This section discusses the incorporation of contrastive learning and information aggregation method, as shown in Figure 9.

Figure 9. Structure of information improved by contrastive learning.

By adjusting the hyperparameters, this section will be discussed in Section 4.3; it is proven that the two methods are compatible and can be superimposed on each other. The superimposed methods achieve better results than the two methods alone on CAIL2018. Moreover, the combined method achieves the best results, as shown in Table 2.

Table 2. Superimposed experiments based on CAIL2018.

Algorithm Models	Micro-F1	Macro-F1
QANet [26]	78.3	61.6
Match-LSTM [24]	77.9	60.3
BiDAF+self-attention+ELMO [25]	84.7	64.5
Performance Mode (ours)	86.5	68.8
Efficient Mode (ours)	84.5	64.5

4. Experiments

4.1. Platform

The test device is a desktop computer with a Core i9-10900k CPU and Nvidia RTX3080 GPU. In the training process, the experiments were run on Ubuntu 20.14, using the Python programming language, and the model implementation was based on the PyTorch framework.

4.2. Evaluation Index

For different MRC tasks, there are different assessment indicators. For cloze and multiple-choice, the most common evaluation index is accuracy. For fragment extraction, the model’s performance is evaluated by calculating the exact matching (EM) and $F1$ values. The widely used evaluation metrics are micro-F1 and macro-F1 because the answers of the free answer are not limited to the context. In this section, we describe these evaluation metrics in detail.

4.2.1. Accuracy and F1 Score

Accuracy based on correct answers is often used in cloze tests and multiple choice evaluations. Given a problem set with m problems $Q=\{Q_1,Q_2,\cdots D_m\}$ if the model accurately predicts the answers to n questions.

Exact matching ($EM$) is a variant of accuracy often used in span-extraction MRC. Specifically, an exact match indicates whether the predicted answer exactly matches the standard answer. The $EM$ value is 1 if the predicted answer matches the standard answer and 0 otherwise.

$F1$ value is a critical evaluation index in the classification task. In MRC, $F1$ is often used in span extraction tasks; it can be calculated using Equation (2).

$$F1=\frac{2·Precision·Recall}{Precision+Recall}$$

(2)

Suppose S is the model’s predicted answer and T is the standard answer. $\left|S\right|$ shows the text length of the predicted answers, and $\left|T\right|$ shows the length of the accurate answers. $|S\bigcap T|$ shows the length S overlapped with T. For Equation (2), the calculation of the $Precision$ and $Recall$ are shown in Equations (3) and (4).

$$Precision=\frac{|S\bigcap T|}{\left|S\right|}$$

(3)

$$Recall=\frac{|S\bigcap T|}{\left|T\right|}$$

(4)

Compared to the $EM$ value, the $F1$ value is used to measure the degree of textual overlap between the model’s predicted answer and the standard answer. The $F1$ value of all QA pairs in the dataset is averaged, which is the $F1$ value of the whole dataset.

4.2.2. Micro-F1 and Macro-F1

The micro-F1 is the micro $F1$ value. Considering the multi-label classification as n binary classifications, there are n confusion matrices. Then we can calculate the $TP$ (True Positive), $FP$ (False Positive), and $FN$ (False Negative) of n categories. Thus we can sum up these metrics to get the overall $T{P}_{all}$, $F{P}_{all}$, and $F{N}_{all}$, so as to find the overall Precision and Recall and then find the micro-F1 value. The calculation process is shown in Equations 5-6.

$$Precisio{n}_{micro}=\frac{{\sum }_{i=1}^{n}T{P}_i}{{\sum }_{i=1}^{n}T{P}_i+{\sum }_{i=1}^{n}F{P}_i}$$

(5)

$$Recal{l}_{micro}=\frac{{\sum }_{i=1}^{n}T{P}_i}{{\sum }_{i=1}^{n}T{P}_i+{\sum }_{i=1}^{n}F{N}_i}$$

(6)

Therefore, the $F1$ value calculated based on the above $Precisio{n}_{micro}$ and $Recal{l}_{micro}$ is the $F{1}_{micro}$, as shown in Equation (7).

$$F{1}_{micro}=2·\frac{Precisio{n}_{micro}·Recal{l}_{micro}}{Precisio{n}_{micro}+Recal{l}_{micro}}$$

(7)

In contrast, macro-F1 is precisely the arithmetic average of $F{1}_n$ calculated by first finding the $F1$ of all classes and then calculating it directly. The procedure is shown in Equations (8)–(10).

$$Precisio{n}_{macro}=\frac{1}{n}\sum _{i=1}^{n}Precisio{n}_i$$

(8)

$$Recal{l}_{macro}=\frac{1}{n}\sum _{i=1}^{n}Recal{l}_i$$

(9)

$$F{1}_{macro}=\frac{1}{n}\sum _{i=1}^{n}F{1}_i$$

(10)

Therefore, in the case of multiple classifications, it is necessary to decide depending on the number of different classes in the dataset:

• Typically, if the amount of data in the dataset is unbalanced for each class and all classes are equally important, then using macro-F1 will work; because macro-F1 is calculated by class and then takes the average F1 as the final value. In comparison, micro-F1 will approximate the F1 of the category with the largest amount of data under extremely unbalanced data, which is not a good measure of the overall effect of the model.
• If we need to focus on weak categories in an unbalanced dataset, weighted-F1 should be utilized, which is a weighted average of macro-F1, and the weight is the weight of category numbers to the overall number.

4.3. Hyperparameters Analysis and Two Mode

In this section, we will discuss how we set up the hyperparameters in our model. Specifically, this includes the number of batch sizes, the number of hidden layers, and other parameters. For the batch size, the larger the number, the better the model is trained in theory, so we set it to the maximum value, i.e., 64, as far as the graphics memory of the card can support, and we also performed experiments to verify, as shown in Table 3. For the setting of hidden layers and other parameters, we refer to the work in [33].

Table 3. Verification experiments on batch size from the MCTest dataset.

Batchsize	Micro-F1	Macro-F1
8	74.1	41.6
16	78.9	44.3
32	79.2	44.8
64	81.6	49.1

In addition, for some scenarios, such as control scenarios, where real-time is more important, we have stripped the information integration branch and named it efficient mode, while the corresponding mode with all functions is called performance mode.

5. Results and Discussion

5.1. Results

From Table 4, we can see that our efficient mode method outperforms other methods on the CoLA dataset, and the operational efficiency is much higher than that of the other methods. It is close to that of the improved multi-label text classification method using BERT as the text encoder.

Table 4. Comparisons for different methods based on the CoLA dataset.

Model	Micro-F1	Macro-F1
QANet	74.2	44.3
Match-LSTM	78.3	47.8
BiDAF+self-attention+ELMO	81.6	48.3
Performance Mode (ours)	84.9	50.1
Efficient Mode (ours)	82.7	48.7

As shown in Table 5, the MCTest dataset has a shallow level of labels, and the problem more closely resembles a traditional multi-label text classification problem.

Table 5. Comparisons for different methods based on the MCTest dataset.

Model	Micro-F1	Macro-F1
QANet	64.3	52.1
Match-LSTM	68.1	52.8
BiDAF+self-attention+ELMO	75.9	56.4
Performance Mode (ours)	77.6	57.7
Efficient Mode (ours)	75.2	56.4

5.2. Discussion

5.2.1. Ablation Experiments for Contrast Learning

This paper conducted ablation experiments on the RCv1 dataset to verify the situation without the two contrastive learning text-matching auxiliary loss functions.

As shown in Table 6, both auxiliary loss functions for contrastive learning text matching enable the model to better use the relationships between labels and the text description information of the labels themselves, which plays a vital role in the text classification task.

Table 6. Ablation experiments based on contrastive learning.

Model	Micro-F1	Macro-F1
QANet	77.2	60.4
Match-LSTM	78.7	60.6
BiDAF+self-attention+ELMO	82.7	63.6
Performance Mode (ours)	84.3	64.0
Without $l_{match}$ (ours)	82.9	63.5
Without $l_{rawmatch}$ (ours)	81.6	63.1
Without loss functions (ours)	81.0	62.8

5.2.2. Agricultural QA Scenario Test

In order to solve the problem of low accuracy of the question-answering system in the agricultural scenario, we applied the proposed information aggregation model to machine reading comprehension datasets for testing and received the following results in Table 7.

Table 7. Testing results by applying the proposed model to agricultural datasets.

Training Dataset	Models	All Dataset		Subset
		EM	F1	EM	F1
SQuAD 1.1	BERT	57.44	66.31	67.43	74.18
	Ours	59.03	67.26	68.31	74.82
SQuAD 2.0	BERT	55.21	62.78	65.33	73.08
	Ours	60.37	66.03	67.90	74.05
SQuAD 1.1 + 2.0	BERT	59.33	62.81	66.29	73.98
	Ours	62.18	63.08	70.29	77.01

From Table 7, we can see that our model’s $EM$ and $F1$ metrics outperform the original BERT model. Such model performance indicates that the proposed method can achieve automatic problems in agricultural scenarios. Figure 10 shows the application after deploying our model to the wechat platform.

Figure 10. Screenshot of automatic question-and-answering in agricultural scenarios based on the wechat platform.

5.2.3. Future Work

First, the current text classification methods still essentially treat categories as mere labels and model each category by the samples in the category, even with semantic matching, as an auxiliary loss function. Therefore, they require a certain amount of the training set in the domain and cannot be directly transferred to a text classification task at other levels without a training set. In real-life text classification tasks, the application scenarios are very diverse, and it is not possible to manually label the data for each specific application scenario in a short period of time, given the high cost of manual labeling. This requires a model with good migration capability, which can be applied to another domain at a low cost by training in one domain, and this will be the next research direction of the authors.

6. Conclusions

Machine realizing machine reading comprehension and question-and-answering are hot topics in the current artificial intelligence field. Applying question-and-answering in agriculture can facilitate data management and practical query use. Therefore, this paper presented a text model based on a textual relation chain to achieve high-performance automatic question-and-answering in agricultural scenarios. In order to improve the effectiveness of the model, we modified the BERT network: the semi-supervision and contrastive techniques were introduced into BERT, and a large number of test experiments and ablation experiments were undertaken based on many datasets. The main contributions of this paper are as follows:

• We proposed a text model based on the chain of textual relations.
• We modified the BERT grid based on semi-supervised and contrastive techniques.
• We used extensive experimental validation on open-source datasets.
• We designed a low-computing-power model for edge-computing scenarios and finally deployed it using an information aggregation method based on contrastive learning.
• We developed applications for the model to be used in agricultural scenarios. The application is capable of providing detailed instructions for planting after being questioned. For example, it can answer the appropriate time to plant vegetables or fruit. In addition, the automatic question-and-answering system will deliver proper treatment for a specific plant disease.

The accuracy of the proposed model can outperform the mainstream models on many datasets. Our method based on contrastive learning reached 86.5 micro-F1, 68.8 macro-F1, and 4.9 classification samples per second on the CAIL2018 dataset. Moreover, it also achieved 84.9 micro-F1, 50.1 macro-F1, and 5.1 classification samples per second on the CoLA dataset. The efficient mode is the best in terms of efficiency. The information aggregation method based on contrastive learning has achieved a state-of-the-art level in the agricultural scenario dataset. Further research will be conducted based on this method to improve the classification effect of agricultural questions.