Exploring the Evolution of Core Technologies in Agricultural Machinery: A Patent-Based Semantic Mining Analysis

Abstract

The semi-automatic construction and analysis of technology roadmaps are at the forefront of applying artificial intelligence techniques. To clarify the development path of the core technologies in the field of agricultural machinery, we propose a core technology evolution path analysis method based on patent text semantic mining. First, the key sentences in the text were extracted, and the BERT model was used to represent the topic’s key sentences in semantic vectorization. Then, the technology roadmap was constructed through the unsupervised LDA topic clustering method, and the main fields of activity, blank fields, and fields of agricultural machinery were visually analyzed. Next, investment in research fields was strengthened. Finally, we mapped the technology roadmap and patent IPC codes and found that the evolution of core technologies in the field of agricultural machinery could be divided into the technology development stage, technology focus stage, and technology transformation stage; this allows us to analyze the evolution and integration of these core technologies. The internal laws of the technology evolution provide a reference for future research plans of governments, enterprises, and institutions, aiding in the patent portfolio planning by revealing the microlevel process of technology integration and the technological trends in agricultural machinery.

1. Introduction

Currently, agricultural machinery is developing in the direction of automation, informatization, and intelligence. Patents are technical documents that convey detailed technological information and play a crucial role in understanding the core technology evolution path [1]. However, there is a lack of studies in the agricultural machinery sector that consider patent analysis within their scope [2]. Therefore, to fully understand the core technology evolution path of the global agricultural machinery field, we perform an analysis of the technical roadmap and IPC classification code of agricultural machinery patent data, which can help us understand the evolution of technology in the field of agricultural machinery; identify past, present, and future technology trends [3,4]; and provide references for corporate technology decisions, industrial planning, and strategic planning of government departments [5].

The technology evolution path aims to describe the development context of technology, that is, the emergence, development, maturity, and extinction, emphasizing the grasp of the law of technological progress and the evolution model from the dynamic perspective of development and change [6]. It is a flexible and powerful tool that has been widely used in national, regional, industrial, and corporate technology development planning to reduce uncertainty in the innovation and planning process [7]. With the development of a technology roadmap research practice, relevant methods and technologies are becoming more abundant. The more widely used methods are the scenario analysis [8], Delphi survey [9], SWOT (strength, weakness, opportunity, threats) analysis [10], and bibliometrics [11]. As part of the development of deep learning, a technology roadmap construction method based on text mining and data mining was proposed, such as the patented product technology keyword method [12], subject–predicate–object analysis [13], and association rule mining [14]. However, some problems remain in the current drawing and analysis of technology roadmaps: First, when traditional methods draw technology roadmaps for patents, papers, policies, and reports, they use the full text as the information source and do not consider noise information. For denoising operations, we used the TextRank model to extract key sentences from the text, and we modified the model algorithm in combination with the characteristics of patent texts to further improve the accuracy of key sentence extraction, improve the expressiveness of patent texts, and remove text noise information. Second, when using deep learning or machine learning models to construct a technical roadmap for text, in terms of the text’s vectorized expression, it cannot effectively represent words in combination with contextual semantics, cannot solve word ambiguity problems, and has insufficient semantic expression capabilities. To overcome this limitation, we used the most advanced vector expression model, the BERT (Bidirectional Encoder Representation from Transformers) model, to accurately express the patent text; we combined it with the LDA (Latent Dirichlet Allocation) topic model for topic clustering and drew a technical roadmap by calculating the similarity between topics in different years. Third, when summarizing and analyzing the patent technology roadmap, this is either carried out only for the analysis of the technology roadmap or it is combined with an expert opinion analysis on this topic; it does not combine the analysis of the patent analysis with the technology roadmap. We conducted a comprehensive analysis by combining the results of the IPC (International Patent Classification) code analysis and the patent technology roadmap. This approach allowed us to identify key technologies, such as real-time embedded implementation for agricultural field monitoring algorithms [15]. By integrating these analyses, we achieve more scientific and objective analysis results of the patent technology evolution path.

The research framework of this paper includes four parts: (1) patent key sentence extraction, (2) key sentence representation learning, (3) topic clustering, and (4) technology roadmap visualization, as shown in Figure 1.

The main contributions of this paper are as follows:

(1)

Enhanced text expression and noise reduction. We utilized text mining techniques, specifically the TextRank model, to extract key sentences from patent abstracts. This approach significantly improves the text’s language expression while reducing unnecessary information and noise. During patent analysis, extracting key sentences enables more efficient and accurate identification of relevant information, reducing data processing requirements and accelerating calculations.

(2)

Patent technology roadmap construction. This study utilized quantitative analysis techniques to construct a patent technology roadmap. First, the patent cluster analysis is performed by integrating the BERT model and the LDA topic model. The resulting roadmap offers a comprehensive visualization of the patent landscape, facilitating strategic decision making. It enables the identification of technological convergence and supports investment considerations in specific technology domains, particularly in emerging areas of technological fusion.

(3)

Technology evolution analysis. The reliability of the clustering method was verified by comparing patent IPC classification codes with the patent technology roadmap. This analysis assists in understanding the trajectory of technological advancements, informing patent portfolio planning by revealing the convergence of multiple technologies, and guiding companies in their R&D (research and experimental development) strategies by identifying emerging technologies originating from technology hotspots.

This paper aims to integrate IPC classification codes and patent semantics to construct a technology roadmap in the agricultural machinery field. The proposed approach enables the identification of technology trends, research hotspots, and areas with potential for further development. Moreover, the findings provide valuable guidance for designing patent development strategies and national innovation strategies.

The main research questions addressed in this paper can be summarized as follows:

(1)

How can text mining techniques be effectively applied to extract key sentences from patent abstracts, enhancing language expression and reducing nonessential information and noise data?

(2)

How can quantitative analysis techniques be employed to construct a patent technology roadmap, combining deep learning models like BERT and topic models like LDA to perform patent cluster analysis?

(3)

How can the combination of IPC classification codes and patent technology roadmaps be used to analyze the technology evolution path and how can should the reliability of the clustering result be validated?

The structure of this paper is as follows: Section 2 introduces the patent technology evolution path analysis, text mining, and related research on technical routes. Section 3 includes the collection of experimental data and patent IPC classification statistics. Section 4 introduces patent key sentence extraction methods, key sentence semantic vectorized expression, topic clustering, and the technical roadmap drawing. Section 5 provides the experimental results and analysis, including five subsections. Section 5.1 describes the TextRank key sentence extraction and ablation experiment. Section 5.2 presents the BERT–LDA clustering and visualization. Statistical tests and other method comparison experiments are presented. In Section 5.3, the first section is the key technology identification, Section 5.4 and Section 5.5 are the technical route drawing and evolution path analysis. Section 6 concludes the paper and proposes further research plans.

2. Literature Review

Technology evolution path analysis aims to analyze the historical process of technology development to discover the changes in technology integration, iteration, and upgrades in the evolution process and to realize the identification and breakthroughs of key technologies. At present, analysis of the technology evolution path mainly uses text mining technology and quantitative analysis to analyze patents based on words or topics. Therefore, the key research content of this paper is expanded on in three ways: analysis of the patent technology evolution path, text mining, and a technology roadmap.

2.1. Analysis of the Patent Technology Evolution Path

The methods for analyzing the patent technology evolution path can be divided into the following two categories: the patent classification code analysis method and the patent text analysis method. (1) The patent classification code analysis method focuses on the changes in technological development and can more clearly discover the fluctuations in the process of technology evolution [16] to reveal the characteristics of technology integration and technology differentiation. Based on patent classification citing network features and bibliometric features, Zhang et al. [17] predicted technology fusion relationships. Mao et al. [18] calculated the similarity between IPC co-occurrence pairs based on a machine learning technology fusion prediction framework and semantic similarity. Moreover, Li et al. [19] revealed evolutionary paths based on concept similarity, technical element entity similarity, and set similarity. (2) The patent text analysis method focuses on starting from the perspective of technological incremental innovation, using high technological similarity as the link between old and new technologies and then deducing the development path of technology. Yang et al. [20] comprehensively utilized the LDA topic model and the Word2vec word vector model, combined with the technology life cycle theory, and analyzed the evolution of technical topics from two aspects: the evolution of a technical topic’s strength and the evolution of a technical topic’s content. Zhai et al. [21] used the BERT-BiLSTM-CRF model to realize the automatic extraction of algorithm terms, and then, according to the rule judgment and citation relationship, they constructed the innovative evolution path of the LDA algorithm. Hou et al. [22] analyzed the core technology evolution path based on the number of patent families, the regional distribution, patent citation analysis, and the patent co-occurrence network.

2.2. Text Mining

In 1995, Feldman et al. [23] formally proposed the concept of text mining for the first time. Text mining, also known as text data mining or text knowledge discovery, refers to the process of extracting previously unknown and potentially useful patterns from large-scale text databases to discover knowledge [24]. In addition to making the connection between text, semantics, and grammar, text mining is used for natural language processing, such as machine translation, information retrieval, and information filtering. Text mining technology has a wide range of applications in technical fields, mainly in two aspects: One is based on statistical analysis of the literature and citation analysis methods. For example, Kostoff et al. [25] used text mining technology based on literature discovery to determine the possible subversive technical disciplines of sexual technology products and include them in the drawing of technical routes. Huang et al. [26] used the method of bibliometrics to discover the key technical fields of the solar cell industry and construct a scientific and technological planning framework for the industry. Zhang et al. [4] combined bibliometrics, qualitative analysis, and visualization techniques to build a hybrid model for writing technology roadmaps. Choi et al. [27] used network embedding and citation analysis to vectorize patents, and they calculated the distance of patents in the vector space to indicate the degree of correlation between patents. The other approach is through methods based on semantic mining and feature extraction. For example, Lee et al. [28] used the LDA topic model to carry out semantic mining on patents to identify emerging fields and trends in financial businesses. Yoon et al. [29] proposed a semantic patent analysis network based on the subject–action–object (SAO) structure to detect state-of-the-art evolution technology through this method. Kim et al. [30] built a keyword network and drew a patent map by extracting features from patents. By analyzing the patent map, they could identify the emergence of emerging technologies and predict their future trends. Joung et al. [31] used text mining tools and techniques to extract features from the text, identify keywords, and then monitor emerging technologies based on keywords.

2.3. Technology Roadmap

Published in 1987, the book Motorola’s Technology Roadmap Process by Willyard et al. [32] aroused widespread concern in the industry. Later, the technical route was applied to various industries, fields, and industries. Kostoff et al. [33] posited that the technology roadmap involves the technical planning of the enterprise to realize the new product by describing the technical path, so as to predict the development of the future market and technology, help make strategic decisions, and realize the strategic goals of the enterprise. Routley et al. [34] asserted that a technology roadmap is an effective method for studying the development trend of emerging technology industries. Galvin [35] proposed that the technology roadmap is an extended outlook on the future development of a specific technology by experts in a certain technical field. In general, technology roadmap methods can be divided into two categories: subjective analysis methods and objective analysis methods. Subjective analysis methods are mainly based on expert opinions and qualitative analysis. For example, Cheng et al. [36] proposed a roadmap construction method based on scenario analysis for strategic planning and decision analysis. Cuhls et al. [37] drew technical routes based on expert opinions, compared the road surveying and mapping projects in China and Germany, and analyzed the technical differences between the two countries. Hooshangi et al. [38] proposed an evolutionary learning approach based on expert opinion for the technological strategic management of Iran’s power industry. Objective analysis methods are mainly based on objective facts and quantitative analysis. As an example, Zhou et al. [39] combined machine learning and visualization methods to discover the convergence process of scientific knowledge in the field of bioinformatics. Han et al. [40] used the LDA topic model to identify the technical topic of patent data and then drew a technology life cycle diagram to analyze the development trends of the mining industry. Joung et al. [31] used text mining technology to identify patent keywords and then clustered them using a hierarchical clustering algorithm to monitor emerging technologies by identifying clusters composed of technical keywords.

Table 1. Summarization of relevant works.

Task	Method	Characteristics of the Method	Reference
Technology evolution analysis based on patents	Patent classification code analysis method	Focuses on the changes in technological development	Zhang et al. [17] Mao et al. [18] Li et al. [19]
Technology evolution analysis based on patents	Patent text semantic analysis method	Deduces the development path of technology, focusing on technological incremental innovation	Yang et al. [20] Zhai et al. [21]
Text mining	Statistical analysis of the literature and citation analysis methods	Based on numerical indicators such as degree centrality	Kostoff et al. [25] Huang et al. [26] Zhang et al. [4] Choi et al. [27]
Text mining	Semantic mining and feature extraction methods	Improves the richness and flexibility of patent processing granularity	Lee et al. [28] Yoon et al. [29] Kim et al. [30] Joung et al. [31]
Technology roadmap construction	Subjective analysis methods	Expert opinions and qualitative analysis	Cheng et al. [36] Cuhls et al. [37]
Technology roadmap construction	Objective analysis methods	Objective facts and quantitative analysis	Zhou et al. [39] Han et al. [40] Joung et al. [31]

shows a summarization of studies that are relevant to the tasks of technology evolution analysis based on patents, text mining, and creating a technology roadmap.

The methods for patent technology evolution analysis include patent classification code analysis and patent text semantic analysis. To achieve a comprehensive analysis, this paper considers both patent classification codes and patent text semantic information. For text mining, semantic mining and feature extraction methods can improve the richness and flexibility of the patent processing granularity. Therefore, this paper utilizes text mining techniques based on semantic mining and feature extraction to extract key sentences from patent abstracts using the TextRank model. In terms of the technology roadmap, objective analysis methods mainly use text mining technology to analyze a large amount of data, dig out the internal connections and laws of the data, and then analyze the test results objectively. This type of method is less affected by subjective factors, and the analysis results are more scientific and objective. Therefore, we used the objective analysis method to construct the technology roadmap, and then, we combined it with the patent analysis technology to conduct a subjective analysis of the technology evolution path.

In summary, the goal of this paper is to develop an automated method or solution for constructing a technology roadmap at a microlevel. The aim is to identify and analyze the evolution path of technologies effectively.

3. Data Acquisition and Analysis

PatSnap is a global patent database that integrates 170 million in-depth processed patent data entries from 16,470 countries and regions since 1790, along with 150 million scientific reports. It offers timely updates and features such as bilingual search, full-text translation, image–text comparison, citation analysis, similarity search, patent mapping, and patent valuation. The Baiten Patent Retrieval System covers 103 countries, regions, and organizations worldwide, with a database of over 180 million records. It supports simple and advanced searches, legal searches, batch searches, and provides patent valuation and various other features. The term “agricultural machinery” is adopted from [2]. This phrase serves as a representative term for the field of agricultural machinery. To ensure a comprehensive collection of relevant patents in this specific domain, this paper selected “agricultural machinery” as the retrieval keyword. From the patent databases of “PatSnap” and “Baiten”, using “agricultural machinery” as the search term, we combed through published invention patents from 2012 to 2021 and extracted the title, abstract, and content of each patent. The preliminary examination of patent documents was based on core keywords. After obtaining all the search data, this paper carried out batch denoising and manual denoising work and batch removal of noise keywords, removed irrelevant patents and documents with rights transfer through manual reading, and finally, obtained a total of 3231 patent data points. The distribution of the number of patents per year is shown in

Table 2. Distribution of the number of patents in the 10-year study period.

Year	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021
Number of patents	211	283	213	282	286	348	364	369	346	529

. The annual data and the total data of the 10-year study period were classified according to the patent IPC. Then, a statistical analysis was carried out on the “department” of the categories “department, major category, subcategory, major group, and small group”, and the results are shown in

Table 3. Proportion of various categories in individual years and the 10-year study period.

	Years	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021	10-Year
Category
A		0.71	0.71	0.66	0.75	0.58	0.66	0.59	0.61	0.57	0.56	0.63
B		0.11	0.14	0.15	0.07	0.18	0.13	0.15	0.15	0.12	0.10	0.13
C		0.05	0.04	0.05	0.07	0.11	0.06	0.09	0.08	0.09	0.08	0.07
D		0.00	0.00	0.00	0.01	0.00	0.00	0.00	0.00	0.00	0.00	0.00
E		0.01	0.01	0.02	0.01	0.02	0.02	0.03	0.01	0.02	0.02	0.02
F		0.06	0.04	0.04	0.04	0.05	0.04	0.05	0.03	0.03	0.04	0.04
G		0.06	0.05	0.06	0.05	0.06	0.08	0.07	0.12	0.16	0.19	0.10
H		0.00	0.01	0.02	0.00	0.00	0.01	0.02	0.01	0.01	0.02	0.01

The bolds indicate values of particular significance, such as maximum or minimum values.

and Figure 2.

Table 3 clarifies that the proportion of Class A patents is the largest in terms of both individual years and the proportion of total data in the 10-year study period, indicating that this has been a popular field of agricultural machinery. In contrast, the proportion of Classes E, F, and H is small, indicating that these have been unpopular fields of agricultural machinery, and the proportion of D is nearly zero, indicating that this has been a blank field of agricultural machinery. Figure 2 shows an overall decrease trend of approximately 15% in the proportion of Class A during the 10-year study period. However, it is still the category with the largest proportion, indicating that agricultural machinery has been developed mainly in this field. Investment in research gradually decreases, but its proportion is still the largest because of its extremely large initial base. Category G exhibits an obvious growth trend, and its proportion increases by nearly 13% in the 10-year study period, indicating that agricultural machinery has seen strengthened investment in this field. The proportions of other categories, that is, B, C, D, E, F, and H, do not change much in the 10-year study period, indicating that agricultural machinery has only maintained investment in these fields.

To further explore the popular field (A) of agricultural machinery, considering the field of strengthening research investment (G) and the blank field (D), we counted the data collected according to the IPC classification code “subclass”, and the results are shown in

Table 4. Statistical results for category A.

Type	Quantity	Total Amount	Type	Quantity	Total Amount	Type	Quantity	Total Amount	Type	Quantity	Total Amount
A01B	613	1990	A01M	42		A23N	11		A63C	2	2
A01C	321		A01N	59		A45F	2	2	A61F	1	18
A01D	572		A01P	1		A47B	1	4	A61K	14
A01F	224		A23B	2	16	A47C	1		A61L	3
A01G	126		A23J	1		A47G	1
A01H	20		A23K	1		A47K	1
A01K	12		A23L	1		A22C	1	1

and

Table 5. Statistical results for category G.

Type	Quantity	Total Amount	Type	Quantity	Total Amount	Type	Quantity	Total Amount	Type	Quantity	Total Amount
G01B	7	79	G01R	1		G05G	2		G16Y	1	1
G01C	17		G02S	13		G06F	44	150	G07C	3	3
G01D	6		G01V	4		G06G	3		G08B	4	8
G01F	6		G01W	5		G06K	27		G08C	1
G01J	2		G02B	3	3	G06N	6		G08G	3
G01K	1		G03B	1	1	G06Q	56		G09G	1	1
G01M	2		G05B	26	77	G06T	14
G01N	14		G05D	49		G10L	1	1

, where “The total amount” represents the amount of each IPC classification code “class”, such as “A01”. From Table 3, it can be found that category A in the popular field of agricultural machinery is mainly concentrated in category “A01”, of which the two subcategories “A01B” and “A01D” account for the largest proportion. Combined with the international IPC classification number of agricultural machinery, “A01” mainly refers to “agriculture, forestry, animal husbandry, hunting, trapping, fishing”, while “A01B” and “A01D” refer to the harvesting of the above aspects, harvesting-related agricultural machinery or parts, and parts or accessories of agricultural tools. In the same way, it can be found that the G category in the field of agricultural machinery, where investment has been strengthened, is mainly concentrated in the category of “G06”. In this category, “G06F” and “G06Q” account for the largest proportion. Combined with the international IPC classification number of agricultural machinery, “G06” refers to “G06F”, and “G06Q” refers to electrical digital data processing and data processing systems and methods. The category of D, which is in the blank field of agricultural machinery, mainly refers to F, for textile and papermaking.

4. Technology Roadmap Construction Method

The technology roadmap construction method in this paper includes four steps: (1) patent key sentence extraction; (2) key sentence representation learning; (3) topic clustering; and (4) technology roadmap visualization.

4.1. Key Sentence Extraction

To eliminate noise data, we used the TextRank model to extract key sentences from the patent abstract and mine the subject center of the abstract, which serves as the basis for the construction of the technology roadmap. The TextRank algorithm originated from Google’s PageRank algorithm idea [41]. When the PageRank algorithm calculates the importance of a webpage, it establishes that the more a webpage is linked by other webpages, the more important the webpage is. Similarly, the TextRank algorithm assumes that the higher the correlation between a sentence and other sentences in the text, the more important the sentence is. The TextRank algorithm first divides the text into sentences and regards the sentences as nodes in the network graph. The weights between nodes are represented by the similarity between sentences, so as to construct a text network graph based on the sentence structure relationship. By assigning node weights, the algorithm can use sorting methods and can extract the top-ranked nodes as the key sentences of the text.

The TextRank algorithm [42] can be expressed as a weighted undirected network graph $\mathrm{G}=\left(\mathrm{V},\mathrm{E},\mathrm{W}\right)$, where V is a set of nodes, the set V includes the nodes of the network graph and each node represents a sentence in the abstract of a patent document, E is a nonempty finite set of edges between nodes, and W is a set of weights on each edge, which are derived based on the similarity between sentences and calculated using cosine similarity. Assuming $\mathrm{V}=\left\{{\mathrm{V}}_1,{\mathrm{V}}_2,\dots ,{\mathrm{V}}_{\mathrm{n}}\right\}$, then we denote $\mathrm{E}=\left\{\left({\mathrm{V}}_1,{\mathrm{V}}_{\mathrm{j}}\right)|{\mathrm{V}}_{\mathrm{i}}\in \mathrm{V},{\mathrm{V}}_{\mathrm{j}}\in \mathrm{V},{\mathsf{\omega }}_{\mathrm{i}\mathrm{j}}\in \mathrm{W},{\mathsf{\omega }}_{\mathrm{i}\mathrm{j}}\ne 0\right\}$ and $W=\left\{{\omega }_{ij}\right|1\le i\le n,1\le j\le n\}$, where ${\omega }_{ij}$ is the weight value of the edge between nodes V_i and V_j. An $\mathrm{n}\ast \mathrm{n}$ similarity matrix M between sentences could be obtained through cosine similarity calculation.

To further improve the accuracy of the sentence weight calculation, considering the position characteristics of the sentence and the similarity between the sentence and the patent title, we used the cosine similarity formula to calculate the similarity between each sentence in the patent abstract and the title, which was recorded as the $\mathrm{T}\left({\mathrm{S}}_{\mathrm{i}}\right)$ matrix, an $\mathrm{n}\ast 1$ matrix where n represents the number of sentences. The formula for calculating sentence position features is shown in Equation (1).

$$\mathrm{POS}({\mathrm{S}}_{\mathrm{i}})=\frac{\mathrm{n}-{\mathrm{p}}_{\mathrm{i}}+1}{\mathrm{n}}\times \frac{\mathrm{m}-{\mathrm{p}}_{\mathrm{i}}+1}{\mathrm{m}}$$

(1)

Here, POS(S_i) represents the position feature weight of the sentence S_i, where n and m represent the number of paragraphs and the number of sentences in each paragraph, respectively. ${\mathrm{p}}_{\mathrm{i}}$ represents the position of the sentence in the paragraph, and it is a positive integer value ranging from 1 to m. The higher the weight, the lower the weight of the sentence in the last paragraph. According to the weight of the position feature, the position feature matrix POS(S_i) was constructed; the weighted similarity of position matrix ${\mathrm{W}}_{\mathrm{POS}}$, shown in Equation (2), is constructed by averaging the values of the matrix $\mathrm{T}\left({\mathrm{S}}_{\mathrm{i}}\right)$ and the matrix $\mathrm{POS}({\mathrm{S}}_{\mathrm{i}})$. The matrix ${\mathrm{W}}_{\mathrm{POS}}$ effectively captures both the position and semantic features of each sentence S_i.

$${\mathrm{W}}_{\mathrm{POS}}=\left[\mathrm{T}\right({\mathrm{S}}_{\mathrm{i}})+\mathrm{POS}({\mathrm{S}}_{\mathrm{i}}\left)\right]\times 0.5$$

(2)

$${\mathrm{S}}_{\mathrm{n}\ast \mathrm{n}}=\mathrm{M}+\left[\begin{array}{ccc}{\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_1\right)& \cdots & {\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_1\right)\\ {\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_2\right)& \cdots & {\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_2\right)\\ ⋮& \ddots & ⋮\\ {\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_{\mathrm{n}}\right)& \cdots & {\mathrm{W}}_{\mathrm{POS}}\left({\mathrm{S}}_{\mathrm{n}}\right)\end{array}\right]=\left[\begin{array}{cccc}{\mathsf{\omega }}_{11}& {\mathsf{\omega }}_{12}& \cdots & {\mathsf{\omega }}_{1\mathrm{n}}\\ {\mathsf{\omega }}_{21}& {\mathsf{\omega }}_{22}& \cdots & {\mathsf{\omega }}_{2\mathrm{n}}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathsf{\omega }}_{\mathrm{n}1}& {\mathsf{\omega }}_{\mathrm{n}2}& \cdots & {\mathsf{\omega }}_{\mathrm{nn}}\end{array}\right]$$

(3)

Then, the weighted similarity matrix S_n∗n is calculated based on Equation (3). The cosine similarity matrix $M$ captures the similarities between sentences in each patent abstract, while the weighted similarity of position matrix $W_{POS}$ incorporates additional weights based on positional information. As a result, the weighted similarity matrix S_n∗n is obtained by adding the values from the cosine similarity matrix M (an $\mathrm{n}\ast \mathrm{n}$ matrix) to the augmented weighted similarity of position matrix ${\mathrm{W}}_{\mathrm{POS}}$ (an $\mathrm{n}\ast 1$ matrix repeated n times to match the dimensions).

From G and the corresponding weighted similarity matrix S_n∗n, the weight of each node could be calculated. For any node, ${\mathrm{V}}_{\mathrm{i}}$, $\mathrm{In}\left({\mathrm{V}}_{\mathrm{i}}\right)$ represents the set of nodes pointing to ${\mathrm{V}}_{\mathrm{i}}$, and $\mathrm{Out}\left({\mathrm{V}}_{\mathrm{i}}\right)$ represents the set of nodes pointing to ${\mathrm{V}}_{\mathrm{i}}$. The weight calculation formula of node ${\mathrm{V}}_{\mathrm{i}}$ is expressed as Equation (4).

$${\mathrm{W}}_{\mathrm{s}}({\mathrm{V}}_{\mathrm{i}})=(1-\mathrm{d})+\mathrm{d}\ast \sum _{{\mathrm{V}}_{\mathrm{j}}\in \mathrm{I}\mathrm{n}\left({\mathrm{V}}_{\mathrm{i}}\right)}\frac{{\mathsf{\omega }}_{\mathrm{i}\mathrm{j}}}{{\sum }_{{\mathrm{V}}_{\mathrm{k}}\in \mathrm{O}\mathrm{u}\mathrm{t}\left({\mathrm{V}}_{\mathrm{j}}\right)}{\mathsf{\omega }}_{\mathrm{j}\mathrm{k}}}{\mathrm{W}}_{\mathrm{S}}({\mathrm{V}}_{\mathrm{j}})$$

(4)

In this formula, ${\mathrm{W}}_{\mathrm{s}}\left({\mathrm{V}}_{\mathrm{i}}\right)$ is the weight of node ${\mathrm{V}}_{\mathrm{i}}$, and d is the damping coefficient, which is generally set at 0.85. ${\mathrm{W}}_{\mathrm{s}}\left({\mathrm{V}}_{\mathrm{i}}\right)$ indicates the probability that a node points to other nodes in the network graph. ${\mathrm{W}}_{\mathrm{s}}\left({\mathrm{V}}_{\mathrm{j}}\right)$ represents the weight value of node ${\mathrm{V}}_{\mathrm{j}}$ after the last iteration, and ${\mathsf{\omega }}_{\mathrm{ij}}$ represents the similarity between node ${\mathrm{V}}_{\mathrm{j}}$ and node ${\mathrm{V}}_{\mathrm{i}}$. Then, the calculation formula of the weight of each node in the text network graph based on TextRank is expressed in Equation (5).

$${\mathrm{W}}_{\mathrm{s}}({\mathrm{S}}_{\mathrm{i}})=(1-\mathrm{d})+\mathrm{d}\ast \sum _{{\mathrm{S}}_{\mathrm{j}}\in \mathrm{I}\mathrm{n}\left({\mathrm{S}}_{\mathrm{i}}\right)}\frac{{\mathsf{\omega }}_{\mathrm{i}\mathrm{j}}}{{\sum }_{{\mathrm{S}}_{\mathrm{k}}\in \mathrm{O}\mathrm{u}\mathrm{t}\left({\mathrm{S}}_{\mathrm{j}}\right)}{\mathsf{\omega }}_{\mathrm{j}\mathrm{k}}}{\mathrm{W}}_{\mathrm{S}}({\mathrm{S}}_{\mathrm{j}})$$

(5)

In this formula, ${\mathrm{S}}_{\mathrm{i}}$ and ${\mathrm{S}}_{\mathrm{j}}$ represent sentences in the text and correspond to the node ${\mathrm{V}}_{\mathrm{i}}$ and node ${\mathrm{V}}_{\mathrm{j}}$, and ${\mathrm{W}}_{\mathrm{s}}\left({\mathrm{S}}_{\mathrm{i}}\right)$ indicates the weight of the sentence ${\mathrm{S}}_{\mathrm{i}}$ in the TextRank network graph.

When using the TextRank algorithm to calculate the weight of each node for the first time, one need not specify the initial value of each node, that is, its own weight; set the initial weight of all nodes to 1, and then ${\mathrm{B}}_0={\left(\mathrm{1,1},\dots ,1\right)}^{\mathrm{T}}$. Thereafter, recursively and iteratively calculate according to the weight of the edge until convergence. The formula for the iterative calculation is expressed in Equation (6).

$${\mathrm{B}}_{\mathrm{i}}={\mathrm{S}}_{\mathrm{n}\ast \mathrm{n}}•{\mathrm{B}}_{\mathrm{i}-1}$$

(6)

It converges well only when the difference between ${\mathrm{B}}_{\mathrm{i}}$ and ${\mathrm{B}}_{\mathrm{i}-1}$ is small and close to zero. When the convergence is reached, the iterative calculation ends, the vector containing the weight value of each sentence is obtained, and then, the sentences are sorted according to the weight value of the sentence. An appropriate number of high-ranked sentences are selected according to actual needs and sorted according to their order in the original text to generate the final key sentence summary.

4.2. Semantic Vectorization Method for Key Sentences

The text vectorization representation method has developed from the initial one-hot word model of representation to the current mainstream doc2vec, Word2Vec, and Glove. These methods have solved the context-dependent problem of words to a certain extent, but they still have problems with word disambiguation. For example, they cannot solve the problem of different meanings of words in different contexts. In recent years, natural language processing models have achieved good results in semantic analysis of text. In 2018, the Google team of Devlin et al. launched the BERT model [43]. BERT uses the transformer language model and combines the attention mechanism to further mine the inner text. Semantic features are used to solve the shortcomings of the traditional vector model. Because of its rich text semantic expression ability, the BERT model has been widely used in semantic analysis of text [44], text clustering [45], question-answering systems [46], and sentiment analysis [47].

The BERT model is mainly obtained by summing the token, segment, and position embeddings of each word and then using the attention mechanism to enhance the contextual semantic representation of words. To integrate all the context words of a word into the vector representation of the word, first multiply the embedding vectors of all words in the input source by the Q, K, and V matrices to obtain the query vector representation of the target word and the key vector of each word in the context representation and the original value representation of the target word and each word in the context. Then, calculate the inner product of the target word query vector and each key vector and perform softmax normalization processing. Use the normalized value as the weight and the value vector of the target word to perform weighted fusion with the value vector of each context word. At this time, the weighted fusion of the target word indicates that the vector has been fused with the context semantics of the word. In the same way, the weighted fusion representation vectors of other words can be calculated, and all weighted fusion representation vectors can be used as the output of attention. The vectorized representation of the BERT model is shown in Figure 3.

The BERT model “english_L-12_H-768_A-12” and related configuration files required for this paper were obtained from Google’s official website, the “subclass” of the patent IPC classification number was used as the classification label, and the patents of the same “subclass” were regarded as the same type of patents. Then, we classified and trained the BERT model, that is, realized the fine-tuning of the BERT model, used the fine-tuned BERT model to vectorize the key sentences of the patent abstract, input the key sentences d_i into the model, and mapped each word to three vectors. We set ω, σ, and ρ as the word vector, text vector, and position vector of the text obtained by the BERT model, respectively. When the BERT semantic feature vector is expressed, the output semantic feature vector d_m is defined as in Equation (7).

$${\mathrm{d}}_{\mathrm{m}}={\mathsf{\varpi }}_{\mathrm{i}\mathrm{j}}(\mathsf{\omega }+\mathsf{\sigma }+\mathsf{\rho })$$

(7)

As can been seen from Equations (6) and (7), the Q matrix represents the query vector for the target word, measuring its relevance to other words in the context. The K matrix represents the key vector for each context word, containing information about individual words. The dot product between the target word’s query vector and a context word’s key vector is represented by ${\mathsf{\varpi }}_{\mathrm{ij}}$, indicating their correlation or relevance. The indices i and j represent the positions of the target word and context word, respectively, which are used for calculating attention weights. The sentence d_i is processed by BERT to generate the semantic feature vector d_m, which enhances the sentence representation with contextual semantics.

4.3. Topic Clustering

To discover core technical topics, we further utilized the LDA topic model to cluster key sentence vectors. The LDA topic model is a model for generating document topics first proposed by Blei et al. in 2003 [48]. It is a three-layer Bayesian structure, including a three-layer structure of topics, documents, and keywords, as shown in Figure 4. The figure shows that documents to topics and topics to words are all subject to multinomial distribution. Through LDA topic modeling on documents, readers and users can quickly understand the information in documents. The core of the LDA [48] topic model is that the document is the probability distribution of the topic, and that the topic is the probability distribution of words, as shown in Equation (8).

$$\mathrm{P}({\mathrm{w}}_{\mathrm{n}}|{\mathrm{M}}_{\mathrm{m}})=\sum _{\mathrm{k}\in \mathrm{K}}\mathrm{P}({\mathrm{w}}_{\mathrm{n}}|{\mathrm{K}}_{\mathrm{k}})•\mathrm{P}({\mathrm{K}}_{\mathrm{k}}|{\mathrm{M}}_{\mathrm{m}})$$

(8)

The meaning expressed by Equation (8) is the product of the probability of word ${\mathrm{w}}_{\mathrm{n}}$ appearing in topic ${\mathrm{K}}_{\mathrm{k}}$ and the probability of topic ${\mathrm{K}}_{\mathrm{k}}$ appearing in document ${\mathrm{M}}_{\mathrm{m}}$, where N represents the total number of document words, M represents the number of documents, and K is the total number of topics. “Document-topic” and “topic-word” obey the Dirichlet prior distribution with parameters α and β, respectively. At the beginning of LDA topic model modeling, these two parameters are randomly assigned, and α and β affect $\mathrm{P}({\mathrm{w}}_{\mathrm{w}}\left|{\mathrm{K}}_{\mathrm{k}}\right)$, the probability value of and $\mathrm{P}({\mathrm{K}}_{\mathrm{k}}\left|{\mathrm{M}}_{\mathrm{m}}\right)$, through nonstop loop iterations, that is, Gibbs sampling learning [49]. The final convergence result is the output of the LDA model. In the LDA topic model, the determination of the topic number K is important to the model. At present, the perplexity value is commonly used as an indicator to determine the number of topics in the model. Generally, when the number of topics extracted is larger, the perplexity value is smaller, the topic’s perplexity value is smaller, and the fit is better. The calculation formula of perplexity is shown in Equations (9) and (10).

$$\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{p}\mathrm{l}\mathrm{x}\mathrm{i}\mathrm{t}\mathrm{y}(\mathrm{D})=\exp \{\frac{{\sum }_{\mathrm{d}=1}^{\mathrm{m}}\log (\mathrm{p}\left(\mathrm{w}\right))\mathrm{d}}{{\sum }_{\mathrm{d}=1}^{\mathrm{M}}{\mathrm{N}}_{\mathrm{d}}}\}$$

(9)

$$\mathrm{p}\left(\mathrm{w}\right)=\mathrm{p}\left(\mathrm{z}\right|\mathrm{d})\times \mathrm{p}(\mathrm{w}\left|\mathrm{z}\right)$$

(10)

Here, w is the word, M is the total number of patents, ${\mathrm{N}}_{\mathrm{d}}$ is the total number of words in the patent data, $\mathrm{p}\left(\mathrm{w}\right)$ is the frequency of words in the sample, and p(w|z) is each word in the dictionary in each topic’s probability of occurrence.

The traditional LDA topic model first segments the text to stop word preprocessing and then builds the id2word word vector based on the thesaurus. To overcome the problems of word order and the lack of semantic expression ability, we could use the fine-tuned BERT model to vectorize the key sentences of the patent abstract, which can make up for the disadvantages of the LDA topic model so that the LDA clustering results obtained are more accurate. This is helpful for the next step of the visual analysis.

4.4. Technology Roadmap Visualization

In this paper, the top 20 feature words of each cluster topic weighting were regarded as the topical center of the topic, and the cosine similarity between different topical centers in adjacent years was calculated as shown in Equation (11) and compared with the threshold value. To judge whether two subjects are similar, we set a and b as two vectors that are needed to calculate the cosine similarity, and the $\mathrm{c}\mathrm{o}\mathrm{s}\left(\mathsf{\theta }\right)$ and the value range is [−1, 1]. The larger the value, the greater the similarity between the two vectors, and the adjacent years were obtained. After calculating the cosine similarity between different topics, we used the Diesel Wisdom Data Visualization Platform (www.511ds.com (accessed on 9 December 2022)) to draw the patent technology roadmap required for this paper.

$$\mathit{cos}(\theta )=\frac{a•b}{\left|\right|a\left|\right|\times \left|\right|b\left|\right|}$$

(11)

5. Experimental Results and Analysis

5.1. Analysis of TextRank Key Sentence Extraction Results

When the TextRank model extracts key sentences from the abstract, it first performs text cleaning operations such as word segmentation, case conversion, and removal of special symbols, and then uses the glove vector to vectorize the words, construct sentence vectors based on the word vectors, and finally calculate the similarity between sentences. The similarity between sentences is used to construct a similarity matrix, based on which a text network diagram is drawn, and the top-ranked sentences are selected as key sentence outputs. In this paper, four sentences were selected from the abstract of each patent as its topic sentence. If the content of a patent abstract did not exceed four sentences, key sentences were not extracted. Examples of key sentence extraction results are shown in

Table 6. Examples of key sentence extraction results.

Patent Abstract

Key Sentence

The invention relates to agricultural machinery, and in particular, to a soil excavating device adopting ultrasonic vibration. The soil excavating device comprises a rack, and further comprises an excavation shovel with an amplitude-change pole, an ultrasonic transducer and an generator ultra the excavation shovel with the amplitude-change pole is fixedly connected to the ultrasonic transducer; the ultrasonic generator transmits an electric signal to the ultrasonic transducer through a signal cable; the ultrasonic transducer is connected to the rack; pole comprises a shovel cutter and a shovel handle; one end of the shovel handle is fixedly connected to the ultrasonic transducer, and the other end of the shovel handle is fixedly connected to the shovel cutter; and a section area, in a vertical direction, of the shovel handle is gradually in transitional change from a fixed connecting end of the shovel handle and the ultrasonic transducer to a fixed connecting end of the shovel handle and the shovel cutter. The soil excavating device adopting ultrasonic vibration can effectively solve the problems of great resistance of soil cutting and excavating operation, high energy consumption and the like.

One end of the shovel handle is fixedly connected to the ultrasonic transducer, and the other end of the shovel handle is fixedly connected to the shovel cutter, of the shovel handle is gradually in transitional change from a fixed connecting end of the shovel handle and the ultrasonic transducer to a fixed connecting end of the shovel handle and the shovel cutter, The soil excavating device adopting ultrasonic vibration can effectively solve the problems of great resistance of soil cutting and excavating operation.

.

Table 6 demonstrates that, by extracting key sentences from the abstract, we can greatly reduce the content of the abstract and remove a large amount of unnecessary noise content; at the same time, the main subject of the abstract can be correctly extracted, such as in the above-presented patent abstract. The abstract mainly describes a digging device using ultrasonic vibration, and the extracted key sentences can also express this meaning. The second part of the abstract of the patent is to introduce a double-sided weeder, and similarly, its key sentences can also express similar meanings. When constructing a technical roadmap for a patent, it is only necessary to know what devices, machines, and equipment are specifically proposed by the patent; a detailed description of it is not necessary information for the construction of a technical roadmap. Extracting key sentences from the abstract can greatly improve the efficiency of patent analysis.

Comparative Experiment

To more scientifically and objectively judge whether using TextRank to extract key sentences from patent abstracts can improve the clustering effect of the text, we designed a comparative experiment to verify this, in which one method uses the full text of the patent abstract for clustering and the other uses the key sentences of the patent abstract for clustering. We calculated the subject coherence (CS, coherence score) and silhouette coefficient (SS, silhouette score) of the two clustering experiment results for comparison. The topic coherence was used to measure whether the semantics of feature words in the same topic are coherent, and its value range is [0, 1]. The smaller the distance between samples in a class and the larger the distance between samples of heterogeneous classes, the larger the value of SS and the better the clustering effect [50]; the value range is [−1, 1]. The experimental results of the two methods are shown in

Table 7. Cluster comparison results of the full text of the abstract and the key sentences of the abstract.

Year	Key Sentence		Summary		Year	Key Sentence		Summary
	CS	SS	CS	SS		CS	SS	CS	SS
2012	0.49	0.39	0.42	0.35	2017	0.37	0.5	0.34	0.3
2013	0.36	0.46	0.25	0.38	2018	0.48	0.52	0.38	0.4
2014	0.46	0.47	0.29	0.44	2019	0.49	0.44	0.41	0.25
2015	0.45	0.38	0.46	0.29	2020	0.37	0.45	0.37	0.27
2016	0.47	0.38	0.37	0.28	2021	0.44	0.39	0.35	0.27

The bold is used to emphasize values that are relatively larger in comparison between the key sentence and the summary.

.

To verify that the method based on key sentences has a significant advantage, we carried out a statistical test analysis on the experimental results of these two methods. First, the Kolmogorov–Smirnov test was used on the two sets of data to determine whether the sample belongs to a normal distribution, and when it did, an independent sample test was performed. The results of the Kolmogorov–Smirnov test showed that the significance is greater than 0.05, which means that the two groups of data obey the normal distribution, so the independent sample test was used to analyze the two groups of data, and the experimental results are shown in

Table 8. Independent sample test results.

Parameters		CS		SS
		Assumes Equal Variance	Does Not Assume Equal Variances	Assumes Equal Variance	Does Not Assume Equal Variances
Levine variance equality test	Significance	0.09	-	0.23	-
t-test for equality of means	Significance (two-tailed)	0.01	0.01	0.00	0.00

.

Table 8 shows that the significance of the Levine variance equality test is greater than 0.05, indicating that the variances of the two groups of data are homogeneous and that the two groups of data obey the normal distribution presented in the previous step, indicating that the t-test can be performed. The significance of the independent sample t-test is less than 0.05, indicating that the difference between the two groups of data is significant. Therefore, the clustering effect of key sentences is better than the clustering effect of the full-text abstract, whether it is in terms of topic coherence (CV) or silhouette coefficient (SS). Thus, by extracting key sentences from the abstract, we can remove the noise information in the abstract, improve the semantic expression ability of the abstract, and improve the text analysis of patents.

5.2. Analysis of BERT-LDA Clustering Visualization Results

We used the fine-tuned BERT model to express the semantic vectorization of patent key sentences, inputted them into the LDA model, and built a “document-topic” and “topic-word” distribution network. We regarded all patents in a certain year as “documents”, regarded “topic” as the clustering topic, and regarded “word” as the feature word of the clustering topic. Then, the clustering of patent data of a certain year could be realized. Before using LDA clustering, we calculated the perplexity to determine the optimal number of clustering topics of patents for each year. When the perplexity obviously tends to be stable, the number of topics at this time is the optimal number of topics. The number of topics at the inflection point is the optimal number of topics. Taking the patent in 2012 as an example, we find in Figure 5 that, when the number of clustering topics is set to 17, the perplexity tends to be stable, indicating that the clustering effect is the best at this time. Similarly, the optimal clustering for the remaining nine years could be calculated. The number of clustered topics per year is shown in Figure 6.

The LDA model was clustered, and we used the UMAP data dimensionality reduction visualization tool to display the clustering results. Taking the data in 2014 as an example, we show the visualization results in Figure 7 and Figure 8. Figure 7 is a two-dimensional representation of each clustering theme. The scatter diagram for distribution shown in Figure 8 is the feature word distribution of each cluster topic, and the size of the word depends on the weight of the word in the topic.

Comparative Experiment

To verify the effectiveness of the proposed clustering method, we compared the proposed clustering method with the traditional clustering method of k-means and the patent clustering model LDA, proposed by Zhang [50]. K-means is a distance-based clustering algorithm. The closer the two objects are, the greater the similarity. The algorithm first selects any K objects as its initial cluster center and then iteratively updates the center point of the cluster until convergence, that is, clustering of data, has been realized. Zhang used the LDA topic model to cluster patents in the blockchain field and then drew a technical roadmap by calculating the similarity between different topics, thereby realizing the technical analysis of the field. By conducting clustering experiments on the patent data used in this paper and then calculating the CS and SS values of different clustering methods, we obtained the experimental results shown in

Table 9. Experimental results table.

Year	K-Means		Zhang-LDA [50]		BERT-LDA
	CS	SS	CS	SS	CS	SS
2012	0.36	0.13	0.44	0.13	0.49	0.39
2013	0.31	0.09	0.32	0.06	0.36	0.46
2014	0.29	0.07	0.35	0.03	0.46	0.47
2015	0.32	0.1	0.38	0.07	0.45	0.38
2016	0.35	0.12	0.41	0.08	0.47	0.38
2017	0.34	0.07	0.35	0.04	0.37	0.5
2018	0.3	0.07	0.35	0.02	0.48	0.52
2019	0.34	0.07	0.35	0.04	0.49	0.44
2020	0.36	0.1	0.35	0.07	0.37	0.45
2021	0.39	0.07	0.35	0.05	0.44	0.39

The bold is used to emphasize values that are relatively larger in comparison among the methods.

.

Similarly, the Kolmogorov–Smirnov test was used for the data in Table 9 to determine whether the sample belongs to a normal distribution, and the experimental results showed that the significance of the SS index is less than 0.05, meaning that it does not obey the normal distribution, and the U-test (a non-parametric method equivalent to the independent sample t-test) was used for analysis. The significance of the CS index is greater than 0.05, meaning that it obeys the normal distribution, in which case we adopted the t-test and obtained the statistical results shown in

Table 10. Statistical table of experimental results.

Method	CS	SS
	Independent Sample t-Test	U-Test
BERT-LDA and k-means	0.001	0.000
BERT-LDA vs. LDA	0.003	0.000
K-means and LDA	0.278	0.019

by sorting out the experimental results.

Table 10 shows that when the index is less than 0.05, there is a significant difference between the data. The BERT-LDA clustering method used in this paper exhibits significant differences to other methods in both the CS coefficient and the SS coefficient, indicating that the clustering method proposed in this paper is the best. The class method works best, and the LDA clustering method proposed by Zhang outperforms the traditional k-means clustering method in terms of SS coefficients. The k-means algorithm determines an initial division based on the initial clustering center and then optimizes the initial division. Choosing a different clustering center produces different clustering results, and the selection of a random initial center affects the stability of the algorithm, thus affecting the clustering effect. The LDA clustering algorithm proposed by Zhang uses a three-layer Bayesian probability model to construct a “document-topic” and “topic-word” hierarchical model for the data set, and then it realizes text clustering, which is more scientific and efficient than the traditional k-means method. However, when this method expresses text vectors, it uses the bag-of-words model for vector representation, which cannot solve the problems of word order and word ambiguity, which leads to the lack of semantic expression ability of the vector representation model, which in turn affects the clustering results. This paper used advanced vector representation. The BERT model can effectively solve these problems and improve the clustering effect.

5.3. Identification of Key Technologies

After clustering the patents in the field of “agricultural machinery” in the 10-year study period by using the method proposed in this paper, we obtained the clustering results of each topic every year, and the top 20 feature words of each topic weighting were used as the topic. The theme center was combined with the international IPC classification number of agricultural machinery, and the key technology of the theme center was expressed. Taking the patent data in 2014 as an example, after clustering and using the proposed method, we obtained the distribution of topic feature words shown in Figure 7. The top 20 feature words of each topic weighting were screened to obtain

Table 11. Distribution of subject keywords.

Topic	Subject Keywords
Topic 1	wheel, frame, machine, water, invent, position, field, device, drive, apparatus, roll, distance, belt, drive, element, bar, material, body, system, roller
Topic 2	device, field, machinery, invent, bale, element, substance, tool, position, soil, agriculture, industrial, rack, angle, machine, surface, construct, plant, form, frame
Topic 3	material, plant, apparatus, stalk, seed, drum, surface, combine, bale, dimension, input, screen, crop, direct, harvest, spread, ground, chamber, invent, method
Topic 4	hours, disintegrate, system, invent, direct, axis, material, rotate, frame, machine, support, bear, plant, wall, table, method, disassemble, device, fertile, machinery
Topic 5	system, device, element, control, field, frame, invent, mechanical, machine, machinery, sheet, implement, operate, agriculture, sieve, position, chamber, harvest, grain, surface

.

As shown in Table 11, Topic 1 contains words such as “water”, “machine”, “device”, and “system”. It can be seen that, combined with agricultural machinery IPC classification numbers B05B and A01C23, the key technology of this topic is a “watering system”. Topic 5 contains words such as “device”, “grain”, “harvest”, and “machine”. Combined with the IPC classification number of agricultural machinery, the key technology of this topic is a “grain harvesting machine”.

Similarly, the key technologies for the year 2014 can be identified, as shown in

Table 12. Key technologies in 2019–2021.

Year	Key Technologies
2019	84. Grain Harvester; 85. Biomass detection machine; 86. Grain processing vehicles; 87. Ethylene glycol testing machine; 88. Automatic watering equipment; 89. Soil crop harvesting and cleaning equipment; 90. Grooving and fertilization equipment; 91. Pest detection equipment; 92. Sowing and fertilizing drones; 93. Cutting transport equipment
2020	94. Field crop management platform; 95. Soil sampling detection system; 96. Air circulation system; 97. Operation control command equipment; 98. Crop sprinkler system; 99. Packing and sorting transportation system; 100. Cutting equipment; 101. Fertilization equipment; 102. Target material valuation system; 103. Transmission system; 104. Cotton cutting and packing machine; 105. Alarm equipment; 106. Field crop threshing equipment; 107. Soil moisture detection system
2021	108. Route guidance system; 109. Heating machinery; 110. Soil carbide synthesis products; 111. Soil monitoring system; 112. Grain processing equipment; 113. Crop image monitoring equipment; 114. Grain seeder; 115. Transport parts; 116. Cotton cultivation; 117. Crop growth air detection system; 118. Auto-photographing vehicle parts; 119. Field crop value analysis system; 120. Grain seed unloading equipment; 121. Automatic seeding equipment

, as well as the themes for each year. Due to space constraints, only the key technologies from the period 2019–2021 are presented. This presents a comprehensive list of key technologies spanning from 2019 to 2021. These technologies are crucial in the field of agriculture and reveal the advancements made during this period. It serves as a reference point to identify the key technologies driving agricultural machinery development over the specified period. It also enables researchers to identify the emergence of new technologies during the specific period.

5.4. Technology Roadmap Visualization

We used the top 20 feature words of each cluster topic weighting as the topical center of the topic and judged whether the two topics are similar by calculating the cosine similarity between different topics in adjacent years and comparing them with the threshold. Taking five topics (79–83) in 2018 and ten topics (84–93) in 2019 as an example, we calculated the cosine similarity between different topical centers and then compared it with the threshold of 0.5. If it is greater than or equal to 0.5, there is similarity between the two subjects; if it is less than 0.5, there is dissimilarity between the two subjects. The similarity calculation results obtained are shown in

Table 13. Similarity of topics in 2018–2019.

	2019	84	85	86	87	88	89	90	91	92	93
2018
79		0.62	0.4	0.45	0.23	0.48	0.45	0.45	0.57	0.48	0.43
80		0.64	0.59	0.32	0.25	0.68	0.37	0.46	0.59	0.55	0.26
81		0.52	0.48	0.35	0.23	0.48	0.35	0.45	0.35	0.4	0.38
82		0.5	0.59	0.48	0.35	0.5	0.33	0.33	0.43	0.48	0.32
83		0.35	0.3	0.4	0.27	0.25	0.4	0.35	0.35	0.35	0.38

The bold is used to emphasize values that exceed 0.5.

. It can be seen from Table 13 that the cosine similarity between topic number 81 in 2018 and topic number 84 in 2019 is 0.52, which is greater than the threshold of 0.5, indicating that the two topics are similar. Similarly, the cosine similarity between different topics in other adjacent years could be calculated. We analyzed whether two themes are similar by judging them based on the threshold, and then we drew a technology roadmap based on similar themes, as shown in Figure 9. Each value in each column in Figure 9 represents the theme number of the year; if the similarity between subjects in adjacent years is greater than or equal to 0.5, they will be connected by ribbon lines.

Figure 9 complements Table 12 by presenting a technology roadmap based on the similarity matrix between the key technologies and adjacent years’ technological themes. This visualization aids in comprehending the trajectory of the evolution of technology within the agricultural domain. It enhances this analysis by providing a visual representation of the relationships and similarities between different technological themes across time. Mapping the connections between key technologies and adjacent years helps in identifying the path of former technological development.

As can be seen from Table 12 and Figure 9, one of the key technologies in 2019 is “84. Grain harvester”. This machine could harvest automatically, thereby increasing precision, reducing human errors, and enhancing overall productivity. From 2019 to 2020, it has led to the emergence of several sub-technologies that have significantly improved agricultural processes.

For instance, “96. Air circulation system” is a sub-technology that enhances the grain harvester’s efficiency. By providing controlled air circulation within the harvester, it prevents moisture build-up and reduces the risk of crop spoilage during harvesting. This technology ensures the preservation of grain quality and extends the shelf life of harvested crops. “99. Packing and sorting transportation system” is a sub-technology that revolutionizes post-harvest processes. It automates the packaging, sorting, and transportation of harvested crops, minimizing manual labor and improving efficiency. “97. Operation control command equipment” is another sub-technology derived from “84. Grain harvester”. It enables farmers to remotely monitor and control the harvester’s operations, such as speed, cutting height, and grain discharge. “101. Fertilization equipment” is a sub-technology that integrates with the grain harvester to optimize nutrient application during harvesting. By precisely targeting fertilizers to the crops, it improves nutrient uptake and maximizes yields. “105. The alarm equipment” is a sub-technology that enhances the grain harvester’s safety and maintenance. It provides real-time alerts for potential malfunctions, such as engine overheating or blockages in the harvesting mechanism.

Figure 10 shows the development mechanism of key technologies for agricultural machinery. By enhancing basic machinery design and fulfilling additional functional demands, a key technology can lead to the development of upgraded machines by improving their efficiency, automation, precision, and safety. It can also inspire the development of new devices that can be integrated with the original machine, meeting new market demands. The examples above illustrate that sub-technologies derived from a key technology have positive impacts on improving agricultural processes, ultimately improving agricultural productivity and sustainability.

5.5. Technology Evolution Path Analysis

Table 12 shows that most of the key technologies of agricultural machinery are related to the machinery or related parts used in the main production processes of economic crops, such as breeding, plowing, planting, irrigation, harvesting, loading, transportation, and storage. Moreover, among the key technologies in 2020 and 2021, many key technologies, such as control systems, management systems, and analysis systems, have emerged, indicating that agricultural machinery is integrated with computer knowledge, which is consistent with the results of data collection and analysis in Section 3 of this paper. In other words, popular fields of agricultural machinery involve harvesting in agriculture, forestry, animal husbandry, hunting, trapping, and fishing, and harvesting-related appliances or parts, as well as processing systems and methods. This verifies the credibility of the cluster analysis of agricultural machinery patents using BERT-LDA in this paper. For the government, enterprises, institutions, and individuals, in the field of agricultural machinery, it is still necessary to focus on research related to crop sowing, growth, harvesting, and transportation. The related agricultural machinery or related parts need to be constantly improved. To upgrade and innovate locally, we must integrate modern information technology; conduct scientific and efficient production operations; reduce carbon emissions, environmental pollution, and waste of resources; and promote the construction of a global ecological civilization.

It can be seen from Table 12 and Figure 9 that some key technologies with the same functions are not connected and represented in the technology roadmap. For the technology “automatic seeding device”, these two key technologies refer to machinery related to sowing, but these two topics are not connected in the technology roadmap, because the characteristic words that constitute these two key technologies are different. That is, different methods and technologies are used to realize the same function, indicating that technological innovation or technological change has been carried out between these two key technologies. However, some key technologies with the same functions are still connected in the technology roadmap, for example, the key technology “harvesting equipment”, numbered 79 in 2018, and the key technology “grain harvester”, numbered 84 in 2019. The themes have the same function and are connected, indicating that the two key technologies use similar technologies to achieve the same function, without much technological innovation.

The evolution of key technologies generally obeys the objective development law of a certain event or is an extended research topic in the previous step, for example, the key technology “separation equipment”, numbered 34 in 2014, and the key technology “screening device”, numbered 46 in 2015. These two themes obey the law of objective development, and they can be better screened in the next step after separation. However, some topics have no obvious correlation but are connected in the technology roadmap, such as the number 38 key technology “grain harvester” in 2015 and the number 57 key technology “automatic spraying vehicle” in 2016. The functions expressed by the two themes are different, and the two themes do not obey the objective development law of events, but the two themes are connected in the technology roadmap, indicating that the basic technology or basic zero of these two key technologies has the same components; only some decisive key technologies or key parts are different, which leads to the difference in the names of the two key technologies. By combining Table 12 and Figure 9, we can see that the key technologies of agricultural machinery can be divided into three stages, as shown in

Table 14. Evolution of agricultural machinery technology in 2012–2021.

Time (Year)	Stage
2012–2015	technology development stage
2016–2019	technology focus stage
2020–2021	technology transition stage

:

Figure 11 shows an example of a technology evolution path, which demonstrates the progression of three stages of technology development.

5.5.1. Technology Development Stage

During the period from 2012 to 2015, by observing Table 13 and Figure 11, it can be observed that there are a relatively large number of key technologies per year from 2012 to 2015, and the correlation between key technologies in different years is not large, without any obvious trend of technological convergence or evolution. As shown in Figure 11, during the technology development stage, the technologies of “2. Fuel control equipment” and “16. Plant water spray control system” emerged, providing precise control for planting. The fusion of fuel control equipment and plant watering control systems formed “24. Plant packaging equipment”, which can provide a stable environment for plants. However, more refined analysis of plant cultivation conditions was still needed, and thus “32. Soil element testing machine” appeared. Then, “32. Soil element testing machine” and “34. Separation equipment” were merged to form “46. Screening device”. It can be presumed that based on the mechanical principles of 32 and the basic functions of 34, the developers designed a screening device that can screen out particles of specific sizes. With the development in agricultural technology, the market demand for plant screening and processing continues to upgrade, requiring the development of more comprehensive and refined equipment. In 2016, the technology development stage transitioned to the technology focus stage. “41. Crop combine harvester” and “36. Screening device” were merged to form “55. Seed screening machine”. It can be presumed that based on the mechanical principles of 41 and the basic functions of 36, developers designed a more functional seed screening device for the critical step of seed selection in agricultural planting. This result is in line with the initial stage of development of a certain field, when the goal is not clear, and all aspects of a technology are involved.

5.5.2. Technology Focus Stage

In the same way, it can be found that from 2016 to 2019, most of the key technologies per year revolve around the sowing, fertilization, watering, harvesting, and transportation of related machinery or parts of crops and the fusion or evolution of key technologies between adjacent years. Clearly, for example, the theme 80 in 2018 stems from the fusion of six key technologies in 2017 and evolves into five other key technologies in 2019. Specifically, Figure 11 can indicate that, during the technology focus stage, the fusion of “55. Seed screening machine” and “67. Fruit automatic packing machine” resulted in the emergence of “73. Plant retting machine”. The fusion of “73. Plant retting machine” and “71. Seed screening equipment” led to the emergence of “82. Automatic packaging machine”. The fusion of “82. Automatic packaging machine” with “79. Harvesting equipment”, “80. Soil trenching machines”, and “81. Seed storage machines” formed “84. Grain harvester”. In the field of agricultural production, full-process management and automated operation of seeds are necessary to improve agricultural production efficiency and seed quality. The fusion of key technologies such as seed screening machines, automatic packaging machines, harvesting equipment, trenching machines, and seed storage machines meets this market demand.

5.5.3. Technology Transition Stage

In some aspects of technical research, the exchange and integration of technologies is frequent. From 2020 to 2021, with the maturity and wide application of computer technology, many management systems and analysis systems related to computer information technology appear in key technologies, indicating that the field of agricultural machinery has developed in relation to electronic information technology. Combination, innovation, and change in traditional methods are in line with the transformation stage of a certain field, and the integration and evolution of technical topics is obvious. Many new technical directions have emerged, and these technologies are all related to a certain field. As Figure 11 shows, in 2020, the technology focus stage transitioned to the technology transition stage. The farms needed to further improve safety and stability, and therefore, “105. Alarm device” was developed based on the technical theory of “84. Grain harvester”. It is a kind of device designed to detect and notify users of potential risks. It utilizes advanced sensors and algorithms to monitor specific parameters and triggers audible or visual alerts when abnormal conditions are detected, providing a safer and more intelligent method of farm management. In the technology transformation stage, “105. Alarm equipment” and “108. Route guidance system” have a high degree of similarity. “Route guidance system” refers to a navigation technology that assists users in finding optimal routes to their destinations. It employs GPS or other positioning systems, combined with real-time data and intelligent algorithms, to calculate efficient routes and provide turn-by-turn directions. And it can be speculated that developers reused the real-time monitoring and data transmission technology of the alarm device to develop the route guidance system.

Through the detailed description of the technological development path of each stage in Figure 11 in the three sections above, it can be seen that the mutual fusion and inheritance of key technologies in the technology evolution path of agricultural machinery reflect the continuous innovation and progress of related technologies in agricultural machinery. In the technology development stage, key technologies focused on meeting the basic market demand for agricultural automation, such as fuel control, plant watering, and soil element testing. In the technology focus stage, the focus gradually shifted to key links such as seed screening, automatic packaging, and harvesting. In the technology transition stage, the application of big data, data analysis, and other technologies further improved the safety and stability of agricultural production. New key technical devices such as alarm devices and route guidance systems reflect the trend of intelligence and full automation in agricultural-machinery-related technologies. Therefore, the technology roadmap drawn in this paper can to some extent reflect the process of technology evolution and help decision makers analyze and summarize the direction of technology development.

6. Conclusions

To clarify the evolutionary law of key technologies in the field of agricultural machinery, we proposed a new method to construct a technology roadmap based on quantitative analysis, and we then integrated it with qualitative analysis and combined the patent IPC classification number analysis and patent technology roadmap. A technology evolution path analysis made the interpretation of results more accurate. The main suggestions based on the specific technology evolution path described above are as follows:

(1)

Foster interdisciplinary collaboration. The fusion of different key technologies throughout the evolution path shows the significance of interdisciplinary collaboration. Decision makers should encourage collaboration between experts in various fields such as mechanical engineering, agronomy, data analysis, and automation. This collaboration can facilitate the development of more refined equipment, meeting the increasing market demand for advanced agricultural machinery.

(2)

Emphasize automation and intelligence. The trend towards full automation and intelligence in agricultural-machinery-related technologies is evident in the technology evolution path. Decision makers should consider the integration of technologies such as big data, real-time monitoring, and data analysis first, while analyzing the direction of technology development.

(3)

Strengthen intellectual property analysis. The integration of patent IPC classification codes and a patent technology roadmap in the research methodology highlights the importance of intellectual property analysis in the agricultural machinery sector. Decision makers should conduct regular analysis of patent landscapes to identify emerging technologies, potential collaborations, and areas for further innovation.

By considering these main suggestions, decision makers can effectively understand the complex landscape of agricultural machinery technology evolution. Interdisciplinary collaboration, an emphasis on automation and intelligence, and intellectual property analysis will contribute to driving the advancement and adoption of innovative technologies in the agricultural machinery industry. While other sources, such as websites or AI models like GPT-3, may provide intuitive information about the recent technological trends, it is worth noting that these platforms rely on large language models (LLMs). Training such models requires significant resources, including billions of training data points and high-performance computing devices. In contrast, the method proposed in this paper offers a more cost-effective approach. It visualizes technological convergence from a microlevel perspective, providing a comprehensive overview of the evolution of technologies over time. By understanding the fusion of different technologies, this approach assists companies in conducting research on existing technologies and constructing patent portfolios planning based on technological integration. It facilitates the identification of the core evolution path of key technologies and continuous improvements, enabling strategic patent positioning. Compared with other approaches that rely heavily on large-scale data and require higher training costs, the method presented in this article offers a valuable framework for understanding the trajectory of the agriculture industry.

Despite the contributions made by our research in analyzing key sentences from abstracts, there are certain limitations that need to be acknowledged. First, our approach only considered semantic correlation among sentences while constructing the similarity matrix, neglecting the positional features between sentences and the semantic correlation between sentences and titles. This oversight may have hindered the capture of important contextual information. Moreover, although we acknowledged the significance of key summary sentences at the beginning or end of the abstract, we did not assign them higher weights, potentially missing out on crucial insights. Additionally, while we recognized the importance of central topic relevance, we did not prioritize it. These limitations highlight areas for improvement and suggest potential avenues for future research.

To address the aforementioned limitations and enhance the effectiveness of key sentences extraction, future studies should explore novel techniques. One direction worth considering is incorporating position features between sentences and utilizing semantic correlations between sentences and titles. By taking the structural and contextual aspects into account, we can potentially improve the accuracy and relevance of extracted key sentences. Additionally, assigning higher weights to sentences at the beginning and end of the abstract, which often contain important information, can further enhance the informativeness of the extracted sentences. Moreover, giving greater weight to sentences that exhibit higher semantic similarity with the title can help identify the most expressive and representative sentences. By pursuing these directions, future research can enhance the precision and comprehensiveness of key sentence extraction, leading to more effective summarization and information retrieval in scientific literature.