Improving Sentence Retrieval Using Sequence Similarity

Abstract

Sentence retrieval is an information retrieval technique that aims to find sentences corresponding to an information need. It is used for tasks like question answering (QA) or novelty detection. Since it is similar to document retrieval but with a smaller unit of retrieval, methods for document retrieval are also used for sentence retrieval like term frequency—inverse document frequency (TF-IDF), $\mathrm{BM}25$, and language modeling-based methods. The effect of partial matching of words to sentence retrieval is an issue that has not been analyzed. We think that there is a substantial potential for the improvement of sentence retrieval methods if we consider this approach. We adapted TF-ISF, $\mathrm{BM}25$, and language modeling-based methods to test the partial matching of terms through combining sentence retrieval with sequence similarity, which allows matching of words that are similar but not identical. All tests were conducted using data from the novelty tracks of the Text Retrieval Conference (TREC). The scope of this paper was to find out if such approach is generally beneficial to sentence retrieval. However, we did not examine in depth how partial matching helps or hinders the finding of relevant sentences.

1. Introduction

Information retrieval involves finding material (e.g., documents) of an unstructured nature (e.g., text), that satisfies an information need from within large collections [1]. Information retrieval systems generally consist of an index of documents and a query provided by the user [2]. Information retrieval systems should rank documents by their relevance after processing the documents. When the information retrieval system receives the query from the user, a system aims to provide documents from within the collection that are relevant to an arbitrary user information need [1].

Sentence retrieval is similar to document retrieval and it’s defined as the task of acquiring relevant sentences as a response to a query, question, or reference sentence [2] or, simply, task of finding relevant sentences from a document [3,4].

It can be used in various ways to simplify the end user task of finding the right information from document collections [4]. One of the first and most successful methods for sentence retrieval is the term frequency—inverse sentence frequency ($\mathrm{TF}\text{-}\mathrm{ISF})$ method, which is an adaptation of the term frequency—inverse document frequency ($\mathrm{TF}\text{-}\mathrm{IDF})$ method to sentence retrieval [3,5]. Also, $\mathrm{BM}25$ and language modeling-based methods are used for sentence retrieval where the sentence is the unit of retrieval [6].

In this paper, we thoroughly tested the effect of partial matching of terms to sentence retrieval. Text matching was the basis for each natural language processing task [7,8].

We tested the $\mathrm{TF}\text{-}\mathrm{ISF}$, $\mathrm{BM}25,$ and language modeling-based method with sequence similarity presented as the partial matching of words.

For the testing and evaluation of new methods, data of the Text Retrieval Conference (TREC) novelty tracks [9,10,11] were used as a standard test collection for the sentence retrieval methods.

Many different information retrieval methods are used for sentence retrieval. These methods are always document retrieval methods which are adapted for sentences. Contrary to document retrieval, when implementing sentence retrieval, no processing is implemented that allows the non-exact or partial matching of words. We think that taking the partial matching of words into account has a great potential to improve sentence retrieval, especially when taking into account how little information a sentence contains and that every clue about the relevance of the sentence can be precious. The remainder of this paper is organized as follows. Previous work is shown in Section 2. The research objectives are presented in Section 3. New methods, experiments, and results are presented in Section 4 and Section 5, respectively, and the conclusion is given in Section 6.

2. Previous Research

Sentence retrieval is similar to document retrieval, and sentence retrieval methods are usually simple adaptations of document retrieval methods where sentences are treated as documents [6]. The sentence retrieval task consists of finding relevant sentences from a document base which has been given a query [6].

Generally, when it comes to information retrieval, the $\mathrm{TF}\text{-}\mathrm{IDF}$ method is still very much present today. For example, the authors of [12] presented a text document search system on distributed high-performance information systems, where initial document weighting was performed using the $\mathrm{TF}\text{-}\mathrm{IDF}$ method. The weighting of text by the $\mathrm{TF}\text{-}\mathrm{IDF}$ method, or the assignment of weight to each linguistic concept in comments from social networks, has also been described by the authors of [13].

The authors of [14] presented two different techniques ($\mathrm{BM}25$ and $\mathrm{TF}\text{-}\mathrm{IDF}$) to extract the keywords from data collection using Twitter. $\mathrm{TF}\text{-}\mathrm{IDF}$ has also been also used for novelty detection in news events [15]. $\mathrm{TF}\text{-}\mathrm{IDF}$ is widely used for text pre-processing and feature engineering [13]. There have also been attempts to outperform $\mathrm{TF}\text{-}\mathrm{IDF}$. For example, the authors of [16] presented a Phrase-Based document similarity, which effectively outperformed term-based $\mathrm{TF}\text{-}\mathrm{IDF}$. The authors of [17] proposed a refined $\mathrm{TF}\text{-}\mathrm{IDF}$ method, called $\mathrm{TA}\mathrm{TF}\text{-}\mathrm{IDF}$, for calculating the weights of hot terms. The vector-space model is one of the most commonly used models for documents and sentence retrieval [2]. The authors of [18] proposed a method of analyzing patent texts based on the vector space model, with the features of patent texts being excluded. Using the proposed algorithm, the authors of [19] calculated the distance between document and topics, and then each document was represented as a vector.

When it comes to sentence retrieval, $\mathrm{TF}\text{-}\mathrm{ISF}$ (the sentence retrieval variation of the $\mathrm{TF}\text{-}\mathrm{IDF}$ method) is one of the first and most widely used methods for sentence retrieval [5]. $\mathrm{TF}\text{-}\mathrm{ISF}$ has been shown to outperform other methods, like BM25-based methods and methods based on language modeling [5,20]. The sentences are represented as a collection of unique words with the weights of words that appear in the selected sentence [21]. The second method popularly used with sentence retrieval is query likelihood [2,6]. There have been multiple attempts to improve the standard $\mathrm{TF}\text{-}\mathrm{ISF}$ method, which include analyzing the context in the form of a document and the previous, following, or current sentences [6]. The authors of [22] analyzed the effectiveness of contextual information for answer sentence selection.

In our research, we adapted tree standard retrieval models, $\mathrm{TF}\text{-}\mathrm{ISF}$, $\mathrm{BM}25$ and the language modeling-based method, to improve sentence retrieval using the partial matching of words.

3. Research Objective

The aim of the research was to determine whether and to what extent the partial matching of words (terms between the query $q$ and the sentence $s$) influences the performance of methods for sentence retrieval. The influence of partial matching of words were presented by experimental results on three ranking models: $\mathrm{TF}\text{-}\mathrm{ISF},\mathrm{BM}25$ and the language modeling-based method. Sequence similarity is a technique which allows matching of terms that are similar but not identical. By testing sequence similarity, we intend to add some weight to the question of whether or not it is generally profitable to use partial matching of terms for sentence retrieval.

In large, our research hypothesis was that the partial matching of words improves of sentence retrieval methods.

4. Partial Match of Terms Using Sequence Similarity

The bulk of sentence retrieval methods proposed in the literature are adaptations of standard retrieval models, such as $\mathrm{TF}\text{-}\mathrm{IDF}$, $\mathrm{BM}25$, the language modeling-based method, where the sentence is the unit of retrieval [6].

In this work, we showed sentence retrieval using sequence similarity and presented the experimental results on three ranking models: $\mathrm{TF}\text{-}\mathrm{ISF}$ (based on vector space model), $\mathrm{BM}25$, and the language modeling-based method. All three models were adapted in such a way to allow us to test the partial matching of terms.

4.1. $TF\text{-}ISF$ Method with Sentence Retrieval

One of the first and most successful methods for sentence retrieval is the $\mathrm{TF}\text{-}\mathrm{ISF}$ defined as [4,5,23]:

$$\mathrm{TF}\text{-}\mathrm{ISF}\left(s,q\right)={\displaystyle \sum }_{t\in q}\log \left(t{f}_{t,q}+1\right)\log \left(t{f}_{t,s}+1\right)\log \left(\frac{n+1}{0.5+s{f}_t}\right)$$

(1)

The $\mathrm{TF}\text{-}\mathrm{ISF}$ method assess relevance of sentence $s$ with regard to query $q$, where

• $s{f}_t$ is the number of sentences in which the term $t$ appears,
• $n$ is the number of sentences in the collection,
• $t{f}_{t,q}$ is number of appearances of the term $t$ in a query $q$, and
• $t{f}_{t,s}$ is number of appearances of the term $t$ in a sentence $s$.

4.2. $BM25$ Model with Sentence Retrieval

The $\mathrm{BM}25$ model uses document ranking, and this model can also be used for sentence retrieval. The ranking function of the $\mathrm{BM}25$ method used to sentence retrieval is defined as [6]:

$$\mathrm{BM}25\left(s,q\right)={\displaystyle \sum }_{t\in q}log\frac{N-sf\left(t\right)+0.5}{sf\left(t\right)+0.5}·\frac{\left(k_1+1\right)c\left(t,s\right)}{k1\left[\left(1-b\right)+b\frac{\left|s\right|}{avsl}\right]+c\left(t,s\right)}·\frac{\left(k_3+1\right)c\left(t,q\right)}{k_3+c\left(t,q\right)}$$

(2)

where

• $N$ is the number of sentences in the collection,
• $sf\left(t\right)$ is the number of sentences in which the term $t$ appears,
• $c\left(t,s\right)$ is the number of appearances of the term $t$ in a sentence $s$,
• $c\left(t,q\right)$ is the number of appearances of the term $t$ in a query $q$,
• $\left|s\right|$ is the sentence length,
• $avsl$ is the average sentence length, and
• $k_1$, $k_3$, and $b$ are the adjustment parameters.

4.3. Sentence Retrieval with Language Model ($LM$)

The language modeling-based method (language model) for document retrieval can be applied analogously to sentence retrieval. The probability of a query $q$ given the sentence $s$ can then be estimated using the standard $\mathrm{LM}$ approach [6]:

$$\mathrm{LM}\left(q,s\right)={\displaystyle \prod }_{t\in q}P{\left(t|s\right)}^{c\left(t,q\right)}$$

(3)

$$P\left(t|s\right)=\frac{c\left(t,s\right)}{\left|s\right|}$$

(4)

where

• $c\left(t,s\right)$ is number of appearances of the term $t$ in a sentence $s$, and
• $\left|s\right|$ is the sentence length.

One of the most commonly used methods when it comes to sentence retrieval is Dirichlet smoothing, which, when applied to Method (3), gives:

$$\mathrm{LM}\left(q,s\right)={\displaystyle \prod }_{t\in q}\frac{c\left(t,s\right)+\mu P\left(t\right)}{\left|s\right|+\mu }$$

(5)

where

• $c\left(t,s\right)$ is the number of appearances of the term $t$ in a sentence $s$,
• $\left|s\right|$ is the sentence length,
• $\mu$ is the parameter that control the amount of smoothing, and
• $P\left(t\right)$ can be calculated using the maximum likelihood estimator of the term in a large collection: $p\left(t, C\right)$ (where $C$ is the collection) [6].

In the previous literature, previously presented ranking functions were combined with data pre-processing and stop word removal. Removing stop words is generally considered to be useful, since stop words do not contain any information [24].

There are several methods for removing stop words, which have been presented by the authors of [25]. Some papers [26] have also proposed time efficient methods. The method we used in this paper is based on a previously compiled list of words. The performance of functions was tested in its basic form with stop word removal.

4.4. Sentence Retrieval Using Sequence Similarity

When it comes to the effect of sequence similarity on sentence retrieval, we made use of the contextual similarity functions presented by the authors of [27]. This procedure enabled us to match of terms that were not identical but similar. Through analyzing the equation for the assessment of the common context by the authors of [27], we concluded that the same analogy could be used to find out if a certain term from query $q$ and a certain term from sentence $s$ share a common subsequence.

In Reference [27], the formula $\mathsf{\delta }\left(N,M\right)$ determines the appearance of subsequence N in a sequence M. We can define the formula $\mathsf{\delta }\left(q,s\right)$ analogous to the formula from Reference [27], with query $q$ (instead of subsequence in [27]) and sentence $s$ (instead of sequence M in [27]) as follows:

$$\mathsf{\delta }\left(q,s\right)={\displaystyle \sum }_{i=1}^{\left|q\right|}{\displaystyle \sum }_{j=i}^{\left|q\right|}\left|q_{ij}\cap s\right|$$

(6)

where

• $q_{ij}$ presents subsequence of sequence $s$.

If sequence $s$ does not contain the subsequence $q_{ij}$, there is no need to check for $qi\left(j+1\right)$ [27].

When the sequence $q$ is a subsequence of $s$, the $\mathsf{\delta }\left(q,s\right)$ = 1.

Furthermore, we have to extend Formula (6) to include the normalization parameter $T_{\sigma }$ that is given in work [27]. This solves the measurement problem that appears when larger sequences that have more subsequences are more similar than the sequences with shorter lengths, which is not correct [27]. In other words, $T_{\sigma }$ is the coefficient which represents the total score that can be achieved under the assumption that the first sequence is a proper subsequence of the second, or N ⊆ M [27]. After including the normalization parameter $T_{\sigma }$, we obtained:

$$\mathsf{\delta }\left(q,s\right)=\frac{1}{T_{\sigma }} {\displaystyle \sum }_{i=1}^{\left|q\right|}{\displaystyle \sum }_{j=i}^{\left|q\right|}\left|q_{ij}\cap s\right|$$

(7)

where

• $T_{\sigma }$ is the normalization parameter.

The importance of a word depends on its frequency inside its sentence [28] or the number of times a particular word occurs in the sentence [29]. However, the exact matching of terms was used, and the partial matching where words are similar but not identical was not considered. The assumption is that instead of the total matching, the existing methods for sentence retrieval can be improved through partial matching between subsequences and sequences, or in other words, between the term from the query and the term from the sentence.

In Equations (1), (2), and (5), $t{f}_{t,s}$ and $c\left(t,s\right)$ represent the number of appearances of term $t$ in the sentence $s$. Here, too, the exact matching of terms was used.

If instead of using the parameter $t{f}_{t,s}$ in the method $\mathrm{TF}\text{-}\mathrm{ISF}\left(s,q\right)$, and $c\left(t,s\right)$ in method $\mathrm{BM}25\left(s,q\right)$ and $\mathrm{LM}\left(q,s\right)$, we define the parameters $t{f}_{t,s \left(partial\right)}$ and $c{\left(t,s\right)}_{\left(partial\right)}$ which are also defined using the method $\mathsf{\delta }\left(q,s\right)$, we can define $t{f}_{t,s \left(partial\right)}$ and $c{\left(t,s\right)}_{\left(partial\right)}$ parameters as follows:

$$sim{\left(t,s\right)}_{partial}=\frac{1}{T_{\sigma }} {\displaystyle \sum }_{i=1}^{\left|\mathrm{t}\right|}{\displaystyle \sum }_{j=i}^{\left|\mathrm{t}\right|}\left|t_{ij}\cap s\right|$$

(8)

where

• $t_{ij}$ presents subsequence of sequence $s$ (term $t$ from the query as a subsequence of term from the sentence $s$ as a sequence.

To include the partial matching of terms, we took all defined ranking functions and replaced $t{f}_{t,s}$ and $c\left(t,s\right)$ with $sim{\left(t,s\right)}_{partial}$, and defined a new ranking functions for all three ranking models.

We further optimized all of the formulas to only consider terms that already appeared in both the query and sentence or $t\in q\cap s$. The assumption is that if we have a minimum of one match between the query and the sentence, it is more probable that additional matches in other terms between the query and the sentence could be found using the sequence similarity.

New ranking functions have been defined in their final form, which assesses the relevancy of the sentences regarding the query $q$, and considers the partial match of the term $t$ from the query in relation to the terms from the sentence (Equations (9)–(11)):

$${\mathrm{TF}\text{-}\mathrm{ISF}}_{partia{l}_{\left(t,s\right)}}\left(s,q\right)={\displaystyle \sum }_{t\in q\cap s}\log \left(t{f}_{t,q}+1\right)\log \left(sim{\left(t,s\right)}_{partial}+1\right)\log \left(\frac{n+1}{0.5+s{f}_t}\right)$$

(9)

$${\mathrm{BM}25}_{partia{l}_{\left(t,s\right)}}\left(s,q\right)={\displaystyle \sum }_{t\in q\cap s}log\frac{N-sf\left(t\right)+0,5}{sf\left(t\right)+0,5}·\frac{\left(k_1+1\right) sim{\left(t,s\right)}_{partial}}{k1\left[\left(1-b\right)+b\frac{\left|s\right|}{avsl}\right]+sim{\left(t,s\right)}_{partial}}·\frac{\left(k_3+1\right)c\left(t,q\right)}{k_3+c\left(t,q\right)}$$

(10)

$${\mathrm{LM}}_{partia{l}_{\left(t,s\right)}}\left(q,s\right)={\displaystyle \prod }_{t\in q\cap s}\frac{sim{\left(t,s\right)}_{partial}+\mu P\left(t\right)}{\left|s\right|+\mu }$$

(11)

where

• $t\in q\cap s$ is the postulate that only terms that are in the query and in the sentence are considered. In this case, there was a minimum of at least one match of the terms from the query and from the sentence.

To repeat the point of the new ranking functions, it considers the total match of the query term and sentence term, as defined by the parameter $t{f}_{t,s}$ and $c\left(t,s\right)$ in the ranking functions $\mathrm{TF}\text{-}\mathrm{ISF}\left(s,q\right)$, $\mathrm{BM}25\left(s,q\right),$ and $\mathrm{LM}\left(q,s\right)$.

The new ranking functions also consider additional appearances of terms from the query as the subsequence of terms from the sentence.

We denoted the new methods and their ranking function as shown in the Table 1.

Table 1. Overview of all sentence retrieval methods tested in this paper.

Method	Ranking Function
$\mathrm{TF}\text{-}\mathrm{ISF}$	$\mathrm{TF}\text{-}\mathrm{ISF}\left(s,q\right)$
${\mathrm{TF}\text{-}\mathrm{ISF}}_{part}$	${\mathrm{TF}\text{-}\mathrm{ISF}}_{partia{l}_{\left(t,s\right)}}\left(s,q\right)$
$\mathrm{BM}25$	$\mathrm{BM}25\left(s,q\right)$
${\mathrm{BM}25}_{part}$	${\mathrm{BM}25}_{partia{l}_{\left(t,s\right)}}\left(s,q\right)$
$\mathrm{LM}$	$\mathrm{LM}\left(q,s\right)$
${\mathrm{LM}}_{part}$	${\mathrm{LM}}_{partia{l}_{\left(t,s\right)}}\left(q,s\right)$

5. Experiments and Results

The experiment was conducted using data from the novelty tracks of the Text Retrieval Conference (TREC). There were three TREC novelty tracks in the years from 2002 to 2004: TREC 2002, TREC 2003, and TREC 2004 [9,10,11]. The task was novelty detection, which consists of two subtasks: Finding relevant sentences and finding novel sentences. Our experiment was entirely focused on sentence retrieval, which represents the first task of novelty detection. Three data collections were used, each consisting of 50 topics (queries) and 25 documents per topic, with multiple sentences (Table 2) [30].

Table 2. Description of dataset characteristics.

Name of the Collection	Number of Topics (Queries)	Number of Documents per Topic	Number of Sentences
TREC 2002	50	25	57,792
TREC 2003	50	25	39,820
TREC 2004	50	25	52,447

When it comes to TREC topics, it must be emphasized that each one has a title, a description, and narrative. These three parts represent three versions of the same query but with different lengths. The title is the shortest query and narrative the longest. In our tests, we used the shortest version called “title.” Figure 1 depicts the title of topic N1 from TREC 2003.

Figure 1. Example of query from Text Retrieval Conference (TREC) 2003 novelty track.

Every query was executed on 25 documents. Each of the documents consisted of multiple sentences as described in Table 2. Each sentence was marked with a beginning and ending tag. Each sentence had a number. Figure 2 shows an extract form document NYT19980629.0465 with sentence number 11.

Figure 2. Example of the sentence part within the document from TREC 2003 novelty track.

Each TREC data collection also contains a list of relevant sentences. Figure 3 depicts an excerpt from the list of relevant sentences of TREC 2003.

Figure 3. An excerpt in the dataset that contains a list of relevant sentences.

The marked line in Figure 3 defines sentence 11 from the document NYT19980629.0465 as relevant to the topic (query) N1 (“partial birth abortion ban”).

To test whether the new methods provide better sentence retrieval results than the existing methods, we compared the performances of the new methods in relation to the existing methods (baseline methods) using the following standard measures: P@10, the MAP, and the R-precision [1,6].

The precision at x or P@x can be defined as:

$$P@x\left(q_j\right)=\frac{\begin{array}{c}\mathrm{number} \mathrm{of} \mathrm{relevant} \mathrm{sentences} \\ \mathrm{within} \mathrm{top} x \mathrm{retrieved}\end{array}}{x}$$

(12)

The P@10 values shown in this paper refer to average P@10 for 50 queries [30].

R-precision can be defined as [1]:

$$R\text{-}precision=\frac{r}{\left|Rel\right|}$$

(13)

where

• |Rel| is the number of relevant sentences to the query, and
• r is the number of relevant sentences in top |Rel| sentences of the result.

The R-precision values shown in this paper are (analogous to P@10) averages for 50 queries [30].

The Mean Average Precision and R-precision gave similar results and were used to test high recall. High recall means it is more important to find all of the relevant sentences, even if it means searching through many sentences including many that are nonrelevant. Meanwhile, P@10 is used for testing precision [30].

In terms of information retrieval, precision means it is more important to get only relevant sentences than finding all of the relevant sentence [30].

To compare the difference between methods, we used a two-tailed paired t-test with a significance level of α = 0.05 [2,4,6]. The results of our tests are presented in tabular form.

Statistically significant differences in relation to the baseline methods are marked with a (*).

In each of the tested methods, we used stop word removal as pre-processing step.

Table 3 shows the results of our tests on two different versions of the $\mathrm{TF}\text{-}\mathrm{ISF}$ method using TRECs 2002, 2003, and 2004.

Table 3. Test results of methods $\mathrm{TF}\text{-}\mathrm{ISF}$ and ${\mathrm{TF}\text{-}\mathrm{ISF}}_{part}$.

Data Collection	Measures	$\mathbf{TF}\text{-}\mathbf{ISF}$	${\mathbf{TF}\text{-}\mathbf{ISF}}_{\mathit{p}\mathit{a}\mathit{r}\mathit{t}}$
TREC 2002	P@10	0.304	0.32
	MAP	0.196	* 0.204
	R-prec.	0.245	0.250
TREC 2003	P@10	0.692	0.714
	MAP	0.576	* 0.591
	R-prec.	0.547	* 0.560
TREC 2004	P@10	0.434	0.468
	MAP	0.324	* 0.335
	R-prec.	0.336	* 0.355

* Statistically significant differences in relation to the baseline methods.

.

Table 3 shows that the ${\mathrm{TF}\text{-}\mathrm{ISF}}_{part}$ methods provided better results and statistically significant differences in relation to the base $\mathrm{TF}\text{-}\mathrm{ISF}$ method when the MAP measures was used in all three collections, and R-prec. in TREC 2003 and TREC 2004 collection. Better results were not achieved when using the P@10 measures.

Table 4 shows the results of our tests on two different versions of the $\mathrm{BM}25$ model using TRECs 2002, 2003, and 2004.

Table 4. Test results of methods $\mathrm{BM}25$ and ${\mathrm{BM}25}_{part}$ ($k1$ = 1.5, $b$ = 0.75, $k3$ = 0).

Data Collection	Measures	$\mathbf{BM}$	${\mathbf{BM}}_{\mathit{p}\mathit{a}\mathit{r}\mathit{t}}$
TREC 2002	P@10	0.142	* 0.33
	MAP	0.105	* 0.209
	R-prec.	0.097	* 0.255
TREC 2003	P@10	0.628	* 0.75
	MAP	0.464	* 0.601
	R-prec.	0.4281	* 0.565
TREC 2004	P@10	0.366	* 0.472
	MAP	0.242	* 0.342
	R-prec.	0.236	* 0.363

* Statistically significant differences in relation to the baseline methods.

The parameters settings for the $\mathrm{BM}25$ model were $k1$ = 1.5, $b$ = 0.75, $k3$ = 0.

Table 4 shows that the ${\mathrm{BM}25}_{part}$ method (method using sequence similarity) provided better results and statistically significant differences in relation to the base method in all measures and collections.

Table 5 shows the results of our tests on two different version of the $\mathrm{LM}$ model using TREC collections from 2002, 2003, and 2004. The parameter settings for $\mathrm{LM}$ were $\mu$ = 100.

Table 5. Test results of methods $\mathrm{LM}$ and ${\mathrm{LM}}_{part}$ ($\mu$ = 100).

Data Collection	Measures	$\mathbf{LM}$	${\mathbf{LM}}_{\mathit{p}\mathit{a}\mathit{r}\mathit{t}}$
TREC 2002	P@10	0.268	* 0.356
	MAP	0.170	* 0.207
	R-prec.	0.215	* 0.250
TREC 2003	P@10	0.71	0.7
	MAP	0.528	* 0.597
	R-prec.	0.501	* 0.567
TREC 2004	P@10	0.388	* 0.458
	MAP	0.287	* 0.334
	R-prec.	0.306	* 0.355

* Statistically significant differences in relation to the baseline methods.

Table 5 shows that the ${\mathrm{LM}}_{part}$ method, as well the ${\mathrm{BM}25}_{part}$ method, provided better results and statistically significant differences in relation to the base method in all measures and collections, except for the P@10 measure for TREC 2003 collection.

6. Conclusions

In this paper, we thoroughly tested sentence retrieval with sequence similarity, which allowed us to match words that were similar but not identical. We tested sentence retrieval methods with the partial matching of terms using TREC data. We adapted $\mathrm{TF}\text{-}\mathrm{ISF}$, $\mathrm{BM}25$ and the language modeling-based method to test the partial matching of terms using sequence similarity.

We found out that the partial matching of terms using sequence similarity can benefit sentence retrieval in all three tested collection. We showed the benefits of partial matching using sequence similarity through statistically significant better results.

The reason for the better position of the sentence when we used adapted methods using sequence similarity is the additional matching of terms between the query $q$ and the sentence $s$.

We conclude that partial matching of words is beneficial when combined with sentence retrieval. However, we did not analyze whether some nonrelevant sentences were falsely high ranked. Therefore, future research will include a thorough analyses of the effect of the partial matching of words on sentence retrieval. Future research will also include experiments using pre-processing methods, such as stemming and lemmatization or some other technique.