Employing AI for Better Access to Justice: An Automatic Text-to-Video Linking Tool for UK Supreme Court Hearings

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This study uses NLP technology to address the problems existing in the court litigation process. The work done is to build an automated tool to associate the content of text judgments with hearing videos. There have been many similar works on text analysis at the legal level using NLP technology, but the content of hearings, due to their long duration and large amount of information, requires manual transcription, thus presenting a challenge in this aspect. From the experiments and results conducted by the authors, they have solved the problem initially proposed and filled the gap in text analysis of hearings using AI technology. To make the article more complete, the authors can consider the following modifications:

1. At the end of the introduction section, list the contributions of this study in bullet points to enable readers to have a clearer understanding of the contributions made by the article.
2. This article provides very little description of Figure 2. It is suggested to add some explanatory notes on Figure 2 to facilitate readers' understanding.
3. Some of the references have publication dates that are too early. It is recommended to add some more recent literature to enhance the timeliness of the article.
4. In the discussion section of the article, the number of literature citations is relatively low. It is suggested that the author increase the citation of literature in this part, which is very necessary.
5. The authors only used GPT-3 as the embedding model. To demonstrate the advantages of this research, I suggest the authors add higher versions of GPT and compare them.
6. Each evaluation metric has its own calculation method and evaluation perspective. Adding an introduction to them will facilitate a more detailed analysis of the strengths and weaknesses of each comparison model.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper introduces an AI-driven platform to enhance public and professional access to UK Supreme Court (UKSC) proceedings through domain-customised Automatic Speech Recognition (ASR) and semantic Information Retrieval (IR). The system maps legal case paragraphs to relevant hearing video segments using customised GPT-based embeddings and a supervised classifier. It demonstrates meaningful gains in transcription accuracy, retrieval relevance, and annotation efficiency compared to baseline tools (e.g., Amazon Transcribe).

The research tackles a critical challenge in legal accessibility, bridging complex spoken judicial content with public-friendly digital interfaces.

Comments (To Be Addressed Before Publication)

1. Clarity and Focus in Abstract and Introduction

Abstract is overly descriptive; introduction lacks explicit objectives: Refactor abstract to clearly state motivation, methods, and quantitative results. Begin Introduction with a clear problem statement, followed by numbered research contributions.

2. Evaluation Metrics for IR and Classifier

Recall@15 is used, but lacks complementary precision or NDCG. Classifier gains are not tested for statistical significance. Add MAP or report significance testing (e.g., t-tests) for Table 6 results, and clarify evaluation protocol and annotator agreement.

3. Dataset Scale and Generalisation

Results focus on a small number of UKSC cases, limiting generalisation. Explicitly state the number of cases and segments used for IR evaluation. Comment on representativeness and legal diversity of chosen cases.

4. Embedding Customisation Needs More Transparency

Mathematical formulation is abstract; workflow unclear. Break down steps used to learn the transformation matrix. Explain threshold sweep logic and how generalisability was validated on unseen cases.

5. Data Augmentation Validation

Augmented data using text-davinci-002 may distort retrieval task realism. Include examples of paraphrased queries and note how many were expert-reviewed. Discuss risks of overfitting to synthetic phrasing.

6. Error Analysis Should Be Expanded

Only one error category is discussed. Categorise common IR failure modes (e.g., lexical ambiguity, domain drift). Include frequency counts or pie charts to visualise error types.

7. Ethics and Data Transparency

Use of public legal content raises potential copyright, consent, and bias concerns. Briefly mention data licensing, compliance with GDPR/FoI, and whether speakers were aware of AI-driven reuse. Also comment on biases in training data (e.g., gendered speech, regional accents).

Other Comments

1. Terminology

Ensure consistent use of “GPT-3” vs “GPT3” and clarify variant names like “GPT3+” (customised embeddings) in captions/text.

2. Typos and Grammar

Fix minor issues: “This is cause by” (line 64) → “caused by”.

3. PMI Thresholds

Justify your PMI>1 threshold for phrase detection with empirical rationale (e.g., comparison to PMI>2).

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

accept