Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper presents a comprehensive and unique dataset of Telegram channels in the field of economics, with almost 3 million messages, which makes it extremely valuable for research. A special advantage is the enrichment of textual data with financial market indicators, which allows for deeper semantic and contextual analysis. Furthermore, the paper introduces an original methodology of synthetic mixing of tagged channels, useful for research on continuous authentication, authorship and anomaly detection, with a high level of transparency and repeatability thanks to publicly available data and scripts.

The paper does not provide empirical validation of synthetically generated tags, so it is not clear to what extent they realistically reflect real scenarios of authorship change or channel compromise. Also, a discussion on bias and representativeness of the data set is missing, as well as a clear threat model in the context of continuous authentication. In addition, there is a terminological inconsistency (continuous vs. extended authentication) and a limited discussion on the ethical aspects of profiling and surveillance.

Author Response

We thank the Reviewer for the thorough and constructive feedback, which has helped us significantly improve the manuscript. We have carefully addressed all comments, as detailed below. All changes have been incorporated into the revised manuscript, with major updates highlighted for the Reviewer's convenience.

“The paper does not provide empirical validation of synthetically generated tags, so it is not clear to what extent they realistically reflect real scenarios of authorship change or channel compromise. “ 

We agree with the importance of empirical verification. In response to these comments, we added a section  3.3 Threat Model, in Background which describes three types of threats related to the compromise of a Telegram channel. The most difficult to detect is related to a change of author, who begins to publish messages on the same topic, imitating the style of the previous author. 

Also, in the Materials and Methods section, a new subsection, “4.4. Empirical Validation of Synthetic Datasets,” has been added. Thirty type 1 datasets and 30 type 2 datasets were used for validation. Considering that synthetic datasets are formed in a common pool of topics, and that anomalous messages are selected so that the average number of words is close, we potentially bring the style of messages closer together. Three complementary metrics—cosine similarity and Fréchet distance—were used to assess their semantic proximity and the degree of overlap between normal messages (label 0) and anomalous messages (label 1). The experimental results confirm that the synthetic data corresponds to threat scenarios that simulate channel compromise and authorship change (both with and without style imitation) within a single topic. The full results and scripts for reproducing the experiment are available in our GitHub repository.

Also, a discussion on bias and representativeness of the data set is missing, as well as a clear threat model in the context of continuous authentication

We have expanded Section 4.1(“ Source of data on collected channels”) and added a specific discussion of biases and representativeness. The main additions include:

1. Source-related limitation (Platform Bias): We explicitly state that our dataset inherits the sampling bias inherent in its source, the TGStat catalog. It represents public, indexable, and popular channels in the “Economy” category. As a result, the dataset may underrepresent or completely exclude:

-Private or hidden channels.

-New or inactive channels.

-Channels that deliberately avoid analytics systems.

2. Thematic & Language Bias: We note that the dataset focuses exclusively on Russian-language economic discourse. 
3. Temporal Bias: The data was collected over a specific period of time, which may not reflect the evolution of language or style, as well as long-term changes in the composition and behavior of the channels' audience.
4. Purpose and scope of the dataset: We emphasize that the dataset is primarily intended as a tool for methodological research and comparative analysis (benchmarking) in stylometry and prolonged authentication tasks, rather than as a comprehensive sample for sociolinguistic or economic research. 

Threat Model for continuous authentication. We have added a new subsection, “2.4. Threat Model for Continuous Authentication,” and described in above. 

1. Source-related limitation (Platform Bias): We explicitly state that our dataset inherits the sampling bias inherent in its source, the TGStat catalog. It represents public, indexable, and popular channels in the “Economy” category. As a result, the dataset may underrepresent or completely exclude:

-Private or hidden channels.

-New or inactive channels.

-Channels that deliberately avoid analytics systems.

2. Thematic & Language Bias: We note that the dataset focuses exclusively on Russian-language economic discourse. 
3. Temporal Bias: The data was collected over a specific period of time, which may not reflect the evolution of language or style, as well as long-term changes in the composition and behavior of the channels' audience.
4. Purpose and scope of the dataset: We emphasize that the dataset is primarily intended as a tool for methodological research and comparative analysis (benchmarking) in stylometry and prolonged authentication tasks, rather than as a comprehensive sample for sociolinguistic or economic research. 

Threat Model for continuous authentication. We have added a new subsection, “2.4. Threat Model for Continuous Authentication,” added in 3.3 and described in the previous remark.

In addition, there is a terminological inconsistency (continuous vs. extended authentication) 

This problem was resolved by carefully checking the manuscript and standardizing the terminology. Now, the term “continuous authentication” has been adopted as the standard term and is used consistently throughout the text.

 limited discussion on the ethical aspects of profiling and surveillance.

We added a new separate section to the manuscript: 3.4. Ethical considerations.  Only public channels were used in the dataset, i.e., not groups, without the possibility of establishing a correspondence between a message and the user-author identifier. Thus, the dataset strikes a balance between usefulness for research and ethical responsibility, allowing the study of authorship, style, and message dynamics without revealing private information. The dataset is distributed openly, along with the software and algorithmic basis for its collection and the formation of synthetic datasets to support reproducibility and further research in the field of continuous authentication and text analysis in social media.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors should:
1.- Explain in detail the main contributions of the work in the introduction section.
2.- Include a related work section where the authors can describe previous work in the field. This section contributes to explaining the relevance of your contribution and describes the gap that you want to resolve.
3.- Unify sections 2, 3, and 4 into one section called Background.
4.- Rename section 5 to Materials and Methods.
5.- Review each paragraph to delete comments, for example: the method allows you.
6.- Explain with more detail in the new section, materials and methods, the analysis used in the article, and why.
7.- Rewrite the conclusion section because the current version points to economic analysis by channels in Telegram.
 
I have the following questions:
1.- How does the proposal solve the problem of continuous authentication?
2.- How to integrate the economic data with the original dataset? The authors need to explain in more detail.
3.- How were the accounts, profiles, or channels chosen to include the economic data? And how to avoid modification or alteration in the analysis?
4.- Which information provides evidence of the benefit of your proposal to validate a process of continuous authentication in Telegram? The current version includes descriptive information about the dataset.
5.- Which evidence appears in Table 12 that explains how to resolve the problem of continuous authentication?
6.- How does the current version of the conclusion section summarize or explain the solution for continuous authentication?

Author Response

We thank the Reviewer for the thorough and constructive feedback, which has helped us significantly improve the manuscript. We have carefully addressed all comments, as detailed below. All changes have been incorporated into the revised manuscript, with major updates highlighted for the Reviewer's convenience.

1.- Explain in detail the main contributions of the work in the introduction section.

We have thoroughly revised the Introduction section (Section 1) to provide a clear, concise, and structured description of our main contributions, going beyond a simple descriptive list and emphasizing the key methodological and resource value of the work.

The contribution is now clearly summarized as follows:

1. An open dataset of Russian-language economic Telegram channels selected through the TGStat catalog has been created.
2. A method for generating synthetic data that simulates author changes while maintaining the subject matter has been developed to test continuous authentication models.
3. Messages have been enriched with financial indicators, allowing us to study the impact of the market on content and use this data as indicators for continuous authentication.
4. The dataset complies with ethical standards, as it consists of public channels where messages cannot be linked to user IDs.
5. Reproducibility is ensured: all data and scripts are published, allowing experiments to be repeated and research to be developed.

We believe that this wording better reflects the scientific novelty and practical usefulness of our work. Detailed descriptions of each contribution are provided in the relevant sections of the manuscript.

2 Include a related work section where the authors can describe previous work in the field. This section contributes to explaining the relevance of your contribution and describes the gap that you want to resolve

We added this section, in which we reviewed works devoted to Telegram datasets and research on continuous authentication (section 3.2.). The key point is that we clearly define the research gap that our work addresses, stating:

that we are not just creating a dataset that fills the gap in previous research, in which Telegram datasets  were mainly focused on general content or socio-political topics. Russian-language content, including economic discourse, has been represented only fragmentarily. Moreover, none of the known datasets provide synthetic data specifically created for continuous authentication tasks. Thus, we are providing not just another dataset, but a toolkit for creating reproducible benchmarks that allow for the exploration of prolonged authentication methods in realistic but ethically safe conditions, without the need for access to private data.

3.- Unify sections 2, 3, and 4 into one section called Background.

Accordingly, we have merged the previous Sections 2, 3, and 4 into a single, cohesive section titled "Background".

4.- Rename section 5 to Materials and Methods

We renamed section 5 as suggested.

5. Review each paragraph to delete comments, for example: the method allows you.

We have performed a thorough stylistic edit of the entire manuscript as suggested. All informal commentary and subjective phrasing (such as "the method allows you", "we can see that") have been removed. The text has been revised to maintain a formal, objective, and concise academic style throughout.

6. Explain with more detail in the new section, materials and methods, the analysis used in the article, and why

We have tried to take this comment into account and describe the methods used in as much detail as possible, justifying their selection. In particular, we justify the representativeness of the dataset, indicate its limitations, and justify the choice of the core for building the dataset—the primary channels. We justify the choice of the document-oriented MongoDB model for the dataset. An additional subsection on synthetic datasets has been added, which describes in detail why such a percentage of anomalies is introduced. Another section, 4.4, has also been added, which performs empirical validation of synthetic datasets and links them to the threat model. All results of the experimental analysis are also presented on GitHub.

7. Rewrite the conclusion section because the current version points to economic analysis by channels in Telegram.

We have rewritten the “Conclusion” section to address this shortcoming. In the revised version, the focus has shifted from economic analysis to the methodological contribution and research value of our work. The new conclusion  now clearly summarizes our main achievements:

1.Providing a structured dataset and toolkit for stylometry. We have collected and structured a dataset of Russian-language Telegram channels on economic topics, enriched with metadata and synchronized financial indicators. Its structure allows us to study linguistic patterns in relation to the external context, maintaining a balance between research value and ethical requirements.

2. Introduction of a methodology for generating realistic synthetic data for continuous authentication tasks.

3. Ensuring reproducibility and creating a benchmark for future research in the field of social network security.

Thus, the emphasis is now on the value of the dataset and methods for machine learning and computational linguistics communities, rather than on economic insights.

I have the following questions:

1.- How does the proposal solve the problem of continuous authentication?

The work does not solve the problem of extended authentication, but presents data and a methodology for forming datasets, taking into account the proposed threat model, so that anomalous data is semantically and stylistically similar. 

2.- How to integrate the economic data with the original dataset? The authors need to explain in more detail.

The messages stored in the dataset have attributes with the economic data.
If a new dataset is created, firstly messages are downloaded, then they are updated with the economic data. The economic data may be downloaded in CSV files from a source like Marketwatch. 

3.- How were the accounts, profiles, or channels chosen to include the economic data? And how to avoid modification or alteration in the analysis?

The initial list of channels is created from the economy category of the TGStat catalog. The initial list has been created within a few seconds. The downloading of messages process took several days. Due to the limitations of the Telegram API, it was not possible to make an immediate snapshot. After the data is downloaded, it is stored in the dataset, while in the source (Telegram) authors may edit or delete messages or entire channels.

4.- Which information provides evidence of the benefit of your proposal to validate a process of continuous authentication in Telegram? The current version includes descriptive information about the dataset.

Our work provides methodological, not applied, validation. The key evidence for the utility of our proposal lies not in building an authentication model, but in empirically validating that our synthetic datasets possess the precise properties required to rigorously test such models.

This evidence is now presented in the new Section 4.4 ("Empirical Validation of Synthetic Datasets").Our validation focused on two critical properties:

   1. High Semantic Similarity sing sentence transformers (rubert-tiny2), we calculated the cosine similarity between message segments from different authors within the synthetic mixtures. 

    2. Analysis of Distributional Similarity via Fréchet Distance: To rigorously quantify the stylistic proximity between the source channels in our synthetic mixtures, we employed the Fréchet Distance between multivariate normal distributions fitted to their sentence embeddings.5.- Which evidence appears in Table 12 that explains how to resolve the problem of continuous authentication?

The table 12 (table 14 in the new manuscript version) proves the presence of channels, messages of which correlate with volatility of the exchange instruments. This fact may be used as an additional behavior metric of the channel author.

6.- How does the current version of the conclusion section summarize or explain the solution for continuous authentication?

The revised conclusion explains that our work solves the problem of continuous authentication by providing the necessary methodological infrastructure rather than a ready-made system. The solution consists of three parts: 1) a realistic dataset for studying style, 2) a method for generating synthetic data that simulates changes in authorship, and 3) an open benchmark for rigorous testing and comparison of algorithms. Thus, the conclusion states that our solution is not to create a single best model, but to provide the necessary infrastructure (data, methods, benchmarks) for the systematic development and objective evaluation of such models.

 

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The current version of the article includes all the recommendations and suggestions indicated previously; however, after reading the following response, “The revised conclusion explains that our work solves the problem of continuous authentication by providing the necessary methodological infrastructure rather than a ready-made system. The solution consists of three parts: 1) a realistic dataset for studying style, 2) a method for generating synthetic data that simulates changes in authorship, and 3) an open benchmark for rigorous testing and comparison of algorithms. Thus, the conclusion states that our solution is not to create a single best model, but to provide the necessary infrastructure (data, methods, benchmarks) for the systematic development and objective evaluation of such models.”, I have a final comment that I hope the authors consider before publication:

1.- Update the title of the document focus to “1) a realistic dataset for studying style, 2) a method for generating synthetic data that simulates changes in authorship, and 3) an open benchmark for rigorous testing and comparison of algorithms. Thus, the conclusion states that our solution is not to create a single best model, but to provide the necessary infrastructure (data, methods, benchmarks) for the systematic development and objective evaluation of such models”. 

In addition to my previous comment, the authors updated the introduction, including five contributions. In consequence, the five contributions point not specifically to the current version of the title but represent the real contribution. 

 

Comments on the Quality of English Language

The current version is good.