Review Reports - TRust Your GENerator (TRYGEN): Enhancing Out-of-Model Scope Detection

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Out-of-model scope (OOMS) detection is critical for guaranteeing the trustworthiness of AI systems, particularly when dealing with input that deviates from the training set. This work suggests an approach that uses generative models to improve OOMS detection, with the goal of increasing the trustworthiness and resilience of AI applications in crucial areas. My suggestions to improve this paper:

1. The manuscript requires considerable proofreading because it has multiple grammatical errors and odd language. This drastically reduces the reading and comprehension of the information.

2. The uniqueness of the research is not properly explained. The authors must explicitly state what new knowledge or contributions this publication adds to the existing body of research.

3. The work lacks a complete examination of the scientific literature. There is insufficient discussion of similar work, and the few references provided are not adequately incorporated into the framework of the study. A full study and discussion of pertinent works is required to situate this work within the present state of the field.

4. The approach is not clearly described. The publication does not go into enough detail about how the research was conducted, making it difficult to judge the validity and trustworthiness of the findings. A full description of the experimental design, data collection, and analytic methods is required.

5. The study lacks a comprehensive overview of error estimation and a thorough examination of the results. The results section should be updated to incorporate a thorough assessment of the uncertainties and limits associated with the findings.

6. The paper's general organization does not adhere to scientific standards. The material flow is disconnected, and section organization has to be greatly changed to adhere to basic scientific writing principles.

7. The text is badly constructed and difficult to understand. Numerous instances of imprecise words and scattered thoughts interrupt the paper's logical flow. The authors must restructure the language to guarantee clarity and coherence.

8. The references are not properly formatted, and several appear unrelated to the topic. The writers should ensure that all references are relevant and properly referenced in accordance with the journal's rules.

Comments on the Quality of English Language

1. The manuscript requires considerable proofreading because it has multiple grammatical errors and odd language. This drastically reduces the reading and comprehension of the information.

Author Response

Comment 1: The manuscript requires considerable proofreading because it has multiple grammatical errors and odd language. This drastically reduces the reading and comprehension of the information.

Response 1: The manuscript was proofread, and grammatical errors and odd language were corrected.

Comment 2: The uniqueness of the research is not properly explained. The authors must explicitly state what new knowledge or contributions this publication adds to the existing body of research.

Response 2: The contribution and uniqueness of our approach are explained in the chapter on introduction and motivation.

Comment 3. The work lacks a complete examination of the scientific literature. There is insufficient discussion of similar work, and the few references provided are not adequately incorporated into the framework of the study. A full study and discussion of pertinent works is required to situate this work within the present state of the field.

Response 3: From the chapter 2 Related Work to the chapter 3 Out of Model Scope monitoring pipeline we discuss and compare other approaches to our novel approach. In the conclusion achieved results are highlighted and compared to similar work that other researchers did. We build upon the work of Guérin, J.; Delmas, K.; Ferreira, R.; Guiochet, J. Out-of-distribution detection is not all you need, which we cited and discussed thoroughly within the whole article.

Comment 4. The approach is not clearly described. The publication does not go into enough detail about how the research was conducted, making it difficult to judge the validity and trustworthiness of the findings. A full description of the experimental design, data collection, and analytic methods is required.

Response 4: The motivation of the method is described within the Introduction, experimental setup, data collection, and results are described on pages 7 – 17.

Comment 5. The study lacks a comprehensive overview of error estimation and a thorough examination of the results. The results section should be updated to incorporate a thorough assessment of the uncertainties and limits associated with the findings.

Response 5: Since we defined our problem on a closed set of images, namely dataset CIFAR-10, which is used as a common dataset to evaluate OOD and OMS problems by practitioners worldwide, we didn't incorporate any assessment of the uncertainties. The preconditions for our experiments are described on page 6.

Comment 6. The paper's general organization does not adhere to scientific standards. The material flow is disconnected, and section organization has to be greatly changed to adhere to basic scientific writing principles.

Response 6: The material flow was changed and adapted to provide a better flow.

Comment 7. The text is badly constructed and difficult to understand. Numerous instances of imprecise words and scattered thoughts interrupt the paper's logical flow. The authors must restructure the language to guarantee clarity and coherence.

Response 7: The clarity and coherence of the article were improved by the native speaker.

Comment 8. The references are not properly formatted, and several appear unrelated to the topic. The writers should ensure that all references are relevant and properly referenced in accordance with the journal's rules.

Response 8: References were formatted according to the journal's rules.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors' approach to designing an architecture capable of classifying Out of Model Scope inputs is both innovative and valuable. The proposed architecture integrates three main components: an encoder, a generator, and a monitor. The encoder, which can be any trained neural network, effectively captures the input features and maps them to a latent space. The generator then reconstructs the input using both the encoder's output and latent values from hidden layers, providing a robust representation of the original input. Finally, the monitor (M) assesses whether the reconstructed input is within the model's intended scope or falls outside it. This combination offers a comprehensive and adaptable solution for handling inputs, enhancing the model's reliability in real-world applications.

The paper provides a clear and thorough description of the encoder, including its construction and role in the overall architecture. The generator's design is also well-explained, with detailed information on its loss function and training process, which adds clarity to how it contributes to the generation of inputs from latent representations. However, the explanation of the monitor model remains insufficiently detailed. The paper lacks clarity on how the monitor is specifically designed, trained, or how it determines whether an input is out of model scope, leaving readers uncertain about this critical component of the proposed architecture. Further elaboration on the monitor's structure and decision-making criteria would greatly strengthen the overall understanding of the approach.

Please consider the next points :

Low level remarks :

- Start the Introduction numering with 1 instead of 0 (Line 17)

- In the inference Algorithm in page 7, the second line x-hat tales G Theta of z, E Theta given y hat. There is a repitition in parameters of G Theta function that takes z as parameter while is returned by E Thetat which is also given as a parameter. Please simplify.

Important remarks :

The autors described in equations 2 and 4 the monitor module that returns 1 or 0 according the probability density function (distribution of data). In subsection 3.2.4 page 10, the authors explained that the Monitor will be trained according by generating the OMS dataset, but there is no information related to the dataset generation or training process. Even if we have a dataset, please explain the link between the definition in equation 4 and the dataset : which garantee that OMS Model trainied on a built dataset will effectivelly returns 1 or 0 using the distribution of data ?

Author Response

Comment 1 - Start the Introduction numering with 1 instead of 0 (Line 17)

Response 1: was changed - thank you

Comment 2 - In the inference Algorithm in page 7, the second line x-hat tales G Theta of z, E Theta given y hat. There is a repitition in parameters of G Theta function that takes z as parameter while is returned by E Thetat which is also given as a parameter. Please simplify.

Response 2: was changed - thank you

Comment 3 - The autors described in equations 2 and 4 the monitor module that returns 1 or 0 according the probability density function (distribution of data). In subsection 3.2.4 page 10, the authors explained that the Monitor will be trained according by generating the OMS dataset, but there is no information related to the dataset generation or training process. Even if we have a dataset, please explain the link between the definition in equation 4 and the dataset : which garantee that OMS Model trainied on a built dataset will effectivelly returns 1 or 0 using the distribution of data ?

Response 3: We described how we trained Monitor as well as how we built the OMS dataset in the Chapter 4.1. In Model Scope and Out of Model Scope datasets and 4.2.4. Out of Model Scope Monitor. Equation 4 describes a functionality of OMS Monitor, where the aim is to build a mapping function that will for each sample from distribution X deliver a binary classification decision; in model scope or out of model scope. Since we train the OMS monitor on known IMS and OMS datasets we can use the binary cross entropy as loss function and so ensure that the model will return 1 or 0 based on information contained in the training data. Under equation 4 is written a very important condition about closed space X. Without this condition we would have no guarantee for future real application use.

Reviewer 3 Report

Comments and Suggestions for Authors

The problem of the research is 'Out of Model Scope' (OMS) detection, identifying all data points that fall outside the model scope. The authors propose an approach that combining the signal from three sources: the original input, latent feature space extracted by encoder, and the synthetic data from generator with latent feature space as input. The OMS pipeline consists four parts, an encoder, a generator, a latent space wrapper and a binary classifier. To show the superiority of the new approach, the authors constructed a IMS and OMS datasets based on CIFAR-10, and compared with previous methods on AUROC, AUPR and FPR95TPR.

Overall this paper is clear and well-organized. However, there are few concerns:
1. Table 8: It's great to have an ablation study on the three sources of data. From the paper, the ablation study were achieved by setting the corresponding input, [1], [2] or [3] to 0. I was wondering if the authors set the input to 0 in both training and testing, or just in testing? Instead of setting the testing part to 0, would that make more sense to build a classifier without the testing part?
2. Section 3.1: The OMS dataset consists covariate shifts and the modifications were applied until the Encoder failed to recognize otherwise correctly classified IMS samples. I am not sure if it's fair to compare the OMS method with OOD method on this set of data given the dataset is dependent on the encoder.
3. Detecting OOD (or OMS) on CIFAR-100 is generally harder than CIFAR-10. Was wondering if the authors have experiments on CIFAR-100.
4. Page 14 Line 442: FPR95TPR: probability that negative example is misclassified as positive when the true positive rate(TPR) is as high as 95%

Author Response

Comment 1. Table 8: It's great to have an ablation study on the three sources of data. From the paper, the ablation study were achieved by setting the corresponding input, [1], [2] or [3] to 0. I was wondering if the authors set the input to 0 in both training and testing, or just in testing? Instead of setting the testing part to 0, would that make more sense to build a classifier without the testing part?

Response 1: During the ablation study we set only the testing part to 0. It is unclear to us what advantages would it bring to set as well the training part to 0. But we agree that instead of using the word ablation study, we should call it sensitivity analysis. We change the naming in the whole manuscript. Thank you!

Comment 2. Section 3.1: The OMS dataset consists covariate shifts and the modifications were applied until the Encoder failed to recognize otherwise correctly classified IMS samples. I am not sure if it's fair to compare the OMS method with OOD method on this set of data given the dataset is dependent on the encoder.

Response 2: The OOD methods that were used, take the information either from the original image, of features space from the Encoder, or logits from the Encoder. Therefore comparison in this case is fair. The work which we build upon, Guérin, J.; Delmas, K.; Ferreira, R.; Guiochet, J. Out-of-distribution detection is not all you need, provided the same reasoning and we found the reasoning logical.

Comment 3. Detecting OOD (or OMS) on CIFAR-100 is generally harder than CIFAR-10. Was wondering if the authors have experiments on CIFAR-100.

Response 3: No, we didn't test it. This could be extended in the next article, where we would like to rather scale to a higher resolution of ImageNet and discuss more thoroughly the other possibilities of Generator.

Comment 4. Page 14 Line 442: FPR95TPR: probability that negative example is misclassified as positive when the true positive rate(TPR) is as high as 95%

Response 4: Reformulated – thank you

Reviewer 4 Report

Comments and Suggestions for Authors

The basic idea (as explained in Figure 1 a) is based on comparing the original sample with the sample reconstructed by the autoencoder. A Latent Wrapper supports the feature extractor, and the Monitor supports the sample comparison. This is a more robust architecture than the simple autoencoder. However, we still believe that the autoencoder trained on the training set shows the difference between the test sample and the reconstructed sample if the test sample may lead to misclassification. For me, the proposed approach trusts your feature extractor more than the autoencoder's generator part. We know that features extracted by autoencoder differ from features extracted within pattern recogniser. Features in the autoencoder are much more detailed, and it raised the question of whether the sample reconstructed not enough precisely will be classified incorrectly. Vice versa, adversarial attacks show that similar samples from a human vision point of view can lead to misclassification. It seems from Table 7 that the proposed architecture also has some problems with adversarial samples if they are not added to the training set.

In my opinion, anomaly detection (OMS detection) has to be tailored to the given pattern recogniser to avoid classifying samples that are far from the training samples. The decision about which samples are far depends on the images as well as on the pattern recogniser. Nevertheless, trying to detect patterns that need to be filtered before entering the pattern recogniser can also be solved universally without knowledge of the recogniser. Still, such a solution will be less accurate.

This problem can be demonstrated in MNIST vs SVHN images. Are handwritten numbers similar to street numbers? This experiment might extend the paper. In our experiments, an SVHN sample was detected as an anomaly with 25% probability, and it decreased to zero after adding SVHN to the training set. Our experiments were also based on trusting the generator, but we used the generator in GAN.

Formatting notes:

- Do not introduce abbreviations in the abstract.

- Do not use abbreviations in subtitles.

- Introduce abbreviations only once at the first occurrence.

Author Response

Comments 1:

Response 1: thank you very much for your ideas, thoughts, and comments. We found them very insightful and they brought us to a further in the discussion about using the Auto Encoder instead of GAN. Furthermore, your last point sounds to us as an ablation study of an OMS Monitor via a differently built OMS dataset. That is also a very interesting idea and can be explored in the next article.

Comment 2:

Formatting notes: Do not introduce abbreviations in the abstract. Do not use abbreviations in subtitles. Introduce abbreviations only once at the first occurrence.

Response 2: all comments were accepted and changed accordingly - thank you

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thanks for revising and responding to the comments. You have good work and potential, but I am not satisfied with the revision for two reasons.

1. The novelty remains unclear; it is simply the application of data! My suggestion is to name your model and focus on how it can solve the out-of-scope problem and differentiate itself using an overfitting concept. Then the focus will shift to model integration, deployment, and mitigation. Finally, include your study, which serves only to validate your model.

2. Again, you must understand how to conduct an academic study, beginning with background, literature review, methodology, discussion, and context. Everything seems jumbled and unstructured to me, and the writing terms and explanations should be in academic English!

Good luck with your revision.

Comments on the Quality of English Language

Thanks for revising and responding to the comments. You have good work and potential, but I am not satisfied with the revision for two reasons.

Good luck with your revision.

Author Response

Comment 1:

"The novelty remains unclear; it is simply the application of data! My suggestion is to name your model and focus on how it can solve the out-of-scope problem and differentiate itself using an overfitting concept. Then the focus will shift to model integration, deployment, and mitigation. Finally, include your study, which serves only to validate your model."

Response:

We have named our model TRYGEN (TRust Your GENerator) to clarify its identity and contributions in the article.

The manuscript has been refocused to highlight how TRYGEN addresses the out-of-scope problem. We have added discussions on model integration, deployment considerations, and mitigation strategies to emphasize practical applicability.

Comment 2:

"Again, you must understand how to conduct an academic study, beginning with background, literature review, methodology, discussion, and context. Everything seems jumbled and unstructured to me, and the writing terms and explanations should be in academic English!"

Response:

The manuscript's chapters have been reorganized and renamed to follow the standard academic structure. While we initially used nomenclature common in articles accepted at CVPR conferences, we recognize the importance of adhering to broader academic conventions for clarity. For your reference, we summarize the chapters' names and their content:

Introduction: Presents the problem, motivation, and contributions.
Related Work: Provides a comprehensive literature review.
Methodology: Details our approach with theoretical foundations.
Experiments: Describes the experimental setup and evaluation metrics.
Results and Discussion: Analyzes findings and their implications.
Conclusion: Summarizes our study and suggests future work.

We have revised the writing to adhere to academic English standards, ensuring clarity, precision, and a formal tone throughout the document. Technical terms are defined, and explanations are structured to enhance readability and comprehension. If there are specific areas where you feel improvements are needed, we would appreciate your guidance on those sections.

Thank you for your valuable feedback, which has significantly improved our manuscript. We believe the revisions address your concerns and enhance the overall quality of the paper.