A Lightweight Multi-View Learning Approach for Phishing Attack Detection Using Transformer with Mixture of Experts
Round 1
Reviewer 1 Report
The paper titled "A Lightweight Multi-view Learning Approach for Phishing Attack Detection using Transformer with Mixture of Experts" proposes a robust method for detecting phishing websites by leveraging multiple views of website information. The authors highlight the limitations of URL-based methods in detecting advanced phishing techniques and the need for additional features such as website content, visual appearance, and behavioral patterns. The proposed method combines URL analysis with attribute, content, and behavioral information to create a multi-view representation of websites. It utilizes a Transformer network with a mixture of experts mechanism to effectively learn and fuse information from different views.
The authors introduce the main contributions of their research, which include:
1. Employing adversarial post-training to transform a pre-trained language model into a URL feature extractor, allowing for semantic-aware URL embeddings.
2. Introducing three informative website features: attribute information, content information, and behavior information, along with the website's URL, to capture phishing attack cues from different perspectives.
3. Utilizing a Transformer network with a mixture of experts mechanism to learn the multi-view information of webpages, considering relationships between and within the constructed views.
4. Demonstrating superior performance in detecting phishing websites compared to state-of-the-art techniques, with higher accuracy and fewer labeled data. The proposed method also exhibits robustness and effectiveness in detecting phishing attacks over time.
The authors are advised to revise the manuscript by taking the below listed comments in consideration.
1. The paper fails to clearly state the research objective or problem statement that the proposed method aims to address. This makes it difficult for readers to understand the motivation behind the research and the significance of the proposed solution.
2. The quality of images must be improved before the resubmission.
3. The paper lacks a comprehensive review of the existing literature and related works in the field. It does not sufficiently discuss prior research on the topic, which is essential for positioning the proposed method within the existing knowledge and identifying the gaps it intends to fill.
4. Although the paper briefly mentions the proposed method, it lacks a thorough explanation of the underlying principles, algorithms, or techniques used. This makes it challenging for readers to grasp the novelty and technical contributions of the proposed method.
5. The paper lacks a detailed experimental evaluation of the proposed method. It does not provide sufficient information about the datasets used, the evaluation metrics employed, or the comparative analysis with existing methods. This hinders the assessment of the proposed method's effectiveness and its performance compared to other approaches.
6. The paper does not adequately discuss or interpret the experimental results obtained from applying the proposed method. It lacks a thorough analysis of the findings and fails to provide insights into the strengths, weaknesses, or limitations of the proposed method.
7. The paper compares the proposed method with baseline approaches, but it does not provide a detailed analysis of the strengths and weaknesses of each baseline method. This makes it difficult to assess the novelty and superiority of the proposed method.
8. Although some implementation details are mentioned, such as the model architecture and optimization parameters, the paper lacks a comprehensive description of the experimental setup and methodology. This makes it challenging for readers to replicate the experiments and evaluate the validity of the results.
9. The paper briefly mentions the PhishTank dataset and the collection of benign URLs, but it fails to provide critical details such as the characteristics of the dataset, the distribution of phishing and benign samples, and any preprocessing steps applied. These details are crucial for understanding the generalizability and reliability of the proposed method.
10. The paper introduces the challenge of imbalanced datasets but does not provide a thorough analysis of how the proposed method handles this issue. It lacks a discussion on the potential bias introduced by the class imbalance and the methods employed to mitigate its impact.
11. The paper primarily focuses on the performance evaluation of the proposed method on a specific dataset (PhishTank). There is a lack of discussion on the generalizability of the method to other datasets or real-world scenarios. It is important to assess the robustness and applicability of the proposed method in diverse settings.
The paper titled "A Lightweight Multi-view Learning Approach for Phishing Attack Detection using Transformer with Mixture of Experts" proposes a robust method for detecting phishing websites by leveraging multiple views of website information. The authors highlight the limitations of URL-based methods in detecting advanced phishing techniques and the need for additional features such as website content, visual appearance, and behavioral patterns. The proposed method combines URL analysis with attribute, content, and behavioral information to create a multi-view representation of websites. It utilizes a Transformer network with a mixture of experts mechanism to effectively learn and fuse information from different views.
The authors introduce the main contributions of their research, which include:
1. Employing adversarial post-training to transform a pre-trained language model into a URL feature extractor, allowing for semantic-aware URL embeddings.
2. Introducing three informative website features: attribute information, content information, and behavior information, along with the website's URL, to capture phishing attack cues from different perspectives.
3. Utilizing a Transformer network with a mixture of experts mechanism to learn the multi-view information of webpages, considering relationships between and within the constructed views.
4. Demonstrating superior performance in detecting phishing websites compared to state-of-the-art techniques, with higher accuracy and fewer labeled data. The proposed method also exhibits robustness and effectiveness in detecting phishing attacks over time.
The authors are advised to revise the manuscript by taking the below listed comments in consideration.
1. The paper fails to clearly state the research objective or problem statement that the proposed method aims to address. This makes it difficult for readers to understand the motivation behind the research and the significance of the proposed solution.
2. The quality of images must be improved before the resubmission.
3. The paper lacks a comprehensive review of the existing literature and related works in the field. It does not sufficiently discuss prior research on the topic, which is essential for positioning the proposed method within the existing knowledge and identifying the gaps it intends to fill.
4. Although the paper briefly mentions the proposed method, it lacks a thorough explanation of the underlying principles, algorithms, or techniques used. This makes it challenging for readers to grasp the novelty and technical contributions of the proposed method.
5. The paper lacks a detailed experimental evaluation of the proposed method. It does not provide sufficient information about the datasets used, the evaluation metrics employed, or the comparative analysis with existing methods. This hinders the assessment of the proposed method's effectiveness and its performance compared to other approaches.
6. The paper does not adequately discuss or interpret the experimental results obtained from applying the proposed method. It lacks a thorough analysis of the findings and fails to provide insights into the strengths, weaknesses, or limitations of the proposed method.
7. The paper compares the proposed method with baseline approaches, but it does not provide a detailed analysis of the strengths and weaknesses of each baseline method. This makes it difficult to assess the novelty and superiority of the proposed method.
8. Although some implementation details are mentioned, such as the model architecture and optimization parameters, the paper lacks a comprehensive description of the experimental setup and methodology. This makes it challenging for readers to replicate the experiments and evaluate the validity of the results.
9. The paper briefly mentions the PhishTank dataset and the collection of benign URLs, but it fails to provide critical details such as the characteristics of the dataset, the distribution of phishing and benign samples, and any preprocessing steps applied. These details are crucial for understanding the generalizability and reliability of the proposed method.
10. The paper introduces the challenge of imbalanced datasets but does not provide a thorough analysis of how the proposed method handles this issue. It lacks a discussion on the potential bias introduced by the class imbalance and the methods employed to mitigate its impact.
11. The paper primarily focuses on the performance evaluation of the proposed method on a specific dataset (PhishTank). There is a lack of discussion on the generalizability of the method to other datasets or real-world scenarios. It is important to assess the robustness and applicability of the proposed method in diverse settings.
Author Response
- Reviewer #1
We express our gratitude to the esteemed reviewer for providing constructive feedback and valuable sugges- tions regarding the positive evaluation of our work. Furthermore, his/her insightful recommendations have significantly contributed to improving the quality and rigor of our manuscript.
1.1. Comment 1
RC: The paper fails to clearly state the research objective or problem statement that the proposed method aims to address. This makes it difficult for readers to understand the motivation behind the research and the significance of the proposed solution.
AR: We appreciate the valuable feedback provided by the reviewer. We have thoroughly revised the manuscript to ensure a clear and concise presentation of the research objective and problem statement. We have provided a comprehensive overview of the specific problem we aim to address and the rationale behind our proposed method. Additionally, we have highlighted the significance of our solution in addressing the existing research gap and advancing the field.
1.2. Comment 2
RC: The quality of images must be improved before the resubmission.
AR: We improved image quality as suggested.
1.3. Comment 3
RC: The paper lacks a comprehensive review of the existing literature and related works in the field. It does not sufficiently discuss prior research on the topic, which is essential for positioning the proposed method within the existing knowledge and identifying the gaps it intends to fill.
AR: We appreciate the reviewer’s feedback. In response, we have thoroughly revised the paper to include a comprehensive review of the existing literature and related works in the field.
Aljofey, Ali, et al. "An effective detection approach for phishing websites using URL and HTML features." Scientific Reports 12.1 (2022): 8842.
Abdul Samad, Saleem Raja, et al. "Analysis of the Performance Impact of Fine-Tuned Machine Learning Model for Phishing URL Detection." Electronics 12.7 (2023): 1642.
Alshehri, Mohammed, et al. "Character-level word encoding deep learning model for combating cyber threats in phishing URL detection." Computers and Electrical Engineering 100 (2022): 107868.
Benavides-Astudillo, Eduardo, et al. "Comparative Study of Deep Learning Algorithms in the Detection of Phishing Attacks Based on HTML and Text Obtained from Web Pages." International Conference on Applied Technologies. Cham: Springer Nature Switzerland, 2022.
Paturi, Radhika, et al. "Detection of Phishing Attacks using Visual Similarity Model." 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC). IEEE, 2022.
Ariyadasa, Subhash, Shantha Fernando, and Subha Fernando. "Combining Long-Term Recurrent Convolu- tional and Graph Convolutional Networks to Detect Phishing Sites Using URL and HTML." IEEE Access 10 (2022): 82355-82375.
1.4. Comment 4
RC: Although the paper briefly mentions the proposed method, it lacks a thorough explanation of the underlying principles, algorithms, or techniques used. This makes it challenging for readers to grasp the novelty and technical contributions of the proposed method.
AR: We appreciate the reviewer’s feedback. In response, we have extensively revised the paper to provide a thorough explanation of the underlying principles, algorithms, and techniques used in the proposed method. We have ensured that readers can fully understand the novelty and technical contributions of our approach through detailed descriptions and clarifications.
1.5. Comment 5
RC: The paper lacks a detailed experimental evaluation of the proposed method. It does not provide sufficient information about the datasets used, the evaluation metrics employed, or the comparative analysis with existing methods. This hinders the assessment of the proposed method’s effectiveness and its performance compared to other approaches.
AR: We greatly appreciate the feedback from the reviewer. In Section 4.2, we have provided comprehensive information regarding the datasets utilized. Additionally, in Section 4.5, we have described the evaluation metrics employed for assessing our proposed method. Furthermore, in the experimental results section, we have conducted a rigorous comparative analysis with existing methods.
1.6. Comment 6
RC: The paper does not adequately discuss or interpret the experimental results obtained from applying the proposed method. It lacks a thorough analysis of the findings and fails to provide insights into the strengths, weaknesses, or limitations of the proposed method.
AR: We appreciate the reviewer’s feedback. We have extensively revised the paper to include a comprehensive discussion and interpretation of the experimental results obtained from applying the proposed method. We have conducted a thorough analysis of the findings, highlighting the strengths, weaknesses, and limitations of our approach.
1.7. Comment 7
RC: The paper compares the proposed method with baseline approaches, but it does not provide a detailed analysis of the strengths and weaknesses of each baseline method. This makes it difficult to assess the novelty and superiority of the proposed method.
AR: We appreciate the reviewer’s feedback. We have expanded the paper to include a detailed analysis of the strengths and weaknesses of each baseline method used in the comparison.
1.8. Comment 8
RC: Although some implementation details are mentioned, such as the model architecture and optimization parameters, the paper lacks a comprehensive description of the experimental setup and methodology. This makes it challenging for readers to replicate the experiments and evaluate the validity of the results.
AR: We greatly appreciate the feedback from the reviewer. In Section 4.4, we have provided a systematic description of the experimental setup, covering all the necessary details. Additionally, in Sections 4.6, 4.7, and 4.8, we have individually introduced the setups for the three specific experiments conducted. Furthermore, in Section 3, we have thoroughly explained the methodology, underlying principles, and workflow of our approach.
1.9. Comment 9
RC: The paper briefly mentions the PhishTank dataset and the collection of benign URLs, but it fails to provide critical details such as the characteristics of the dataset, the distribution of phishing and benign samples, and any preprocessing steps applied. These details are crucial for understanding the generalizability and reliability of the proposed method.
AR: We appreciate the reviewer’s feedback. We have provided additional descriptions of the datasets in Section 4.2, including the selection and construction methods employed. In Section 3.2, we have elaborated on the procedures for extracting features from web pages. Furthermore, in regards to the URLs of the web pages, we have adopted a pure natural language processing paradigm, treating the URLs as textual strings and utilizing the tokenization capabilities of the BERT tokenizer.
1.10. Comment 10
RC: The paper introduces the challenge of imbalanced datasets but does not provide a thorough analysis of how the proposed method handles this issue. It lacks a discussion on the potential bias introduced by the class imbalance and the methods employed to mitigate its impact.
AR: We appreciate the reviewer’s feedback. In the newly added discussion section, we have provided a compre- hensive analysis of how the proposed method effectively addresses the challenge of imbalanced datasets.
1.11. Comment 11
RC: The paper primarily focuses on the performance evaluation of the proposed method on a specific dataset (PhishTank). There is a lack of discussion on the generalizability of the method to other datasets or real-world scenarios. It is important to assess the robustness and applicability of the proposed method in diverse settings.
AR: We utilized the PhishTank dataset to evaluate the proposed method due to its crowd-sourced nature, wide
distribution, and realistic representation. We collected a diverse set of phishing samples over an extended period, ensuring the diversity of benign samples as well. This realistic dataset allows us to approximate real-world scenarios and assess the model’s performance in practical online practices. Additionally, we conducted evaluations of the trained model’s performance after a three-month period to explore its robustness and generalization capabilities, validating its long-term effectiveness. Therefore, while the PhishTank dataset is the primary focus of our study, we have taken into consideration the evaluation of the proposed method’s generalizability and applicability.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Consider providing more detailed information about the dataset used in the evaluation. Including details such as the size, diversity, and source of the dataset will enhance the credibility of the experimental results and allow for better reproducibility.
Elaborate on the specific tactics used by highly concealed phishing websites to masquerade URL addresses. Providing examples or case studies can help readers understand the challenges and complexities involved in detecting such deceptive practices.
Discuss the potential limitations or vulnerabilities of the proposed multi-view Transformer model. Identify and address any potential biases, overfitting concerns, or trade-offs between accuracy and computational complexity that may arise when utilizing multiple views of website information.
Conduct a comparative analysis with other state-of-the-art methods in addition to discussing their outperformance. By including a comparison with existing approaches, you can provide a more comprehensive evaluation of the proposed method's effectiveness and highlight its unique advantages or limitations in relation to other techniques.
Discuss the potential real-world implications and applications of the proposed method beyond the experimental settings. Consider exploring how the developed model could be integrated into existing cybersecurity systems or utilized by security professionals to enhance phishing detection and prevention efforts. Additionally, discuss the scalability and deployment considerations when applying the proposed method to large-scale and real-time scenarios.
Kindly refrain from using sources that were released before 2019. Cite recent studies that are highly relevant to your subject. The paper also doesn't have enough citations. Another key stage is to compare the topic of the article to other recent publications or works that are comparable to broaden the research's ramifications beyond the subject. Authors may cite and rely on these important works while discussing the subject of their article and the problems of the present.
A. Heidari, M. A. J. Jamali, N. J. Navimipour and S. Akbarpour, "A QoS-Aware Technique for Computation Offloading in IoT-Edge Platforms Using a Convolutional Neural Network and Markov Decision Process," in IT Professional, vol. 25, no. 1, pp. 24-39, Jan.-Feb. 2023, doi: 10.1109/MITP.2022.3217886.
A. Heidari, N. Jafari Navimipour and M. Unal, "A Secure Intrusion Detection Platform Using Blockchain and Radial Basis Function Neural Networks for Internet of Drones," in IEEE Internet of Things Journal, vol. 10, no. 10, pp. 8445-8454, 15 May15, 2023, doi: 10.1109/JIOT.2023.3237661.
Catillo, M.; Pecchia, A.; Villano, U. A Deep Learning Method for Lightweight and Cross-Device IoT Botnet Detection. Appl. Sci. 2023, 13, 837. https://doi.org/10.3390/app13020837
Nwakanma, C.I.; Ahakonye, L.A.C.; Njoku, J.N.; Odirichukwu, J.C.; Okolie, S.A.; Uzondu, C.; Ndubuisi Nweke, C.C.; Kim, D.-S. Explainable Artificial Intelligence (XAI) for Intrusion Detection and Mitigation in Intelligent Connected Vehicles: A Review. Appl. Sci. 2023, 13, 1252. https://doi.org/10.3390/app13031252
Author Response
- Reviewer #2
We express our gratitude to the esteemed reviewer for providing constructive feedback and valuable sugges- tions regarding the positive evaluation of our work.
1.1. Comment 1
RC: Consider providing more detailed information about the dataset used in the evaluation. Including details such as the size, diversity, and source of the dataset will enhance the credibility of the experimental results and allow for better reproducibility.
AR: We greatly appreciate the feedback from the reviewer. In the revised manuscript, we have provided a more detailed description of the dataset in Section 4.2, including the data collection strategy, data scale, and data applications. In the experimental section, we have independently described the data usage settings for each experiment.
1.2. Comment 2
RC: Elaborate on the specific tactics used by highly concealed phishing websites to masquerade URL addresses. Providing examples or case studies can help readers understand the challenges and complexities involved in detecting such deceptive practices.
AR: The case study of highly disguised phishing websites is of great value. However, due to the difficulty in obtaining real examples of highly disguised phishing websites online, we propose an alternative approach to demonstrate the performance of our method in this challenging scenario. Firstly, we conduct an extensive search on the internet to identify previously well-known cases of highly disguised phishing websites and record their phishing URLs (as shown in Table 1). Secondly, we utilize a variant of our method, specifically a URL-based model, to identify these cases.
The experimental results are as follows: Our variant model, which solely relies on the URL perspective, accu- rately predicted all phishing samples listed in Table 1. Furthermore, the model achieved high prediction confi- dences of 90% or above for nearly all samples. The only exception was the example "https://www.bocws.tk," which was predicted as a phishing website with a confidence level of 61%.
We further investigated the example "https://www.bocws.tk" by analyzing its embedded features. Our analysis revealed that the embeddings of the tokens "https://", "www", "boc", and "ws" exhibited low responses, while the token ".tk" showed a significantly higher response. We conducted further investigation on the domain ".tk" and discovered that it is frequently abused by malicious users for hosting malicious websites
Table 1: URLs of Highly Concealed Phishing Websites
Benign Camouflaged
https://www.facebook.com https://www.faceb00k.com
https://www.paypal.com/signin https://www.paypal–accounts.com/signin
https://www.amazon.com https://www.amazonn.com/ap/signin
https://fackebook.com https://fackelook.cixx6.com
https://www.kucoin.com/ hppts://mts.kucoinregister.com https://login.blockchain.com/#/login?product=wallet hppts://login-blokchain.com.openspotlight.org https://www.kucoin.com/ https://www.kuktoin.com/
https://www.bittrex.com https://www.bittrex.itd
https://www.boc.cn https://www.bocws.tk
https://login.taobao.com/ https://login.towbao-com.ck/member/login.jhtml.aspx
and phishing activities. Our research indicates that the proposed method effectively captures crucial phishing features and balances the importance of features after considering the contextual relationships between tokens. Additionally, since all these phishing webpage examples are offline, we were unable to obtain information from other perspectives such as the number of redirections and webpage loading time, which are important supplementary factors for identifying phishing webpages. Highly concealed phishing webpages typically involve multiple redirections and slow loading times. In other words, theoretically, we believe that our model would achieve a higher level of detection for online phishing instances.
1.3. Comment 3
RC: Discuss the potential limitations or vulnerabilities of the proposed multi-view Transformer model. Identify and address any potential biases, overfitting concerns, or trade-offs between accuracy and computational complexity that may arise when utilizing multiple views of website information.
AR: We thank the reviewer for raising important points regarding the potential limitations and vulnerabilities of our proposed multi-view Transformer model. We have thoroughly discuss these aspects in the revised manuscript.
1.4. Comment 4
RC: Conduct a comparative analysis with other state-of-the-art methods in addition to discussing their outper- formance. By including a comparison with existing approaches, you can provide a more comprehensive evaluation of the proposed method’s effectiveness and highlight its unique advantages or limitations in relation to other techniques.
AR: We appreciate the reviewer’s suggestion. In the revised manuscript, in the newly added discussion section, we have conducted a comparative analysis with other state-of-the-art methods, highlighting the advantages of the proposed method by analyzing their strengths and limitations.
1.5. Comment 5
RC: Discuss the potential real-world implications and applications of the proposed method beyond the experi- mental settings. Consider exploring how the developed model could be integrated into existing cybersecurity systems or utilized by security professionals to enhance phishing detection and prevention efforts. Ad- ditionally, discuss the scalability and deployment considerations when applying the proposed method to large-scale and real-time scenarios.
AR: We appreciate the reviewer’s suggestion. In the revised paper, we have included a discussion on the potential real-world implications and applications of our proposed method. We explore how the developed model can be integrated into existing cybersecurity systems and utilized by security professionals to enhance phishing detection and prevention efforts. Furthermore, we discuss scalability and deployment considerations when applying the method to large-scale and real-time scenarios.
1.6. Comment 6
RC: Kindly refrain from using sources that were released before 2019. Cite recent studies that are highly relevant to your subject. The paper also doesn’t have enough citations. Another key stage is to compare the topic of the article to other recent publications or works that are comparable to broaden the research’s ramifications beyond the subject. Authors may cite and rely on these important works while discussing the subject of their article and the problems of the present.
AR: In the revised manuscript, we have followed the reviewer’s suggestion and included additional citations to relevant literature. We have analyzed the strengths of the approaches reported in these cited papers and discussed their differences compared to our proposed method. The referenced papers include, but are not limited to:
- Heidari, M. A. J. Jamali, N. J. Navimipour, and S. Akbarpour, "A QoS-Aware Technique for Computation Offloading in IoT-Edge Platforms Using a Convolutional Neural Network and Markov Decision Process," IT Professional, vol. 25, no. 1, pp. 24-39, Jan.-Feb. 2023, doi: 10.1109/MITP.2022.3217886.
- Heidari, N. Jafari Navimipour, and M. Unal, "A Secure Intrusion Detection Platform Using Blockchain and Radial Basis Function Neural Networks for Internet of Drones," IEEE Internet of Things Journal, vol. 10, no. 10, pp. 8445-8454, May 15, 2023, doi: 10.1109/JIOT.2023.3237661.
Catillo, M.; Pecchia, A.; Villano, U. "A Deep Learning Method for Lightweight and Cross-Device IoT Botnet Detection." Appl. Sci. 2023, 13, 837. https://doi.org/10.3390/app13020837
Nwakanma, C.I.; Ahakonye, L.A.C.; Njoku, J.N.; Odirichukwu, J.C.; Okolie, S.A.; Uzondu, C.; Ndubuisi Nweke, C.C.; Kim, D.-S. "Explainable Artificial Intelligence (XAI) for Intrusion Detection and Mitigation in Intelligent Connected Vehicles: A Review." Appl. Sci. 2023, 13
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
It can be accepted now.
