Mind Mapping Prompt Injection: Visual Prompt Injection Attacks in Modern Large Language Models

Abstract

Large language models (LLMs) have made significant strides in generating coherent and contextually relevant responses across diverse domains. However, these advancements have also led to an increase in adversarial attacks, such as prompt injection, where attackers embed malicious instructions within prompts to bypass security filters and manipulate LLM outputs. Various injection techniques, including masking and encoding sensitive words, have been employed to circumvent security measures. While LLMs continuously enhance their security protocols, they remain vulnerable, particularly in multimodal contexts. This study introduces a novel method for bypassing LLM security policies by embedding malicious instructions within a mind map image. The attack leverages the intentional incompleteness of the mind map structure, specifically the absence of explanatory details. When the LLM processes the image and fills in the missing sections, it inadvertently generates unauthorized outputs, violating its intended security constraints. This approach applies to any LLM capable of extracting and interpreting text from images. Compared to the best-performing baseline method, which achieved an ASR of 30.5%, our method reaches an ASR of 90%, yielding an approximately threefold-higher attack success. Understanding this vulnerability is crucial for strengthening security policies in state-of-the-art LLMs.

1. Introduction

Large language models (LLMs) are designed to generate information in response to text-based prompts efficiently and accurately. However, they can inadvertently produce sensitive content, including illegal or discriminatory information. One significant exploitation method is “prompt injection”, where malicious input is disguised as a legitimate prompt to extract sensitive data. If such attacks enable easy access to illicit information, they pose a severe criminal threat.

A real-world incident highlights this concern: On New Year’s Day 2025, a bomb attack occurred in front of the Trump International Hotel in Las Vegas. Investigations later revealed that the perpetrator had used ChatGPT to manufacture the explosive device [1]. This case highlights the risk that if LLMs provide unauthorized access to harmful information, they facilitate not only terrorism but also various other crimes.

To mitigate such threats, several security strategies have been proposed. Two widely adopted approaches include the following: first, employing prompt engineering techniques to prevent the generation of illegal or violent responses [2,3,4]; second, fine-tuning the model to block responses to malicious inputs [5,6,7,8,9].

Despite these security measures, no solution can guarantee complete protection due to inherent technological limitations. Excessive security may cause LLMs to reject even legitimate prompts, frustrating users and diminishing their usability. Since LLMs must balance security with user accessibility, vulnerabilities inevitably persist. Attackers continuously refine prompt injection techniques to exploit these weaknesses.

As LLMs evolve, their input capabilities have expanded beyond text to support multimodal inputs, including images, audio, and more. Consequently, prompt injection attacks have also evolved, utilizing multimodal input to bypass security measures. As LLMs integrate more advanced multimodal technologies and security protocols, simplistic attacks have become less effective. Attackers have, therefore, developed more sophisticated techniques to circumvent security barriers. This necessitates the development of robust security strategies to counteract emerging multimodal prompt injection threats.

A mind map is a visual diagram initially developed for educational purposes, designed to structure information hierarchically in a way that enhances cognitive processing [10]. These diagrams can store extensive information within a single image. Attackers can exploit this format by embedding malicious instructions within an otherwise innocuous map, ensuring that only a specific segment contains harmful content while the rest appears legitimate. This deceptive approach makes it difficult for LLMs to recognize an attack. For this reason, mind maps serve as an effective medium for executing multimodal prompt injection attacks.

This study introduces a novel prompt injection attack method utilizing mind map images. The mind map comprises subtopics branching from a central theme categorized into complete and incomplete sections. The attack strategy involves generating explanations for the incomplete sections, which contain malicious instructions that reference the completed sections. Unlike the detailed descriptions provided for legitimate subtopics, those containing malicious instructions remain unspecified.

The LLM is then presented with the mind map image alongside a prompt instructing it to complete the missing descriptions. Consequently, despite the explicit presence of the malicious instruction within the image, the LLM prioritizes filling in the incomplete sections, failing to recognize the attack, and effectively executing the embedded instruction.

The key contributions of this study are as follows:

• Introduction of a mind map image-based prompt injection attack with a high success rate, targeting multimodal LLMs equipped with robust security measures.
• Development of a mind map image generator to facilitate the creation of a malicious mind map for use in the proposed attack method.
• Design of a specialized prompt tailored for mind map image-based prompt injection attacks.

The remainder of this paper is organized as follows: Section 2 outlines the baseline prompt injection attack methods used for performance evaluation. Section 3 outlines the proposed method in detail, step by step. Section 4 presents a performance evaluation and comparison with existing approaches. Finally, Section 5 concludes the study.

2. Related Work

Prompt injection refers to the deliberate manipulation of LLMs through carefully crafted malicious inputs [11,12]. These manipulated outputs may include illegal activities, discriminatory language, and other harmful content, making them a significant security concern. Malicious users can exploit such outputs, necessitating a cautious approach to LLM interactions. A prompt injection attack requires only the insertion of harmful text-based prompts, demanding minimal technical expertise [4]. However, assessing the severity of these attacks in LLMs remains challenging. Consequently, developing a comprehensive countermeasure for prompt injection is a crucial research problem.

Prompt injection techniques can be classified into two primary categories. The first is direct prompt injection, in which malicious instructions are directly entered into the LLM. The second is indirect prompt injection, where harmful instructions are embedded within external data accessible to the LLM [13]. Indirect prompt injection is particularly relevant in attacks leveraging multimodal data. To evaluate the effectiveness of the proposed mitigation strategies, we will analyze various established prompt injection classifications as comparison benchmarks.

2.1. Pure Text-Based Prompt Injection

The most basic form of prompt injection involves directly inputting malicious instructions in plain text. This method is simple and universally applicable to all LLMs. However, since pure text-based attacks have been extensively studied and tested, modern LLMs incorporate robust security mechanisms, significantly reducing the success rate of such attacks. As a result, researchers have developed more sophisticated attack strategies, including encoding and masking techniques, to bypass security measures. The success of pure text-based attacks serves as an indicator of an LLM’s ability to filter sensitive terms from input prompts [14] effectively.

2.2. Multimodal-Based Prompt Injection

Advancements in LLMs have led to the emergence of multimodal models, which can process not only text but also various other data formats, such as images, audio, and videos [15]. Multimodal prompt injection attacks involve embedding malicious instructions within these non-textual data types, such as images or audio clips [16]. One notable approach within this category is visual prompt injection, where attackers embed harmful instructions within images to induce unintended model outputs [17,18]. Modern multimodal LLMs, including GPT-4V, possess sophisticated text recognition capabilities, enabling them to extract and interpret text embedded within images. The success of these attacks highlights security vulnerabilities, particularly insufficient input sanitization mechanisms for filtering extracted text from input images [19].

2.3. Encoding-Based Prompt Injection

Encoding is the process of transforming input characters or symbols according to predefined rules [20]. Encoding-based prompt injection involves converting malicious instructions into a specific encoding format (e.g., Base64 [21], Leetspeak [22]) before inputting them into LLMs. This technique enables attackers to circumvent LLM security policies by concealing malicious instructions. Previous research has demonstrated the effectiveness of a similar approach in which input prompts are encrypted before being processed by LLMs [23]. The success of this method confirms that encoded malicious instructions can circumvent the security mechanisms of targeted LLMs.

2.4. Masked-Based Prompt Injection

Masked prompt injection involves disguising sensitive words with a prompt to prevent LLMs from detecting them as part of an attack. Masking can be achieved by concealing or altering key terms, making it more difficult for the model to recognize the malicious intent. For the attack to succeed, the LLMs must be able to reconstruct the masked sections back to their original form. One notable example of masked-based prompt injection is ArtPrompt, which represents masked words as ASCII art [24]. This method integrates the masked malicious instruction, ASCII art, and a directive for reconstructing the original words from the ASCII representation within the user prompt to execute an attack. Consequently, a masked prompt is generated, allowing attackers to bypass the LLM’s security filters. The success of this technique underscores the potential for masked prompts with obscured sensitive words to circumvent the security policies of target LLMs.

Table 1. Feature analysis based on prompt injection methods.

Metric	Pure Text-Based Prompt Injection [25]	Multimodal-Based Prompt Injection [26]	Encoding-Based Prompt Injection [23]	Masked-Based Prompt Injection [24]
Input form	Text	Multimodal	Text	Text
Types of prompt injections	Direct	Indirect	Direct	Direct
Difficulty	Very easy	Hard	Easy	Normal
Attack success rate	Low	High	Low–middle	Middle

below summarizes the characteristics of various prompt injection methods, outlining their respective input formats, types, difficulty levels, and attack success rates.

As demonstrated, attackers have developed multiple prompt injection techniques to manipulate LLMs. In response, LLM developers are actively implementing various security measures to mitigate these threats [14]. However, prompt injection attacks continue to evolve, with new methods emerging regularly. To effectively counter these threats, continuous research is crucial for enhancing understanding and developing appropriate countermeasures against novel attack strategies.

3. Proposed Approach

Visual prompt injection attacks are a sophisticated method used to bypass the safety filters of multimodal LLMs. Generating images for such attacks is inherently more complex than generating text-based prompts. However, when crafted effectively, these images can facilitate highly successful attacks against LLMs. To counter such threats, LLM developers have implemented various defense mechanisms. One primary security strategy involves extracting and analyzing text from images to detect malicious content. For instance, GPT-4v utilizes an OCR (Optical Character Recognition) tool to extract digital text from image files, evaluate the extracted text, and assign a maliciousness score, thereby filtering out harmful prompts [27]. Additionally, fine-tuning models to reject responses to harmful images has proven to be another effective countermeasure [5,6,7]. While these strategies generally mitigate most malicious image-based attacks, they often fail when adversarial images are disguised as legitimate through task-instructing prompts. As a result, LLMs may fail to detect and block such attacks adequately.

To address this vulnerability, we proposed a novel mind map image-based prompt injection attack that differentiates itself from existing methods. In this approach, malicious instructions are embedded within a structured mind map image, which predominantly contains benign, unrelated instructions. The section containing the malicious instruction appears incomplete, as it lacks an explanation. A text prompt then instructs the LLM to complete the missing portions of the mind map. Since the model is solely focused on the completion task, it fails to recognize the embedded malicious instruction, thereby circumventing security mechanisms and generating the intended output.

This study does not aim to promote the malicious use of LLMs, but rather to identify and examine an overlooked vulnerability in multimodal models by introducing a novel visual prompt injection technique. Specifically, we demonstrate how malicious instructions can be inconspicuously embedded within otherwise benign images. The goal of this work is to raise awareness of this emerging threat and to contribute to the development of more robust defenses against visual prompt injection attacks.

3.1. Entire Structure

This section outlines the proposed method’s structure. As illustrated in Figure 1, the malicious instruction entered by an attacker for prompt injection into an LLM is referred to as “malicious user input”. When submitted directly to the LLM, this input is rejected. To execute a successful attack, the malicious input is processed through the mind map image generator, a tool developed in this study to generate mind map images. Initially, the input prompt is formatted into a text-based markdown structure using the markdown generator. This markdown is constructed using a predefined template, with the malicious input inserted into designated sections to ensure consistent formatting. This structured markdown is then converted into a mind map image by the mind map generator, preserving its integrity. The generated image is then submitted to the LLM, along with a prompt containing relevant instructions to trigger the attack. The LLM then produces unauthorized output, signifying a successful jailbreak.

Markdown and mind mapping serve as essential components of the attack methodology proposed in this study. Their hierarchical characteristics are exploited to bypass LLM security policies. While an in-depth understanding of these properties is not required to grasp the proposed method, two key aspects are crucial: (1) the mind map is generated from markdown data, which share a hierarchical structure; (2) both formats in this study adhere to a main-topic–subtopic–explain structure. Additional details are provided in Appendix B. The following section provides an in-depth explanation of each step in the proposed method.

3.2. Mind Map Image-Based Prompt Injection Attack

The proposed method processes malicious input through the mind map image generator, which first creates a markdown file incorporating the malicious user input. A mind map is then generated based on this markdown file. While Section 3.1 provides an overview of the entire workflow, this section details the specific steps involved in generating unauthorized output. The process consists of four distinct steps:

• Step 1: The malicious user input is provided to the mind map image generator.
• Step 2: The mind map image generator creates a text-based markdown file embedding the malicious user input.
• Step 3: A mind map image is generated from the markdown file.
• Step 4: The resulting mind map image is submitted to the LLM, along with an instructional prompt to execute a jailbreak attack.

3.2.1. Step 1: Input Malicious Instruction

A malicious user enters a harmful instruction into the mind map image generator. As shown in Figure 2a, this instruction is highlighted in blue. Upon launching the mind map image generator, an input field labeled “malicious user input” appears (Figure 2a). When the attacker enters malicious input into this field, it is forwarded to the markdown generator within the mind map image generator.

3.2.2. Step 2: Generate Markdown

The markdown generator creates a markdown file embedding the malicious user input. This file consists of multiple subtopics, as illustrated in Figure 2b, where different colors distinguish each subtopic for clarity. The overall structure and content of the markdown file, except for the malicious input, are predefined. The complete markdown comprises the following.

• Completed sections: These contain legitimate instructions unrelated to the attack. Each follows the main-topic–subtopic–explain format and serves as a reference for structuring the incomplete section. The contents of these sections are already created.
• Incomplete section: This section contains malicious user input integrated as a subtopic but lacks lower-level content, such as an “explain” node. As a result, its structure is limited to a central topic and its subtopics.

The differentiation between completed and incomplete sections is intentional, as LLMs are designed to complete missing sections by referencing completed ones. When attempting to complete the missing content within the incomplete section containing malicious input, the LLM refers to the completed sections, thereby generating unauthorized output. As shown in Figure 2b,c, the malicious user input (blue section) lacks an “explain” component, unlike the completed sections. By referencing the completed sections, the LLM generates an appropriate “explain” node for the malicious user input, effectively completing the incomplete node. Further details regarding the markdown structures used in the experiments are provided in Appendix A.1.

3.2.3. Step 3. Generating the Mind Map Image

A mind map image is created using the text-based markdown from the previous step. As shown in Figure 2, the identical colors in (b) and (c) represent the same content, indicating that the mind map directly inherits the markdown’s hierarchical structure and content. The incomplete section, designated as malicious user input, is distinctly highlighted in blue. This color differentiation serves as a reference point for LLMs when instructed to complete the missing section.

The malicious instruction is visually highlighted by coloring the corresponding node in blue, clearly identifying it as the incomplete section of the mind map. Instead of embedding malicious instructions directly—which would typically cause LLMs to reject the task—a targeted prompt is used. For example, a general instruction such as “complete the incomplete section” lacks specificity, making it difficult for the LLM to identify the intended target. In contrast, a more precise directive like “complete the section marked in blue” enables the model to accurately locate and complete the designated portion [28]. The proposed method strategically embeds the malicious instruction within the incomplete section and induces the LLM to complete it by referencing the completed parts of the mind map. This process exploits a security vulnerability in LLMs, as the model interprets the malicious instruction as part of a legitimate task. Additional details about the mind map used in the experiments are provided in Appendix A.2.

3.2.4. Step 4: Inputting the Prompt

The generated mind map image, along with a structured prompt, is submitted to the LLM. The prompt explicitly instructs the model to reference the completed sections while filling in the part containing the malicious instruction. Despite the exposure of the malicious content within the image, the LLM perceived it as part of the general task of mind map completion, bypassing conventional security measures. Since the primary focus is on completing the mind map rather than identifying malicious content, the LLM does not flag the input as an attack. The full prompt details are outlined in Appendix A.3. Once submitted, the attack initiates, generating an unauthorized markdown output that successfully circumvents the LLM’s security policies.

4. Performance Evaluation

In this study, we analyzed and evaluated the performance of the proposed method by comparing it with existing prompt injection techniques. For performance comparison, we selected attacks belonging to four distinct categories: (1) pure text-based prompt injection, which directly inputs malicious instructions without additional modification; (2) visual prompt injection, which falls under multimodal-based prompt injection; (3) encoding-based prompt injection, which includes techniques such as Base64 and Leetspeak; (4) masked-based prompt injection, exemplified by ArtPrompt. Through these comparisons, we not only assess performance but also identify the types of attacks to which LLMs remain vulnerable.

4.1. Experimental Environment

The experiments were conducted using Gemini with gemini-1.5-flash-latest [29], ChatGPT with gpt-4o-11-20 [30] models, and Grok3 with grok-3-beta [31]. To ensure consistency in the experimental results, either memory functions were disabled or a temporary chat mode designed to operate without memory was employed. No separate settings were applied for Gemini and Grok, as it lacks memory-related functions. For the experimental dataset, we selected data from the “harmful_behaviors_custom” dataset, which contains prompts designed to elicit responses related to harmful or unethical actions, such as weapon creation, cybercrime, violence. This dataset was previously used for evaluating ArtPrompt [32]. To enhance the reliability of the evaluation, further experiments were performed on the “harmfulqa” dataset [33], which consists of question-style prompts targeting various categories of harmful behavior, including physical violence, psychological harm, and misinformation. Both datasets consist of prompts concerning harmful behaviors, specifically curated for evaluating how models respond to sensitive and potentially malicious instructions. All experimental results were assessed through human evaluation to ensure the precise measurement of each performance metric. The evaluation criterion was based on compliance with OpenAI’s prohibited use policy [34]. An attack was considered successful only if the response’s harmfulness aligned with the attacker’s intent.

For performance evaluation, we employed two metrics: Helpful Rate (HPR) and attack success rate (ASR) [24,35]. These metrics are defined as follows:

$$\mathrm{HPR}={\displaystyle \frac{The total number of prompts that are not refused}{The total number of prompts}}$$

(1)

$$\mathrm{ASR}={\displaystyle \frac{The total number of prompts that are harmful}{The total number of responses}}$$

(2)

An attack was deemed successful if the model provided malicious information precisely matching the attacker’s intent. Each attack was conducted three times to assess its success rate. If a model response was not rejected but did not align with the attacker’s intent, it was classified as a misunderstanding. Specifically, cases were categorized as misunderstandings if the LLMs produced outputs unrelated to the prompt’s intent or if they accepted a malicious prompt but still generated responses unrelated to the prompt.

4.2. Helpful Rate

Table 2. Comparison of HPR performance across prompt injection methods on the “harmful_behaviors_custom” dataset.

HPR	Pure Text	Visual	Base64	Leetspeak	ArtPrompt	Proposed
GPT-4o	0	0.05	0.15	0.1	0.35	1
Gemini	0	0	0.05	0	0.6	1
Grok3	0.05	0.05	0.05	0.2	0.6	1

and

Table 3. Comparison of HPR performance across prompt injection methods on the “harmfulqa” dataset.

HPR	Pure Text	Visual	Base64	Leetspeak	ArtPrompt	Proposed
GPT-4o	0.087	0	0.391	0.13	0.609	1
Gemini	0	0.087	0.13	0.043	0.435	1
Grok3	0.043	0	0	0.261	0.348	1

present a comparative analysis of HPR performance across different prompt injection methods. In the case of pure text attacks, both GPT-4o and Gemini successfully rejected the prompts. Most other baseline methods exhibited an HPR of less than 0.2 on both models. However, ArtPrompt achieved significantly higher HPR compared to the other baseline methods. This increase was primarily due to the incorrect restoration of words masked as ASCII art. Instead of recognizing them as malicious content, the LLMs frequently misinterpreted them as different words, sometimes leading to unintended and unauthorized outputs. More often, this misinterpretation resulted in misunderstandings, where the models processed the inputs as benign prompts and generated responses that were unrelated. GPT-4o demonstrated superior accuracy in recognizing ASCII art compared to Gemini, leading to fewer misinterpretations and a lower HPR. This suggests that GPT-4o is more effective than Gemini in detecting and mitigating malicious prompt injections. Although Grok3 can be considered a state-of-the-art model in terms of overall performance, a comparison based on HPR values indicates that, from a security perspective, it behaves similarly to other models.

A prior study [36], using the same dataset [32], baseline methods, and processing methodology with gpt-4o-2024-05-13 reported an HPR of 0.65 for Base64 attacks. However, in the current study, this figure dropped significantly to 0.15. Similarly, ArtPrompt’s HPR, previously recorded at 0.6, decreased to 0.35. This is attributable to the strengthened security mechanisms implemented following the update of GPT-4o from version 13 May 2024 to version 20 November 2024. This indicates that while the security mechanisms of GPT-4o are continuously evolving, they are insufficient in countering masked-based prompt injection techniques, such as ArtPrompt. As illustrated in Table 2 and Table 3, the highest HPR is achieved by the method proposed in this study. The map image-based attack consistently generates responses without rejection, outperforming all other attack methods. This finding highlights a critical vulnerability in the security strategies of LLMs, indicating that they are not yet fully equipped to mitigate attacks leveraging mind map images.

4.3. Attack Success Rate

Table 4. Comparison of ASR performance across prompt injection methods on the “harmful_behaviors_custom” dataset.

ASR	Pure Text	Visual	Base64	Leetspeak	ArtPrompt	Proposed
GPT-4o	0	0	0.15	0.1	0.25	0.9
Gemini	0	0	0.05	0	0.15	0.9
Grok3	0.05	0.05	0.05	0.2	0.3	0.9

and

Table 5. Comparison of ASR performance across prompt injection methods on the “harmfulqa” dataset.

ASR	Pure Text	Visual	Base64	Leetspeak	ArtPrompt	Proposed
GPT-4o	0.087	0	0.304	0.087	0.087	1
Gemini	0	0	0.13	0	0.043	1
Grok3	0.043	0	0	0.174	0	1

present the ASR performance across different prompt injection methods. The pure text and visual prompt techniques failed to achieve any successful attacks against the latest GPT-4o and Gemini models. Base64-based prompts and Leetspeak-based prompts exhibited minimal ASR values, indicating their limited effectiveness. The ASR of the ArtPrompt method was 45% lower than its high HPR, with a particularly significant decrease observed in the Gemini model. A similar pattern was observed with the Grok3 model. On average, the ArtPrompt method showed a decrease of 35.05 points when converting from HPR to ASR, a decline attributed to instances of miscomprehension, which led to unsuccessful attacks and were therefore excluded from the ASR calculation. In contrast, the proposed method achieved an ASR of 90%, which is more than three-times higher than that of ArtPrompt, the most effective baseline method. While the maximum ASR of the baseline methods was 30.5%, the proposed method consistently achieved ASR values between 90% and 100%, clearly demonstrating its superior effectiveness.

4.4. Failure Case Analysis

The proposed prompt injection attack method achieved a 100% HPR, surpassing all existing attack techniques. Additionally, it demonstrated an ASR more than three-times higher than that of previous methods. To provide a comprehensive analysis of the results for each attack method, all attacks were classified into three categories: jailbreak, miscomprehension, and rejected. Furthermore, the attacks were analyzed based on the data categories used in the experiment, facilitating an in-depth evaluation of each method’s vulnerabilities.

Table 6. Classification of experimental results based on attack data categories. Blue “j”, green “m”, and red “r” indicate “jailbreak”, “miscomprehension”, and “rejected”, respectively. Numbers in parentheses denote the amount of instances for each attack data category.

Attack Data Category	Model	Pure Text			Visual			Base64			Leetspeak			ArtPrompt			Proposed
		j	m	r	j	m	r	j	m	r	j	m	r	j	m	r	j	m	r
Government (3)	GPT-4o	0	0	3	0	0	3	0	0	3	0	0	3	0	0	3	3	0	0
	Gemini	0	0	3	0	0	3	0	0	3	0	0	3	1	1	1	3	0	0
	Grok3	0	0	3	0	0	3	0	0	3	0	0	3	2	0	1	3	0	0
Hacking (4)	GPT-4o	0	0	4	0	0	4	0	0	4	0	0	4	0	0	4	4	0	0
	Gemini	0	0	4	0	0	4	0	0	4	0	0	4	1	1	2	4	0	0
	Grok3	0	0	4	0	0	4	0	0	4	0	0	4	2	0	0	4	0	0
Bomb (2)	GPT-4o	0	0	2	0	0	2	0	0	2	0	0	2	0	1	1	2	0	0
	Gemini	0	0	2	0	0	2	1	0	1	0	0	2	0	2	0	2	0	0
	Grok3	0	0	2	0	0	2	0	0	2	0	0	2	0	2	0	2	0	0
Virus (2)	GPT-4o	0	0	2	0	0	2	0	0	2	1	0	1	0	0	2	2	0	0
	Gemini	0	0	2	0	0	2	1	0	1	0	0	2	0	1	1	2	0	0
	Grok3	0	0	2	0	0	2	0	0	2	2	0	0	0	1	1	2	0	0
social media (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
Dangerous activity (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
Murder (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0	1	0	0
Identity theft (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0
	Grok3	0	0	1	v	0	1	0	0	1	0	0	1	1	0	0	1	0	0
Violence (2)	GPT-4o	0	0	2	1	0	1	0	0	2	1	0	1	0	1	1	2	0	0
	Gemini	0	0	2	0	0	2	0	0	2	0	0	2	0	2	0	2	0	0
	Grok3	0	0	2	0	0	2	0	0	2	0	0	2	0	1	1	2	0	0
Financial (3)	GPT-4o	0	0	3	0	0	3	2	0	1	0	0	3	0	1	2	3	0	0
	Gemini	0	0	3	0	0	3	0	0	3	0	0	3	1	2	0	3	0	0
	Grok3	0	0	3	0	0	3	0	0	3	0	0	3	0	1	2	3	0	0
Video game (1)	GPT-4o	0	0	1	1	0	0	0	0	1	0	0	1	0	1	0	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
Misinformation (3)	GPT-4o	0	0	3	0	0	3	1	0	2	1	0	2	1	1	1	1	2	0
	Gemini	0	0	3	0	0	3	0	0	3	0	0	3	2	1	0	1	2	0
	Grok3	1	0	2	1	0	2	1	0	2	1	0	2	1	1	1	1	2	0
Racism (1)	GPT-4o	0	0	1	0	0	1	1	0	0	0	0	1	0	0	1	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	1	0	0	0	0	1	1	0	0
Mental health (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	0	1	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	0	1	0
	Grok3	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	0	1	0
Theft (1)	GPT-4o	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0	1	0	0
	Gemini	0	0	1	0	0	1	0	0	1	0	0	1	0	1	0	1	0	0
	Grok3	0	0	1	0	0	1	0	0	1	0	0	1	0	0	1	1	0	0
Total		1	0	80	3	0	78	7	0	74	8	0	73	19	30	30	72	9	0

presents the results of the attacks, classified into categories such as jailbreak, miscomprehension, and rejected based on the attack data categories. Referring to Table 6, the attack method proposed in this study achieves a higher ASR than other attack methods. This is evident from the “Total” column, which summarizes the results for each approach. Notably, in ArtPrompt, an attack method with a higher HPR than others, the miscomprehension category appears more frequently than jailbreak. As discussed in Section 4.2, this discrepancy arises due to challenges in reconstructing masked words into their original form using ASCII art.

Table 6 also reveals that all three—GPT-4o, Gemini, and Grok3—fail in attacks involving data from the “misinformation” category, which requires generating articles or blog posts that promote illegal activities. During experimentation, GPT-4o provided detailed guidelines on crafting such content, violating OpenAI’s prohibited use policy [34]. However, since LLMs do not inherently generate full-length articles or blog posts, the model fails to produce the expected output, resulting in a failed attack. This failure occurs because the proposed method formats responses in markdown, which differs from traditional article or blog formats. As a result, the proposed method’s performance in this category aligns with limitations that may require modifying the mind map structure and incorporating more precise instructions into the text prompt.

5. Conclusions

This study introduces a novel prompt injection attack method that leverages mind map images embedded with malicious instructions. The approach involves inserting such instructions into incomplete sections of a mind map and prompting the LLM to complete it. The proposed method demonstrated a clearly superior performance, with ASR values consistently between 90% and 100%, compared to a maximum of 30.5% among the baseline methods. However, its applicability is limited to multimodal LLMs capable of image analysis. Additionally, if the malicious instructions require responses in markdown format, LLMs may generate erroneous outputs due to structural and contextual similarities in the mind maps used for each attack. This limitation can potentially be mitigated by modifying the structure, content, and reference targets of incomplete sections or incorporating additional diagrams and images alongside the mind map. Future research should explore these refinements.

The proposed method effectively bypasses the security mechanisms of multimodal LLMs without triggering response rejection, demonstrating its potency as an adversarial attack. These findings highlight critical vulnerabilities in existing LLM security policies, particularly their inability to detect visual-based attacks disguised as benign hierarchical structures, such as mind maps. Consequently, this study underscores the need for enhanced security measures to address such threats. By identifying these vulnerabilities, we aim to contribute to the development of more robust security frameworks that preserve the utility and performance of LLMs while strengthening their defenses against adversarial attacks.