Streamlining Haptic Design with Micro-Collision Haptic Map Generated by Stable Diffusion

Abstract

Rendering surface materials to provide realistic tactile sensations is a key focus in haptic interaction research. However, generating texture maps and designing corresponding haptic feedback often requires expert knowledge and significant effort. To simplify the workflow, we developed a micro-collision-based tactile texture dataset for several common materials and fine-tuned the VAE model of Stable Diffusion. Our approach allows designers to generate matching visual and haptic textures from natural language prompts and enables users to receive real-time, realistic haptic feedback when interacting with virtual surfaces. We evaluated our method through a haptic design task. Professional and non-haptic designers each created one haptic design using traditional tools and another using our approach. Participants then evaluated the four resulting designs. The results showed that our method produced haptic feedback comparable to that of professionals, though slightly lower in overall and consistency scores. Importantly, professional designers using our method required less time and fewer expert resources. Non-haptic designers also achieved better outcomes with our tool. Our generative method optimizes the haptic design workflow, lowering the expertise threshold and increasing efficiency. It has the potential to support broader adoption of haptic design in interactive media and enhance multisensory experiences.

1. Introduction

Haptic feedback is critical in a virtual environment(VE), enabling users to experience more realistic physical sensations [1]. Simulating real-world tactile sensations in virtual environments is a focus research objective, such as rendering haptic feedback for virtual surface textures [2]. The expansion of the haptic device market is largely fueled by advancements in the entertainment and the experience economy, as seen in the widespread use of compact mobile devices such as game controllers and VR headsets equipped with linear resonant actuators (LRAs) [3]. As demand for haptic experiences grows, many studies have explored advanced acquisition devices and haptic actuators [4]. While these technologies enhance user experiences, they also raise costs. Additionally, we observed that current haptic design workflows, such as Core Haptics of Apple (Apple Inc., Cupertino, CA, USA) [5] and RichTap Creator of AAC Technologies (AAC Technologies Holdings Inc., Shenzhen, China) [6], often require designers to rely on proprietary editors or have programming expertise. Although some of these tools support audio-based haptic feedback generation, creating haptic feedback that matches with visual texture information remains mainly a case-by-case process, resulting in significant workload.

With the widespread adoption of generative artificial intelligence technologies, knowledge can now be embedded into the design process through generative methods. As a result, leveraging generative techniques to support haptic design has become an inevitable trend and holds great potential for enhancing the quality of haptic experiences. Generative methods have been applied in haptic design to generate matching vibrotactile feedback from visual and auditory modalities [7,8,9,10,11]. To streamline the haptic design workflow, enabling high-fidelity haptic feedback on widely available devices, we introduce micro-collision haptic (MCHa) maps and use a fine-tuned variational autoencoder (VAE) decoder of Stable Diffusion. This method simultaneously generates visual textures and matching MCHa maps. Visual texture maps define surface contact characteristics via displacement and roughness information. User interactions with micro surface dimples can be modeled as a trajectory composed of sequential collisions. Haptic feedback for virtual texture enables users to sense material properties through micro texture collisions [12], with pronounced feedback over significant height variations. Thus, height and roughness data from 2D visual texture maps can be utilized to derive a corresponding 2D haptic map.

To further understand the impact of our generative tool on the haptic design workflow, we conducted a user study. We designed a haptic creation task in which designers were asked to create haptic feedback for four materials—gravel, metal, fabric, and wood—and assign them randomly to a 4 × 4 tactile square. Eight professional haptic designers, each with over 3 years of experience, first completed the task using a traditional workflow, and then repeated it using our generative haptic tool. In parallel, ten non-haptic interaction designers were recruited. After a brief training, they completed the same task using both the conventional workflow and our generative tool. After each task, the designers provided subjective evaluations of their experience. To ensure the practical applicability and relevance of our approach, all experiments were conducted on an iPhone 13 mini with LRA, representing one of the most widely accessible consumer-level platform types for tactile interaction.

To assess the output quality, 30 users were recruited to evaluate the haptic feedback from all four groups. The participants also performed a material identification task on the 4 × 4 tactile square to assess the recognizability of the designed textures. The NASA-TLX results revealed no significant differences in overall user preference, but the designers reported significantly lower workload and improved efficiency when using our generative tool. Subjective evaluations showed that our method significantly improved the overall ratings and consistency of the haptic designs, and many materials were perceived to be more realistic.

Overall, our work contributes to the field of haptic design in the following ways:

• Lowering the expertise threshold:
• Our generative workflow enables novice designers to create effective haptic feedback without extensive technical training.
• Reducing professional workload: It streamlines the design process, allowing expert designers to work more efficiently and focus on creative tasks rather than manual implementation.
• Promoting broader adoption: By simplifying the workflow, our method encourages the integration of haptic design in various media platforms, including gaming, streaming, and other interactive applications.
• Enhancing user experience: The tool supports the generation of realistic, material-consistent tactile feedback, contributing to more immersive multisensory experiences in virtual environments.

2. Related Works

2.1. Haptic Texture

In haptic research, texture is a key attribute, referring to the sensation experienced when touching the fine structural irregularities on a surface. Vision and touch are the primary modalities for texture perception, with haptic texture generally consisting of multiple dimensions, such as roughness–smoothness, warmth–coldness, hardness–softness, and stickiness–slipperiness [13]. Rendering texture through vibrotactile feedback has long been a central focus in haptic interaction research [14]. Vibrotactile feedback is one of the main methods of haptic feedback [15]. When using vibrotactile feedback to render texture, it effectively portrays the texture distribution and roughness [16]. Furthermore, vibrotactile feedback demonstrates good experience when rendering collisions [17]. The haptic perception of textures can be regarded as collisions with different shapes and distributions.

Texture perception requires users to engage through intermediaries or direct skin contact, exploring via motion. Strohmeier et al. [7] examined motion-coupled vibrotactile feedback during interaction and found that amplitude influences the perceived texture intensity, while force and timbre create different experiences, emphasizing the significance of motion in haptic texture perception. Heravi et al. [18] introduced a learned action-conditional model that uses the force and speed applied by the user as input to generate haptic texture signals. Meanwhile, due to the complexity and variety of natural textures and the uncertainty of user actions during interaction, some studies focus on improving the experience of freely exploring complex textures. Kuchenbecker et al. [19] proposed a method using linear predictive coding to render isotropic textures. Shin et al. [14] proposed a hybrid framework combining force and vibrational feedback, using linear predictive coding to render highly realistic uniform or non-uniform textures. Chan et al. [10] utilized height maps of virtual textures to extract height profiles from the movement trajectories on the texture, and then synthesized haptic signals based on these profiles to drive actuators. These studies are dedicated to allowing users to freely explore virtual materials and obtain as realistic vibrotactile feedback as possible.

While these studies have significantly improved user experience, they may rely on high-performance actuators, additional auxiliary devices, or more complex computational and development workflows. Currently, the most widely used haptic devices include game controllers, smartphones, and wearables. Haptic designers also seek to achieve haptic experience with simpler development, easier usability, and lower costs [20].

2.2. Generative Haptic Feedback

With the widespread use of generative artificial intelligence (GenAI), large diffusion models such as Stable Diffusion [21] enable users to generate high-quality, requirement-compliant images with simple natural language [22,23,24]. This inspires us to consider whether a similar approach could be applied to generate haptic texture in a simpler, more user-friendly, and cost-effective way. Virtual materials are rendered using texture mapping based on visual information to represent material properties such as geometry and roughness [25]. The main parameters of vibrotactile signals include mainly intensity and frequency. During surface interactions, appropriate wave peaks and frequencies can effectively render the micro-contact of the surface texture and the roughness of the material [26,27,28]. These share similar underlying parameters to those of haptic signals [11].

Generative visual–haptic techniques have made initial progress and are undergoing rapid advancement: Ujitoko et al. [8] were among the first to apply generative adversarial networks (GANs) for generating vibrotactile signals from pen movements on surfaces, achieving realistic haptic experiences. However, while users could input label information or texture images to generate vibration signals, the method lacked support for real-time vibrotactile feedback on virtual surfaces, constraining the exploration of complex, unevenly distributed textures. Cai et al. [29] employed conditional generative adversarial networks (cGANs) with a residue-fusion module to achieve vision-to-tactile transformation, markedly improving the visual resemblance between generated data and ground truth. In another study [30], they employed a transformer-based network framework to generate multimodal haptic signals. Acceleration signals were generated from the visual data of material surfaces, and a pen equipped with actuators was used to render the haptic feedback. Fang et al. [11] developed a conditional flow model operating within the latent feature space of a VAE to achieve bidirectional mapping between visual and haptic data. Leveraging the latent feature space helps to minimize the disparity between visual information and haptic signals, offering an elegant solution for generating high-resolution images.

At the same time, the discriminability of generated textures is crucial. In virtual environments, recognizing texture features through haptics is an essential part of the experience. A distinguishable texture not only allows users to freely explore the virtual surface—meaning the system can generate feedback in real time based on the user’s touch position—but also enables users to perceive the presence of textures and further recognize and identify them through continuous tactile feedback. Li et al. [31] proposed an image-based method to extract haptic texture information from static 2D images, enabling users to perceive realistic contours and textures through a force-feedback device. Rekik et al. [32] introduced a rendering technique based on finger position and velocity. By using a grid of taxels to encode texture, their method provides consistent haptic experiences on high-density textures and reduces user error rates. Shin and Choi [14] developed a hybrid framework for highly realistic modeling and rendering of both uniform and non-uniform textures. This technique enhances the perceptibility of fine-grained features in virtual textures, improving tactile discriminability. Normand et al. [33] used a wearable device with integrated vibration actuators to augment surface texture perception in augmented reality. They enhanced visual textures with data-driven haptic feedback and evaluated users’ recognition accuracy across nine representative visual–haptic texture pairs. Papadopoulos et al. [34] investigated the effects of various texture parameters on the perception of virtual textures in 3D haptic systems. Their findings show that, in force-feedback-based virtual environments, variations in texture shape improve user recognition. These studies focus on enhancing users’ experience when interacting with fine virtual textures, particularly in terms of texture discriminability. This not only improves user performance in virtual environments but also enhances realism, thereby enriching the overall experience.

These prior studies inspire us to combine generative techniques with discriminable haptic textures. With the widespread adoption of generative technologies across domains such as commerce, education, industry, healthcare, and entertainment, a large number of highly realistic and fine-grained visual textures are now being produced through generative means. Haptic design must generate corresponding tactile textures for these visual textures that are both realistic and capable of real-time interaction. Our goal is to support the creation of generative visual textures and their matching haptic counterparts in a more accessible way—under lower technical barriers and hardware requirements—and to enable their rendering on widely available LRA devices, such as smartphones and wearable devices.

3. Methods

3.1. Micro-Collision Interaction

Visual textures, which provide displacement maps with height information, can be directly mapped to vibrational signals. For instance, height values can be mapped to vibration intensity, making this method potentially effective when using specific vibration actuators [10]. However, height-mapping techniques are not entirely intuitive, as regions with greater texture height do not necessarily produce stronger vibrations. Instead, areas with significant height differences should elicit a more pronounced sense of collision.

The experience of haptic textures occurs during movements; it is necessary to synchronize vibrotactile feedback with movements [7]. When interacting with textures, finger movements across the surface induce micro-collisions between the skin and the local texture. These tactile sensations combine with visual observations to form a unified perception of texture [13,26,35]. Thus, when interacting with virtual textures, the contact between the user and the texture can be understood as a series of micro-collisions. The finger’s movement trajectory should correspond to the pixel sequence of the visual texture and the haptic feedback signal sequence, jointly enhancing the perception of virtual textures.

When users interact with uneven virtual textures, the visual texture reveals height variations, such as raised obstacles and fragmented rough surfaces. When users touch the edges of texture features, significant height changes result in stronger collisions, while smaller changes lead to gentler feedback due to the typically steep height gradients at texture edges (Figure 1). The height variations along the user’s movement path on the virtual texture influence the collision intensity, and the sequence of collision feedback during interactions with the virtual texture provides the haptic sensation of the virtual texture (Figure 1).

3.2. MCHa Map

To generate the MCHa map [36], all relevant parameters must be encoded into a single texture (Figure 2). We divide the MCHa map into two layers: an intensity layer and a frequency layer.

3.2.1. Intensity Layer

The intensity layer represents the vibration strength experienced when the visual texture is touched. We designed a converter to transform visual texture images into the MCHa intensity layer. To suppress image noise, Gaussian blur is applied as an initial step. This widely used smoothing technique employs a Gaussian function as a convolution kernel, averaging each pixel with its surroundings to achieve a smooth, refined image. $I(x,y)$: input image; $G(u,v)$: Gaussian kernel weights; and $\sigma$: standard deviation of the Gaussian function, controlling the blur strength.

$$I_{\mathrm{blurred}}(x,y)=\sum _{u=-k}^k\sum _{v=-k}^{k}G(u,v)·I(x+u,y+v)$$

$$G(u,v)={\displaystyle \frac{1}{2\pi {\sigma }^2}}exp\left(-{\displaystyle \frac{u^2+v^2}{2{\sigma }^2}}\right)$$

Then, during micro-collision interactions, noticeable height differences occur when the fingertip comes into contact with or leaves bumps. These transitions are expected to produce more pronounced vibrations, making it necessary to detect the edges of such bumps in the visual texture. To achieve this, we applied the Sobel algorithm for edge detection in images [37,38]. To extract edge details from the image, the Sobel algorithm [37,38] is used to compute the gradient by assessing pixel value variations in horizontal and vertical directions. Prominent edges in the visual texture are highlighted with color values mapped proportionally to their gradient significance. The horizontal gradient is

$$G_x=I\ast K_x$$

The vertical gradient is

$$G_y=I\ast K_y$$

The Sobel kernels are defined as

$$K_x=\left[\begin{array}{ccc}-1& 0& +1\\ -2& 0& +2\\ -1& 0& +1\end{array}\right],K_y=\left[\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ +1& +2& +1\end{array}\right]$$

Using the horizontal gradient $G_x$ and vertical gradient $G_y$, we calculate the gradient magnitude for each pixel, which represents the edge strength.

$$M(x,y)=\sqrt{G_x^2+G_y^2}$$

For the intensity layer, we mapped the grayscale values of the MCHa maps to the Intensity parameter in Apple’s Core Haptics API. In this case, the mapping is direct: higher grayscale values represent stronger vibration intensity. Let the grayscale intensity $G\in [0,255]$, and the corresponding Intensity parameter $I\in [0,1]$; we define a linear mapping as

$$I={\displaystyle \frac{G}{255}}$$

(1)

This ensures that a pixel with the maximum grayscale value ($G=255$) produces the highest haptic intensity ($I=1$), and a pixel with $G=0$ results in the minimum intensity.

In our test environment, the Intensity parameter corresponds to a physical vibration strength ranging approximately from 0 g to 1.3 g, where g denotes the gravitational acceleration. This linear relationship enables a consistent and scalable mapping from the generated MCHa maps to perceptually meaningful vibration feedback on typical LRA-equipped devices.

3.2.2. Frequency Layer

Previous studies have well established a correlation between perceived roughness and vibration frequency: lower-frequency vibrations are generally associated with rougher textures, while higher-frequency vibrations are perceived as smoother [39]. The roughness map provides an intuitive representation of the varying roughness across different regions of the visual texture [40,41]. We mapped the roughness map of the visual texture to vibration frequency, providing a frequency parameter to render the perceived roughness when the user interacts with that specific area. To avoid redundant work, the roughness maps for the visual textures were generated using open-source texture generation tools [42].

To map the grayscale values of the roughness maps to the Sharpness parameter defined in Apple’s Core Haptics API (ranging from 0 to 1), we established an inverse linear mapping function. In our roughness maps, higher grayscale values (closer to 255) indicate greater surface roughness. However, in the Core Haptics API, a lower Sharpness value corresponds to a rougher tactile sensation. Therefore, we applied an inverse mapping from the grayscale intensity $G\in [0,255]$ to the Sharpness parameter $S\in [0,1]$, defined as

$$S=1-{\displaystyle \frac{G}{255}}$$

(2)

This mapping ensures that a pixel with maximum roughness (i.e., $G=255$) corresponds to the lowest Sharpness value ($S=0$), producing a coarse haptic effect as intended. On our test device, which are equipped with a LRA, the Sharpness parameter approximately corresponds to a vibration frequency range of 80–260 Hz. A higher Sharpness value tends to produce higher-frequency, sharper vibrations, while lower values yield slower, duller tactile sensations. This frequency behavior aligns well with the perceptual representation of surface roughness in haptic rendering.

3.3. Variational Autoencoder Training

We conducted the training on a system equipped with an AMD 7900X CPU (Advanced Micro Devices, Santa Clara, CA, USA) and an NVIDIA RTX 4090D GPU (NVIDIA Corporation, Santa Clara, CA, USA). To prevent uncontrolled color variations from three-channel VAEs, we adopted a single-channel VAE, encoding vibration intensity as grayscale pixel values. We used the Adam optimizer (learning rate $1\times {10}^{-4}$), batch size 2, and a gradient accumulation step of 1, optimizing pixel-level reconstruction by minimizing loss between generated outputs and ground truth. A discriminator network was introduced, alternating updates between the generator and the discriminator. The discriminator used AdamW $1\times {10}^{-4}$ with a custom learning rate scheduler (500-step warm-up, 100,000 total training steps), with gradient clipping at 1.0 to prevent overfitting. We incorporated pre-trained perceptual loss [43], balancing pixel-level reconstruction and high-level feature learning through adaptive weighting of perceptual and adversarial losses.

3.4. Dataset

Our training is based on a custom dataset. Previous research has used generative adversarial networks (GANs) with public datasets like the Penn Haptic Texture Toolkit (HaTT) [16] and LMT-108-Surface Materials [44] for image generation training. While these datasets provide haptic textures based on high-precision acquisition devices, they do not enable a complete and smooth workflow from visual textures to vibration-driven outputs, which is essential for our objective. To achieve this, we constructed a dataset with 16,000 samples based on the MCHa map generation process. We first used Stable Diffusion to generate visual textures, covering four material types commonly found in games: gravel, metal, textile, and wood. These material categories feature irregular ridges, depressions, and varying roughness, which effectively reflect the tactile characteristics of different surfaces. Our dataset consists of a total of 16,000 pairs, with 4000 pairs for each material. Each pair is composed of a visual texture and its corresponding MCHa map, which was generated using the generator described in the section intensity layer above. The entire dataset preparation process strictly followed the fine-tuning tutorial for Stable Diffusion 3.5 Medium available on Hugging Face [45].

3.4.1. Post-Process

As the vibration actuator signals are directly output to deliver vibration feedback, it is crucial to account for human perceptual capabilities and implement suitable post-processing to optimize the user experience. Psychophysical studies indicate that the just noticeable difference (JND) for human perception of vibration feedback intensity is approximately 0.2 g, while the JND for frequency perception is around 40–60 Hz [46]. Accordingly, taking into account user experience and the strength and frequency constraints of the actuator, the intensity layer of MCHa is divided into six levels (0–1.2 g with 0.2 g increments), and the frequency layer is divided into five levels (80–240 Hz with 40 Hz increments).

3.4.2. Generated Result

We have enabled the generation of MCHa maps directly from natural language. The MCHa map is displayed as an image, intuitively matched with the corresponding albedo texture, and supports real-time, unrestricted virtual texture exploration by directly inputting vibration parameters into the vibration actuator in Unity.

To generate MCHa maps using Stable Diffusion, we provided natural language prompts that described the desired tactile characteristics of each material. For instance, to generate a wood texture, we used the following positive prompt:

“Old wooden surface close-up, top–down view with visible ridges, grooves, cracks, splits, and weathered texture. Rugged, tactile wood grain with natural damage and worn matte finish. Soft, diffused lighting showing texture depth without harsh shadows. Light to dark brown tones. Ultra-detailed, photorealistic, macro.”

To guide the model away from undesired characteristics, we also included a negative prompt:

“smooth surface, polished, glossy, plastic, uniform grain, perfect texture, no cracks, flat, blurry, clean edges.”

These prompts enabled the model to generate visual textures with detailed and realistic tactile features, which were then used as the basis for producing matched haptic feedback through the corresponding MCHa maps.

The following are our generated results (Figure 3). We evaluate the generation quality using perceptual similarity (LPIPS, Learned Perceptual Image Patch Similarity) and mean squared error (MSE). The MCHa map generated by the fine-tuned VAE shows no noticeable difference in overall features compared with the one in the dataset. At the detailed regions of the virtual texture, there may be differences in the vibration intensity of small textures, and some fine textures may feel differently rendered. However, given that the vibration intensity of these fine textures is relatively low, their effect on overall perception is negligible.

4. User Study: Streamlining Haptic Design with Generative MCHa Map

We evaluated how the generative MCHa map simplifies the haptic design workflow. Specifically, we assessed the quality of haptic designs produced by both professional and non-professional designers, with and without the use of our method. Finally, we conducted interviews with participating designers to gather qualitative feedback on their experiences with our tool.

4.1. Haptic Design Flow

Haptic content creation typically involves interdisciplinary collaboration among engineers, designers, and domain experts. The process generally begins with the definition of design goals based on specific application scenarios. First, the workflow includes identifying key interaction events, such as gameplay elements or multisensory video segments requiring haptic augmentation (Figure 4). Subsequently, designers develop sensory mapping models that associate these events with appropriate vibrotactile feedback—for instance, simulating impacts, explosions, or heartbeats. Finally, based on the specifications of the employed vibration actuator, we determined the design parameters and utilized corresponding development environments and software tools—such as RichTap Creator and Apple’s Core Haptics—to design vibration patterns. The patterns were then exported as corresponding .ahap (Apple Haptic and Audio Pattern) and .he (Haptic Effect) files. A prototype system was subsequently built to enable rapid iteration and testing. Engineers are responsible for characterizing actuator behavior—such as voltage-to-acceleration curves—to support parametric design. Designers, on the other hand, conduct contextual analyses and tune multisensory synchronization to deliver cohesive user experiences.

4.2. Experiment Design

To evaluate the effectiveness of our generative MCHa map tool in supporting haptic design tasks, we conducted a controlled experiment targeting the following questions:

• Can non-haptic designers, without a background in haptics, utilize our tool to create compelling haptic experiences effectively?
• Can professional haptic designers benefit from our tool in terms of quality and efficiency?

The experimental procedure is briefly summarized in Figure 5.

We invited 8 professional haptic designers, each with over 3 years of experience in haptic design. They were highly familiar with tools such as Apple Core Haptics, RichTap Creator, and the design of vibration feedback for gaming controllers like those from Xbox and Sony. Additionally, we recruited 10 designers without a haptic design background but with expertise in interaction design and user experience design.

The participants were assigned to the following experimental groups:

• GP: Professional haptic designers used traditional tools (RichTap Creator) to complete the design task.
• GPG: The same professionals repeated the task using our generative tool; they were allowed to tune the output with their specialty.
• GU: Non-haptic expert designers used traditional tools; they were taught to use the tools before the experiment.
• GUG: Non-haptic expert designers completed the task solely using our tool.

Since visual texture design is not the primary focus of this study, we used Stable Diffusion to generate consistent visual textures for all materials.

For professional haptic designers, we first asked them to complete the task using traditional tools. Specifically, they used RichTap Creator by AAC Technologies to design vibration-based haptic feedback corresponding to each texture, simulating the sensation of brushing across the surface (GP). Specifically, designers were first provided with a unified set of visual textures. Based on these textures, they sketched the expected finger-surface interaction profiles while sweeping from left to right across the texture, focusing on surface height variations and potential collision events. They then estimated a vibration intensity envelope according to the magnitude of undulations and the severity of perceived impacts. Next, they assessed the perceived surface smoothness and marked expected changes in roughness along the sweep trajectory, leading to a sketched frequency envelope representing perceived vibration frequency variations. Finally, these envelopes were implemented in RichTap Creator, where users generated corresponding vibration fragments by editing the intensity and frequency curves. The resulting vibration patterns were exported as .he files, containing the complete LRA drive signals. Then, we asked the same designers to use our proposed tool to complete the same task. They used Stable Diffusion in combination with a VAE model trained by us to directly generate haptic texture maps for the four materials. After generation, they fine-tuned the parameters for each texture’s haptic feedback using our methods (GPG).

For non-haptic designers, we first guided them through the traditional design process. Before starting the formal design task, we provided step-by-step instructions on how to use conventional methods, along with multiple practice sessions. This was intended to reduce unfamiliarity with the tool and mitigate any potential learning effects due to repeated design tasks. After the practice phase, they completed the haptic feedback design for the texture stimuli (GU). Afterwards, we instructed them to use our tool, which allowed them to directly generate the four material textures using Stable Diffusion and our trained VAE model. After the practice phase, they completed the haptic feedback design for the texture stimuli (GUG).

We did not impose a strict time limit for the design task. All participating designers completed the task within 40 min. After completing each round of the design task, the designers filled out a modified version of the NASA-TLX questionnaire to assess their perceived workload.

The comparison between GP and GPG illustrates how our method improves both the quality and efficiency of haptic design for professionals. The comparison between GU and GUG demonstrates how our tool simplifies the design process and enhances usability for non-experts. The comparison between GPG and GUG further reflects the extent to which our tool enhances design quality across different user groups.

4.3. Design Task

We designed a haptic design task in which the participants were asked to create haptic feedback for a demo. The terrain consisted of a $4\times 4$ square composed of sixteen $1:1$ textures (Figure 6). These textures were made up of four different texture types: gravel, metal, fabric, and wood. Each texture type shared a consistent visual texture. The pointer could freely move across the square and triggered vibration-based haptic feedback corresponding to the underlying texture.

This demo was used for user testing. In the testing phase, players were asked to explore the $4\times 4$ square, while their visual input was blocked. Their goal was to identify and reconstruct the material distribution of the terrain using only the haptic feedback (Figure 6). The design task was entirely text-guided and consisted of the following steps:

• Design one visual texture for each of the four types of textures: gravel, metal, fabric, and wood.
• Create matching haptic feedback for each of the visual textures.
• Randomly assign each material to four textures in the $4\times 4$ square. All tiles of the same material shared the same visual texture.

4.4. Apparatus

Our demo and all completed designs were presented on an iPhone 13 mini (Apple Inc., Cupertino, CA, USA). The iPhone 13 mini supports a maximum vibration intensity of 1.3 g and a vibration frequency range of 80–260 Hz. The demo was developed using Unity (Unity Technologies, San Francisco, CA, USA), version 2022.3.10f1c1.

4.5. User Study

We conducted a user study with 30 participants (aged 18–35, mean: 24.1) who reported no sensory impairments. Each participant experienced the haptic textures designed in conditions of GP, GPG, GU, and GUG in randomized order to counterbalance potential sequencing effects. This experiment was approved by the Ethics Review Committee of Shanghai Jiao Tong University (E2022741), and each participant was informed about the study and signed a consent form.

In each demo, the $4\times 4$ texture square was visually occluded. The participants were asked to explore the space using vibrotactile feedback and reconstruct the spatial texture layout. In GPG and GUG, the designers created entirely novel textures from scratch using our generative tool. Given that conventional vibration fragments cannot provide real-time feedback based on the user’s finger position, inconsistent sweeping speeds may lead to mismatched haptic sensations. To ensure fair comparison across trials, a visual speed indicator was presented during each touch of haptic texture, guiding participants to sweep across the texture at a consistent and appropriate speed (200 pixels per second). In addition, the participants were allowed to freely explore each tactile texture without time or interaction constraints.

Following the task, the participants completed a subjective questionnaire evaluating the following:

• Overall quality of haptic design (1 (very poor)–7 (very good) Likert scale).
• Perceived consistency across textures(1 (very poor)–7 (very good) Likert scale).
• Realism of individual textures (gravel, metal, fabric, wood (1 (very poor)–7 (very good) Likert scale)).

Additionally, the accuracy of reconstruction was recorded to examine the discriminability of the generated haptic textures.

4.6. Result

All statistical analyses were conducted using Python (v3.13) with the scipy and scikit-posthocs libraries. As the data did not meet normality assumptions (assessed visually and confirmed via Shapiro–Wilk tests), non-parametric methods were adopted. For NASA-TLX workload scores, we used Wilcoxon signed-rank tests for within-subject comparisons (GP vs. GPG; GU vs. GUG) and Mann–Whitney U tests for between-group comparisons (GPG vs. GUG), as these involved independent samples. For user ratings across four design conditions (GP, GPG, GU, and GUG) and identification accuracy, Friedman tests were employed to account for repeated measures, followed by Nemenyi post hoc tests to identify pairwise differences when significance was found.

4.6.1. Workload of Design Workflow

We conducted statistical analyses on six NASA-TLX subscales to evaluate perceived workload across three group comparisons: GP vs. GPG (professional designers using traditional vs. generative tools), GU vs. GUG (non-professional designers using traditional vs. generative tools), and GPG vs. GUG (professional vs. non-professional using generative tools) (Figure 7).

Results from Wilcoxon signed-rank tests revealed significant reductions in physical, temporal, performance, and effort workloads when professional designers used the generative tool (GP vs. GPG, all p < 0.05), with no significant differences found for mental and frustration. For non-professional designers, generative tools significantly reduced perceived workload across all subscales (GU vs. GUG, all p < 0.01). Furthermore, Mann–Whitney U tests comparing professional and non-professional users of generative tools showed that GPG consistently reported lower workload than GUG in all subscales except mental demand, with statistical significance across five subscales (all p < 0.05).

4.6.2. User Evaluation

To assess the quality of the design outputs, we conducted Friedman tests followed by Nemenyi post hoc comparisons focusing on two key comparisons: (1) professional haptic using traditional tools (GP) versus our methods (GPG), and (2) professional haptic (GPG) versus non-haptic designers (GUG) using our methods (Figure 8).

For Overall ratings, the Friedman test revealed a significant difference (${\chi }^2\left(2\right)=10.23$, $p=0.0060$). Post hoc analysis showed that designs created with the intelligent tool by professional designers (GPG) were rated significantly higher than those created with traditional tools (GP) ($p=0.0084$), while no significant difference was found between GPG and GUG ($p=0.4002$). For Consistency, a significant difference was observed (${\chi }^2\left(2\right)=40.63$, $p<0.0001$). GPG designs were rated significantly more consistent than GUG ($p<0.0001$), although no significant difference was found between GP and GPG ($p=0.1469$).

Regarding Gravel Realism, the Friedman test indicated a significant difference (${\chi }^2\left(2\right)=22.04$, $p<0.0001$). GPG designs achieved significantly higher realism ratings compared with GP ($p=0.0002$), whereas no significant difference was observed between GPG and GUG ($p=0.5561$). For Metal Realism, no significant difference was found among groups (${\chi }^2\left(2\right)=3.95$, $p=0.1385$). For Fabric Realism, a significant difference was detected (${\chi }^2\left(2\right)=8.25$, $p=0.0161$). GPG designs outperformed GP with a significant difference ($p=0.0222$), and there was no significant difference between GPG and GUG ($p=0.4375$). For Wood Realism, the Friedman test again showed significant differences (${\chi }^2\left(2\right)=9.88$, $p=0.0072$). GPG designs were significantly preferred to GP ($p=0.0184$), while no significant difference was found between GPG and GUG ($p=0.8303$).

Finally, for Discriminability (identification accuracy) (Figure 9), a highly significant difference was observed (${\chi }^2\left(2\right)=34.94$, $p<0.0001$). Designs produced by professional designers with the intelligent tool (GPG) were rated significantly higher than those using traditional methods (GP) ($p<0.0001$), and no significant difference was observed between GPG and GUG ($p=0.5561$).

5. Discussion

5.1. Generative MCHa Map

First, our MCHa map offers a structural advantage for real-time interaction. Although traditional methods can achieve good performance in Consistency and in the Realism of certain materials when the user’s brushing speed is controlled, due to carefully designed feedback that matches the expected collision patterns, our method shows greater advantages during free exploration. This may explain why our method received higher Overall ratings (Figure 8). Unlike traditional approaches that rely on pre-designed, unidirectional tactile sequences, our method supports interactive feedback during free-form exploration. It accommodates touches in any direction, providing a more realistic and immersive haptic experience.

In particular, our method demonstrates a significant advantage in the discriminability of fine textures. Such textures often require users to explore them repeatedly from multiple directions to be recognized, and our approach clearly outperforms traditional methods in this aspect (Figure 9). Compared with existing methods, our approach demonstrates superior performance in both mechanistic rationale and practical implementation: The work of Ujitoko et al. [8] also realized end-to-end haptic feedback, but their method cannot achieve free exploration of haptic textures like MCHa due to the limitations of the haptic data. The work of Chan et al. [10] is based on the micro-contact mechanics, where vibration feedback signals are mapped from the height map of virtual textures. Their method shows good discrimination for simple textures when using specific actuators. However, in free exploration and an absolute identification experiment, the recognition accuracy of users is lower than that of the MCHa map. Fang et al. [11] also used a VAE model to generate haptic data and proposed a visual–tactile cross-modal generative algorithm to enhance the consistency between visual and tactile modalities, but it cannot directly apply the generated output to the actuator input, as our method does. Compared with those prior works, the novelty of our work is in the micro-collision-based haptic feedback. We treat the interaction with virtual textures as a sequence of collision feedback on a virtual plane, simplifying the generation of tactile textures while retaining the texture’s tactile features. Additionally, by integrating Stable Diffusion, we enable direct generation of haptic textures from natural language prompts, which can directly drive vibration actuators, significantly simplifying the haptic interaction design process.

In summary, our method offers an elegant and efficient way for designers to generate both visual texture maps and matching MCHa maps directly from textual prompts. This not only helps professional haptic designers improve efficiency and reduce workload but also enables non-experts to quickly create usable haptic feedback.

5.2. Design with Generative Haptic Methods

Our method offers a convenient and efficient workflow for haptic design. For professional haptic designers, when using our method, they are relieved from spending extensive time and effort on aligning visual textures with haptic feedback. Our system directly transforms the visual texture’s surface features into corresponding haptic responses, eliminating the need for manual calibration and fine-tuning of vibration intensity envelopes typically required in traditional workflows (Figure 7). This enables professional haptic designers to focus their efforts on more critical aspects of the design process—such as expressive articulation and the creative nuances of haptic feedback—thereby making a more substantial impact on the overall user experience. In our experiments, professional designers using our method tended to invest more energy in refining the details of the haptic feedback. This post-generation fine-tuning contributed to significantly higher scores in both overall quality and consistency compared with designs from non-professional users (Figure 8). These findings suggest that our approach can further enhance the quality of expert-level haptic design.

Our method offers an intuitive and user-friendly tool for novice and non-haptic-specialist designers. For non-haptic designers, one of the main challenges lies in accurately describing the desired texture using precise language, even though they are typically experienced in working with generative AI tools. Unlike other visual or interaction design assets, texture generation demands careful consideration of camera angles, surface patterns, lighting effects, and fine-grained texture features. The unfamiliar technical vocabulary can increase the cognitive load when using generative haptic tools (Figure 7). Another challenge involves adjusting the generated outputs. Although our method provides highly usable haptic feedback directly, users without knowledge of haptic parameters or tactile dynamics often struggle to make efficient and meaningful adjustments based on their subjective impressions. They typically require multiple rounds of fine-tuning before finalizing a satisfactory design. Nevertheless, compared with using traditional tools, non-professional designers reported a significantly reduced workload (Figure 7). This indicates that tools based on generative AI offer a more user-friendly experience for novice designers. The reduced reliance on specialized knowledge and the simplified workflow allow them to complete design tasks with greater ease.

Our method significantly enhances the quality of haptic design, particularly in terms of perceived realism. In addition, designers without a professional background in haptics were also able to generate tactile feedback with a high degree of realism using our method. In the single-task design scenario of our experiment, the tactile feedback they produced for the four material types did not differ significantly in perceived realism compared with that of professional haptic designers (Figure 8). This suggests that our approach can substantially lower the barrier to entry for tactile design, improving overall quality. For basic and straightforward design tasks, non-haptic designers can achieve high-quality results with ease using our method.

5.3. The Streamlining of the Haptic Design Process

These results suggest that the generative tool effectively reduces perceived workload across most dimensions, particularly in terms of temporal demand, effort, and performance, for both professional and non-professional designers. The consistent improvement observed in the GPG vs. GP and GUG vs. GU comparisons indicates that the generative tool provides ergonomic and cognitive benefits regardless of design expertise. Interestingly, while mental demand did not show a significant difference for professionals, non-professional users experienced notable reductions. This may imply that the generative tool offers more cognitive scaffolding for less experienced users, potentially by automating or simplifying complex design processes. The frustration level only significantly decreased among non-professional users, suggesting that experienced designers may require more control or transparency to feel satisfied with AI-supported workflows. Overall, the results reinforce the value of intelligent tools in democratizing design by lowering cognitive barriers, especially for novices, while still delivering measurable efficiency gains for experts.

5.4. Limitations

However, the micro-collision-based approach still has room for improvement. Our method is particularly well suited for textures with pronounced surface contact features, such as the granular surface of gravel or the ridged structures typical of wood. In contrast, for relatively smooth textures like metal, which lack significant surface height variation, our method may struggle to convey distinct tactile cues. To address this limitation, we suggest augmenting the MCHa map with auditory feedback—such as metallic friction sounds—to provide a more realistic and immersive multisensory experience.

Although our method provides support for novice designers, professional haptic designers still need to invest considerable cognitive effort in formulating design strategies—for example, ensuring consistency in perceived intensity across interactions. To address this challenge, our ongoing work explores the integration of large language models (LLMs). We are currently constructing a structured knowledge base of haptic design principles and techniques. By leveraging retrieval-augmented generation or fine-tuning approaches, we aim to enable LLMs to offer expert-level guidance in haptic design workflows.

Moreover, responsiveness to the user’s movement speed across the surface is also crucial. Due to current device limitations—specifically, the relatively low touchscreen sampling rates (typically below 240 Hz) and the response latency of vibration actuators (ranging from 1 to 5 ms)—the system may fail to provide immediate and accurate feedback for each micro-collision during fast swiping. This limitation suggests the need to prioritize surface features by importance, allowing the system to respond to the most salient micro-collisions first. Such prioritization can help ensure recognition accuracy under real-time interaction constraints. We plan to address this issue in future work.

5.5. Implication for Design and Future Research

We plan to further develop the MCHa map tool by creating plugins for development platforms such as Unity, enabling fast and high-quality haptic texture generation. Building on our work, the tool can be integrated with haptic design SDKs such as Apple Core Haptics or RichTap Creator. Since different vibration actuators have varying amplitude and frequency characteristics, incorporating device-specific control parameters can help produce more perceptually consistent haptic feedback. Our method also has potential for further development in media applications such as games and videos. By combining content understanding, haptic knowledge systems, and our generative texture framework, the tool could deliver real-time, expressive, and realistic haptic feedback for dynamic scenes—such as surface contact and friction in racing games or cinematic sequences. For future research, one direction is to build a structured database of expert haptic design terminology. This would include strategies, guidelines, and methods extracted from the literature, along with psychophysical data on human tactile perception. A clean, accurate, and professionally curated database would greatly benefit generative haptic design tools. Since haptic design is often used in conjunction with visual and auditory modalities, future models will need strong cross-modal integration capabilities. Research into multimodal information processing and cross-modal generation involving haptics will be essential in advancing this field.

6. Conclusions

In this study, we introduce MCHa maps, a simple and user-friendly representation for vibrotactile textures. We also train a generator that enables developers and designers to efficiently produce visual textures along with precisely matched MCHa maps using latent diffusion models (LDMs), such as Stable Diffusion, driven by natural language prompts. Our fine-tuned VAE model exhibits high performance in texture generation, especially for stochastic and irregular patterns, such as gravel and wood surfaces. We further evaluated the proposed generative haptic design tool, and found that it significantly reduces designers’ workload compared with traditional tools, particularly in terms of physical effort. In addition, we assessed the quality of the generated haptic feedback. Our method delivers superior performance during free exploration tasks and more faithfully captures the tactile characteristics of surface interactions. The proposed method and tool substantially lower the workload and expertise barrier in haptic design, facilitating broader accessibility and contributing to the democratization of tactile content creation. Moreover, our approach improves the overall quality of haptic design, particularly by enabling the creation of realistic tactile feedback in applications such as gaming and immersive media.