Controllable Generation of Yangtze River Aquatic Organism-Themed Virtual Humans Based on Semantic Mapping of Human Attribute Features

Abstract

Existing virtual human generation methods often inadequately preserve individual attributes, lack coherent visual-semantic mapping, and insufficiently integrate regional cultural features. To address these limitations, this study proposes a controllable generation method combining human attribute recognition, visual-semantic mapping, and conditional image generation, using visual features of Yangtze River aquatic organisms as the application context. The system recognizes age group, gender, clothing category, and clothing colour from an input human image and encodes these attributes into a structured representation. A visual-semantic mapping mechanism then links human attributes with fish-specific colour, texture, and morphological features, converting them into structured generation descriptions. These descriptions guide a generative model to produce mermaid-like virtual characters that retain key human attributes while incorporating distinctive fish-inspired visual elements. Experimental results indicate stable performance in attribute consistency, structural integrity, visual-style controllability, and applicability to cultural tourism scenarios. The proposed method establishes an interpretable and extensible pipeline from attribute recognition to cross-domain semantic translation and conditional generation, offering a practical approach to personalized virtual character creation and a methodological reference for the digital presentation of regional ecological and cultural resources.

1. Introduction

Virtual human systems have been increasingly applied in cultural tourism, digital exhibitions, interactive communication, and science education, and have gradually become an important medium connecting cultural content, spatial experience, and public engagement. With the rapid development of artificial intelligence-generated content (AIGC), the construction of virtual humans has shifted from conventional manual design and predefined representation toward generative approaches characterized by data-driven modelling, conditional control, and scenario adaptation. This technological evolution has not only enhanced the flexibility of virtual human content generation, but has also created new possibilities for personalized representation and the integration of specific cultural content. Against this background, the development of virtual human representations that simultaneously incorporate personalized characteristics, cultural attributes, and communication value through artificial intelligence has become an important topic in current research and applications.

The advancement of generative image models has provided a technical foundation for the automated generation of virtual human representations. By introducing different types of conditional constraints, the appearance and visual characteristics of the generated content can be effectively controlled. In this process, human attribute recognition can extract key information from input images, including age, gender, clothing colour, and clothing category, and provide essential cues for subsequent personalized visual modelling and generation. This enables new forms of personalized virtual human applications in regional cultural communication and interactive experience scenarios. Nevertheless, several limitations remain in existing research. First, most studies focus on individual technical components, such as improving recognition accuracy or optimizing generation quality, whereas relatively limited attention has been paid to systematically transforming front-end recognition results into generation conditions and establishing an integrated recognition–translation–generation pipeline. Second, current virtual human design generally emphasizes visual styling and entertainment-oriented presentation, while the relationship between regional species-specific visual characteristics and their potential value for knowledge communication has not been sufficiently explored. Therefore, effectively bridging human attribute recognition results with virtual human generation conditions, and further applying the generated outputs to scenarios with regional cultural relevance and science communication value, remain important research and practical challenges.

Representative fish species associated with the Yangtze River Basin provide suitable visual references for virtual human generation because of their distinctive morphological characteristics and relevance to regional cultural and ecological communication. On the one hand, Yangtze River fish species exhibit substantial differences in body contours, body colouration, fin structures, texture patterns, and local morphological features, resulting in strong visual distinguishability. These characteristics not only facilitate the construction of a stable visual prototype library, but also make it possible to establish correspondences between fish-specific visual features and human appearance attributes, thereby supporting interpretable cross-domain visual translation. On the other hand, Yangtze River fish species embody distinctive regional cultural characteristics and ecological conservation significance. Transforming these visual features into personalized mermaid-like characters is intended to support engaging artistic interactions and provide a potential medium for communicating knowledge about Yangtze River fish species in museum and cultural tourism settings.

To address these issues, this study establishes a four-stage technical pipeline comprising human attribute recognition, structured attribute encoding, visual-semantic mapping, and conditional image generation. First, human attribute recognition is used to extract key information from human images, including age group, gender, clothing colour, and clothing category. The recognition results are then standardized through structured attribute encoding, enabling data exchange across different modules using predefined fields and a consistent data order. Subsequently, visual-semantic mapping is employed to establish interpretable correspondences between human appearance attributes and the visual characteristics of Yangtze River fish species in terms of colour range, texture, contour, and tail structure. Finally, the mapping results are organized into structured generation descriptions and fed into a locally deployed diffusion-model workflow to achieve the conditional generation of mermaid-like virtual characters. Through this pipeline, front-end recognition results are no longer limited to perceptual labels, but are further transformed into controllable conditions for downstream generation, thereby providing the overall virtual human generation process with a clearer logical structure and a higher degree of controllability.

The methodological contribution of this study lies in a task-oriented visual-semantic mapping mechanism that connects recognized human attributes with fish-inspired virtual character generation. Rather than directly combining age group, gender, clothing colour, and clothing category as undifferentiated textual prompts, the proposed framework assigns distinct functional roles to these attributes. Age group and gender are retained as character-level appearance constraints, whereas clothing colour and clothing category are translated into fish-related chromatic, textural, and morphological descriptors through an explicitly defined visual prototype library and mapping rules. The resulting semantic fields are subsequently organized into structured generation conditions, enabling the relationships between the original human attributes, intermediate visual descriptors, and final generation inputs to be traced throughout the pipeline.

The principal contribution is therefore the interpretable construction and organization of cross-domain generation conditions, rather than the development of a new diffusion backbone. By integrating structured attribute encoding, rule-based visual-semantic translation, and conditional image generation, the framework provides a task-specific approach to transforming human appearance information into controllable virtual character descriptions. Its implementation using fish species occurring in the Yangtze River Basin demonstrates the feasibility of connecting personalized human attributes with regional biological visual references in a unified generation workflow. The transferability of the mapping strategy beyond the selected visual prototypes remains an area for further investigation. Furthermore, the proposed mapping framework can be adapted to other visual themes by updating the visual prototype library and the corresponding mapping data while retaining its core attribute-processing and semantic translation logic. This provides a reusable methodological reference for cross-domain virtual character generation in different application contexts.

The remainder of this paper is organized as follows. Section 2 reviews related research on human attribute recognition, virtual human generation, and cross-domain visual-semantic mapping. Section 3 introduces the overall methodology and system framework. Section 4 describes the design and implementation of the human attribute recognition modules. Section 5 presents the visual-semantic mapping mechanism and conditional virtual human generation process. Section 6 reports the experimental results, including attribute recognition, semantic mapping, conditional generation, and comparative evaluations. Section 7 discusses the findings, summarizes the main conclusions, and outlines directions for future research.

2. Related Work

2.1. Research on Human Attribute Recognition

Human attribute recognition aims to transform appearance information in human images into computable and retrievable semantic labels, such as age, gender, clothing colour, and clothing category. This task extends beyond conventional image classification because different attributes depend on different regions of the human body, while recognition performance can also be affected by occlusion, pose variation, background interference, and fluctuations in image resolution [1]. Existing studies generally agree that a key challenge in human attribute recognition is to identify effective regions associated with specific attributes from complex visual information while maintaining the stability and interpretability of attribute representations [2]. Accordingly, research in this field has gradually shifted from early coarse-grained recognition based primarily on global appearance features toward joint modelling of local discriminative regions, inter-attribute correlations, and robustness under complex visual conditions [3].

Age and gender recognition are commonly regarded as soft biometric recognition tasks, although they differ considerably in terms of recognition difficulty and modelling strategies. Gender recognition is generally formulated as a relatively stable binary classification task with comparatively clear class boundaries. Age recognition, by contrast, is more susceptible to variations in facial texture, the continuous nature of ageing, and ambiguous boundaries between adjacent age groups, which can result in less stable predictions. Existing studies have mainly addressed these challenges from three perspectives. First, attention mechanisms have been introduced to enhance facial regions that are particularly relevant to age and gender, thereby improving discriminative capability in complex environments [4]. Second, improved convolutional architectures, ranking-consistent regression, and deformable convolutions have been employed to better capture subtle age-related variations [5]. Third, manifold learning and feature-reduction techniques have been incorporated into high-dimensional deep representations to reduce interference from redundant features during age estimation [6]. Recent studies have also explored vision–language-based approaches to pedestrian age recognition. Ozaki et al. [7] introduced a contrastive learning framework using automatically generated age descriptions to enhance the discriminability of age-related features in full-body images. These studies indicate that method selection for specific application scenarios should consider not only the shared visual basis of age and gender recognition, but also the output requirements and system stability demanded by downstream tasks.

Clothing attribute recognition represents another important research direction within human attribute recognition. Research in this area has evolved from large-scale dataset construction and fine-grained attribute annotation toward detection frameworks and deployment-oriented optimization for real-world scenarios. The introduction of DeepFashion provided a representative benchmark for clothing recognition research. Its rich annotations, including clothing categories, attributes, and cross-domain image pairs, have facilitated the development of clothing classification, attribute prediction, and image retrieval methods [8]. Previous studies have also shown that single-stage detection frameworks incorporating attention enhancement and multi-scale feature extraction can effectively improve recognition performance under common challenges such as scale variation, occlusion, and background interference [9]. In practical applications, clothing colour and clothing category can generally be recognized more reliably than finer-grained attributes such as style, material, or decorative details. Consequently, recent research has increasingly emphasized detection efficiency, robustness in complex environments, and deployment adaptability. Ultralytics introduced YOLO11 in 2024 as an object detection model designed to balance accuracy, inference speed, and computational efficiency [10]. Its efficient detection capabilities provide a practical technical basis for adaptation to clothing category recognition in interactive applications.

Existing studies have demonstrated that age, gender, and clothing attributes can be recognized with relatively stable performance using visual models. However, most previous work has primarily focused on improving attribute-level recognition accuracy, robustness, or deployment efficiency. In the context of this study, human attribute recognition is not regarded as the final objective, but rather as the input stage for subsequent structured encoding, visual-semantic mapping, and conditional image generation. Based on this task-oriented perspective, the present study focuses on four categories of information: age group, gender, clothing colour, and clothing category. These attributes are directly related to character configuration, colour organization, and structural translation in subsequent mermaid-like virtual human generation. Moreover, because these attributes are supported by relatively mature recognition techniques, they are more suitable for constructing a clear, stable, and verifiable front-end input layer.

2.2. Research on Virtual Human Generation and Cross-Domain Visual-Semantic Mapping

In recent years, research on virtual humans has gradually shifted from an early emphasis on appearance modelling and motion realism toward the integrated organization of character generation, interactive feedback, and scenario adaptation [11]. Existing reviews have shown that the perceived realism of virtual humans is determined not only by their visual appearance, but also by multiple factors such as motion characteristics, behavioural continuity, evaluation methods, and application contexts [12]. Accordingly, recent studies have paid increasing attention to how virtual humans are used in specific scenarios, including cultural exhibitions, educational communication, and situated interaction [13]. Previous studies have also indicated that many interactive virtual human systems rely primarily on static face recognition or a single type of visual input, resulting in limited capability to perceive complex nonverbal information and, consequently, reduced responsiveness and contextual adaptability [14].

In cultural communication scenarios, virtual humans have become an important medium for museums and digital cultural exhibitions. However, existing studies have mainly focused on technical functionality and users’ willingness to adopt such systems, while relatively limited attention has been paid to how the visual identity of a virtual character can embody cultural content or establish stronger associations with user input [15]. These studies suggest that the focus of virtual human research is gradually shifting from whether a digital character can be generated to how such a character can convey clearer scenario-specific meaning, support coherent interaction logic, and function as an effective carrier of cultural content.

From the perspective of system integration, recent research has also highlighted the value of modular architectures for combining heterogeneous artificial intelligence components. Forootani et al. developed Bio-Eng-LLM AI Assist, a modular platform integrating large language models, retrieval-augmented generation, image understanding, and diffusion-based image generation for interdisciplinary research and education [16]. Although its application objectives differ from those of virtual character generation, the study demonstrates a software integration approach in which multiple AI capabilities are coordinated within a unified system. In the present study, modular integration provides the implementation architecture, while the specific research focus lies in the structured semantic interface that connects human attribute recognition with cross-domain visual translation and conditional character generation.

Meanwhile, the continuous development of image generation technologies has provided a solid technical foundation for the construction of digital virtual humans. Existing reviews have identified diffusion models as one of the dominant technical paradigms in current image generation research because of their advantages in generation quality, visual detail, and training stability [17]. In text-to-image generation, research attention has also shifted from whether images can be generated from textual descriptions to how generated images can more reliably satisfy predefined semantic constraints [18]. Studies represented by DreamArtist have demonstrated that when generation tasks involve preserving specific concepts, maintaining visual characteristics from limited samples, or selectively enhancing local visual features, open-ended natural-language prompts alone often fail to produce sufficiently stable and controllable results. Additional adaptation mechanisms are therefore required to improve control over the generation process [19]. Similarly, studies on locally controllable text-to-image generation have shown that insufficient correspondence between textual constraints and visual details can easily lead to inconsistencies in local structural integrity and semantic logic [20]. These findings are directly relevant to the present study, whose objective is to preserve key personalized characteristics of the input individual, including age group, gender, and clothing attributes, while simultaneously integrating distinctive visual features of fish species associated with the Yangtze River Basin, such as colour ranges, textures, and morphological structures. Accordingly, a more explicitly structured condition-organization layer is required at the front end of the generation pipeline so that human-specific attributes and fish-related visual features can be embedded into subsequent generation stages in a stable and interpretable manner.

Existing controllable image generation methods introduce external conditions to guide image synthesis. Zhang et al. [21] proposed ControlNet, which incorporates spatial information such as edges, depth maps, and human poses. Multiple ControlNet branches can also be combined (Multi-ControlNet) to support simultaneous control using different spatial conditions. Mou et al. [22] developed T2I-Adapter to introduce additional visual control signals into diffusion models. Unlike these methods, which primarily focus on model-level conditioning, the present study transforms recognized human attributes into structured semantic descriptions. Age group and gender constrain character appearance, while clothing colour and category are mapped to fish-related chromatic and morphological features. This rule-based approach emphasizes interpretability and traceability without requiring multiple ControlNet branches. The two approaches are complementary, although their relative performance has not been directly evaluated.

Cross-domain visual-semantic mapping provides the theoretical basis for integrating human attributes with the visual characteristics of Yangtze River fish species. Although existing studies have not directly addressed the generation of mermaid-like virtual humans, they offer important methodological references for the construction of the mapping mechanism proposed in this study. A central research problem in this field is how to transform graphical features, local structures, and symbolic relationships within visual objects into structured semantic representations that can be continuously parsed, retrieved, and optimized by computational systems. Research on the semantic interpretation of graphical symbols has shown that the correspondence between visual forms and semantic meanings directly affects the efficiency and stability of cross-domain translation. Stronger visual relevance and clearer mapping logic generally contribute to more robust semantic decoding [23]. Studies on semantic mapping and vision-language cross-modal fusion have further suggested that basic visual detection results alone are insufficient to ensure the completeness and generalizability of semantic mapping in complex application scenarios. Instead, strategies such as label-system integration, preservation of core semantic information, and enhancement of vision-language representations are required to improve the reliability and applicability of mapping results [24].

Research on structured data transformation provides another relevant methodological perspective. Liu et al. introduced TINTOlib, a Python library that converts tabular data into synthetic image representations for subsequent processing by deep neural networks [25]. Although TINTOlib addresses a different computational task, it illustrates the importance of explicit transformation procedures and unified data interfaces when connecting heterogeneous representations. In contrast to tabular-to-image encoding, the present study translates discrete human attribute labels into domain-specific visual-semantic descriptors that are subsequently incorporated into conditional image generation. The relevance of TINTOlib to this work therefore lies in the general principles of structured representation transformation and modular data processing, rather than in an equivalent semantic mapping algorithm.

These studies highlight the importance of a structured intermediate semantic layer in cross-domain visual mapping. There is no inherent one-to-one correspondence between human visual attributes and the visual prototypes of Yangtze River fish species, which constitutes a central challenge of the present study. More specifically, a standardized mapping mechanism and feature library that simultaneously provide interpretability and practical applicability remain lacking for cross-domain mapping between multidimensional human attributes and the visual characteristics of fish species occurring in the Yangtze River Basin [26]. To address this gap, this study constructs a customized set of mapping rules and a semantic library linking human attributes with the visual features of Yangtze River fish species. Through a standardized intermediate translation process, four core attributes obtained from front-end recognition—clothing colour, clothing category, age group, and gender—are transformed into visual-semantic descriptions that can be effectively interpreted by the generative model. This approach avoids reducing the generation process to a mechanical combination of abstract concepts and enables a deeper integration of personalized human characteristics with the biological visual features of Yangtze River fish species.

3. Methodology

3.1. System Overview

This study develops a task-oriented virtual human generation framework that connects established human attribute recognition and image generation technologies through a structured cross-domain semantic interface. The framework consists of four stages: human attribute recognition, structured encoding, visual-semantic mapping, and conditional image generation. Its methodological focus is the transformation of recognized human attributes into interpretable, task-specific generation conditions. Specifically, age group and gender are organized as character-level appearance constraints, while clothing colour and clothing category are translated into fish-inspired chromatic, textural, and morphological descriptors through predefined mapping rules. The resulting semantic fields are then assembled into structured descriptions that guide the generation of personalized mermaid-like characters. By explicitly linking recognized attributes, intermediate semantic representations, and generation inputs, the framework establishes a traceable processing procedure that can be adapted to other visual themes through modifications to the prototype library and mapping data. The overall system framework is shown in Figure 1.

Figure 1. Framework of the personalized mermaid-like virtual human generation system based on human attribute recognition. The proposed pipeline consists of human attribute recognition, structured attribute encoding, cross-domain visual-semantic mapping, conditional image generation, and result presentation. The lower section illustrates (A) the technical foundation supporting the main generation pipeline and (B) the evaluation and feedback mechanism for assessing system performance.

The system receives real-time image input from an on-site camera. When a person is detected within the central region of the frame, the system temporarily acquires a clear image frame for human attribute recognition, extracting three main categories of information: gender, age group, and clothing-related visual characteristics. In this study, gender is operationalized using two appearance labels, male and female, to define the basic gender-related appearance of the generated virtual character. Age is divided into four categories—child, teenager, adult, and older adult—based on a simplified grouping derived from WHO age-grouping recommendations and adapted to the requirements of the present generation task [27]. Clothing-related visual characteristics comprise two dimensions: clothing colour and clothing category. Clothing colour is represented using a simplified set of 11 basic colour labels, whereas clothing categories are selected with reference to relevant national standards and the semantic compatibility of text-to-image generation models. This design results in a standardized clothing-label system that is suitable for generation control.

After entering the structured encoding module, the human attribute recognition results are reorganized into four predefined fields: age group, gender, clothing colour, and clothing category. This process produces a standardized representation that is readable, recordable, and directly usable in subsequent computational procedures. The encoded attributes are then passed to the visual-semantic mapping module, where interpretable correspondences are established between human attributes and the visual prototypes of Yangtze River fish species. Age group and gender primarily provide stage-related and basic appearance constraints for the anthropomorphic character, whereas clothing colour and clothing category serve as the principal mapping variables, corresponding to the colour characteristics and structural features of the fish prototypes, respectively. The mapping outputs are subsequently organized into generation-description fields and passed to the input of the diffusion-based image generator, allowing image generation to be performed under the joint constraints of human attributes and fish-specific visual features.

During the conditional image generation stage, the generation-description fields are used as unified inputs to a locally deployed diffusion-model workflow. These fields control key visual characteristics of the generated mermaid-like virtual human, including character configuration, dominant colour range, local textures, body contour, and tail structure. The resulting images are then presented through the display interface together with corresponding science communication information on the associated Yangtze River fish species. In this way, the generated results provide not only personalized visual representations but also additional value for science communication.

At the system-integration level, a local scheduling mechanism is adopted to connect the recognition, visual-semantic mapping, and conditional generation modules into a unified pipeline. The human image captured by the camera is first passed to the human attribute recognition module, which outputs four structured attributes: age group, gender, clothing colour, and clothing category. These results are subsequently transferred to the visual-semantic mapping module and reorganized into generation-description fields. The resulting fields are then written into the diffusion-model workflow to generate a personalized mermaid-like virtual human image. The system-integration layer is also responsible for returning intermediate results and presenting the final visualization, thereby ensuring clear, stable, and traceable data flow throughout the entire pipeline.

To ensure the verifiability of the proposed method while protecting participant privacy, the system records only textual recognition and processing results at key stages. These records include attribute recognition outputs, structured encoding results, semantic mapping results, and generation-description fields. This design provides a traceable data basis for experimental evaluation and result analysis, while enabling the proposed cross-domain semantic mapping and conditional generation mechanisms to be evaluated within a unified framework.

3.2. Human Attribute Recognition

Human attributes refer to observable visual characteristics that describe a person’s appearance and state, including gender, age, clothing category, clothing colour, pose, and accessories. Before a human image enters the generation pipeline, human attribute recognition is performed to extract information that can reliably characterize the individual’s appearance. In this study, three categories of human information are considered: gender, age group, and clothing-related visual characteristics. The recognition tasks are performed automatically using visual models without requiring manually entered textual descriptions, thereby ensuring consistency in the source of input information.

For gender recognition, two labels, male and female, are used in this study. The visual model determines the gender label based on facial structural features and overall appearance contours and outputs a discrete classification result. This label serves as a basic constraint on the appearance of the generated virtual character, helping maintain consistency between the generated result and the input individual in terms of gender-related visual characteristics.

To improve recognition stability, age is represented using discrete age groups rather than an exact numerical estimate. The system determines the age interval of an individual according to visual appearance characteristics. Based on the age-grouping recommendations of the World Health Organization (WHO), and considering the requirements of the proposed virtual character generation task, this study consolidates the recommended age intervals into four categories: child (0–9 years), teenager (10–24 years), adult (25–59 years), and older adult (≥60 years) [27]. This simplified grouping provides standardized age-related appearance constraints for subsequent structured attribute encoding and conditional image generation. This interval-based representation confines age-estimation errors within predefined ranges and provides a more stable semantic basis for subsequent visual-semantic mapping. Differences among the age groups can also be reflected in the generated virtual humans through variations in facial texture, facial proportions, and overall appearance characteristics.

Clothing-related visual characteristics are represented along two dimensions: clothing colour and clothing category. For clothing colour recognition, the dominant colour is defined as the colour occupying the largest proportion of the clothing region. The system performs pixel-level analysis of the human region, calculates the distribution of different colour channels, and identifies the colour with the highest proportion as the dominant clothing colour. To ensure consistency and reproducibility of colour labels across different scenarios, the recognized colours are mapped to a simplified standardized colour set derived from GB/T 21898-2023 [28], Textiles—Method for Colour Representation. The resulting label set consists of 11 basic colours: black, white, grey, red, orange, yellow, green, blue, purple, pink, and brown. This standardized representation ensures that model outputs can be consistently assigned to a fixed set of colour labels. For clothing category recognition, no single mandatory classification standard has yet been universally adopted in the computer vision field. Existing studies generally construct clothing taxonomies based on annotation schemes from widely used fashion datasets such as DeepFashion. Considering both the generalization capability of the recognition model and the semantic compatibility required for subsequent image generation, this study refers to commonly used fashion-vision classification schemes and further conducts preliminary experiments to select clothing categories with high recognition accuracy and high compatibility with generative models. Twelve clothing labels are ultimately retained: T-shirt, shirt, long-sleeved top, short-sleeved top, vest, sweater, hoodie, jacket, suit jacket, skirt, dress, and trench coat. Since the proposed system only requires visual information related to upper-body garments and skirts or dresses, categories such as trousers, shoes, and accessories are excluded. These selected categories correspond to basic garment types with visually distinctive characteristics, exhibit relatively stable recognition performance in both public datasets and real-world images, and can be directly interpreted and rendered as textual prompts by image generation models.

3.3. Structured Attribute Encoding

The outputs of the human attribute recognition module are typically represented as discrete labels. However, the output fields, value ranges, and representation formats vary across different recognition tasks. Directly transferring raw recognition results to subsequent visual-semantic mapping and image generation modules may lead to inconsistent attribute fields, ambiguous label interpretations, and incompatible input formats across modules. To ensure consistency and reusability in data exchange throughout the system pipeline, a structured attribute encoding stage is introduced to standardize and reorganize the recognition results.

In this study, the attribute record of an individual sample is defined as

$$x=(a,g,c,k)$$

(1)

where a, g, c, and k denote age group, gender, clothing colour, and clothing category, respectively. According to the age-group definitions introduced above, age is divided into four categories: child, teenager, adult, and older adult. Gender is represented using two discrete labels, male and female. Clothing colour is restricted to the predefined set of 11 basic colours, whereas clothing category is limited to the 12 garment labels directly relevant to the present task. Through these unified constraints, the raw recognition outputs are transformed into standardized attribute records with fixed fields, a fixed ordering, and predefined value ranges.

For computational representation, the attribute record is further encoded as a discrete vector:

$$z=[f_a(a),f_g(g),f_c(c),f_k(k)]$$

(2)

where f_a, f_g, f_c, and f_k denote the encoding functions for age group, gender, clothing colour, and clothing category, respectively. For example, according to the attribute representation defined in Equation (1), an individual characterized as “teenager, female, blue, dress” can be represented as x = (teenager, female, blue, dress), which can then be encoded into the corresponding discrete vector. This encoding procedure is not introduced solely for mathematical representation. More importantly, it provides a unified data interface among the recognition, semantic mapping, and generation modules, enabling different modules to operate sequentially and interact consistently on the basis of the same attribute structure.

3.4. Cross-Domain Semantic Mapping

In this study, cross-domain semantic mapping is used to translate human appearance information into retrievable visual features derived from Yangtze River fish prototypes and subsequently incorporate these features into the generative model. Four mapping variables are retained: age group, gender, clothing colour, and clothing category. However, these variables serve different functions within the system.

Age group and gender are not used as the primary variables for selecting fish prototypes. Instead, they are introduced as character-level constraints during generation. This is because age group and gender primarily affect the developmental stage and basic appearance of the anthropomorphic character, whereas differences among fish prototypes are mainly reflected in body contour, colour distribution, fin structure, and surface texture. These fish-specific characteristics exhibit more direct visual correspondences with clothing colour and clothing category. Based on this rationale, gender is defined as a basic appearance constraint and does not participate in fish prototype selection; it is used only to determine whether the generated character follows a mermaid or merman configuration. Age group is defined as a stage-related constraint that controls the perceived facial age, proportional characteristics, and texture density of the generated character. It is incorporated into the generative model through four age-specific interfaces: child mermaid/child merman, young mermaid/young merman, adult mermaid/adult merman, and elderly mermaid/elderly merman.

Clothing colour and clothing category are defined as the primary mapping variables. Clothing colour is used to determine the dominant colour range and local texture colours of the fish visual prototype, whereas clothing category is used to determine its contour and structural characteristics. This design is based on the observation that both colour and clothing category are appearance attributes that can be recognized relatively reliably by visual models and correspond to fish body colouration and morphological structure at a comparable visual level. Consequently, this mapping strategy maintains the interpretability of the proposed method while preserving a clear generation-control pathway.

After defining the roles of age group, gender, clothing colour, and clothing category in the mapping process, a corresponding set of fish visual prototypes is required. Given the large diversity of fish species in the Yangtze River Basin and the substantial differences among species in body colouration, morphology, texture, and local structural characteristics, this study does not employ a complete taxonomic classification system. Instead, representative fish visual prototypes are selected primarily according to the distinctiveness and translatability of their visual features. Finally, a visual prototype set consisting of six representative fish species is established to provide feature references for subsequent cross-domain semantic mapping.

3.5. Conditional Image Generation

The conditional image generation mechanism is responsible for transforming the human attribute constraints and fish visual prototype features obtained in the preceding stages into executable generation conditions and producing the final images within a locally deployed environment. In this study, image generation is performed using a locally deployed diffusion-based generative model. Here, the diffusion model refers to a generative framework that starts from random noise and progressively denoises it under the guidance of conditional information to reconstruct the target image.

During image generation, age group and gender are used to constrain the age-related appearance and basic character configuration, whereas the fish visual features derived from clothing colour and clothing category are used to control the dominant colour range, local textures, body contour, and caudal-fin structure. These conditions are integrated into a unified generation-description template and used as textual conditioning inputs for the diffusion model. This enables the generated mermaid-like images to preserve the basic appearance characteristics of the input individual while simultaneously incorporating the structural features of the corresponding Yangtze River fish visual prototype.

Preliminary experiments showed that when only a general-purpose diffusion model and textual prompts were used, the generated images were prone to structural defects, including missing fish tails, unnatural transitions between the human body and the tail, and outputs that resembled conventional human portraits rather than complete mermaid-like characters. To improve the stable representation of the composite structure consisting of a human upper body and a fish tail, a dedicated mermaid character model was introduced on top of the base diffusion model. This additional model enhances the structural completeness and morphological consistency of the generated mermaid-like characters.

The final generated images are visualized on a local display device. The system also supports image export in PNG format and automatically generates corresponding QR codes, allowing the generated results to be accessed and shared via mobile devices. This output mechanism improves the presentability and reusability of the generated content and provides a reliable output format for subsequent visual analysis and practical applications.

4. Design of the Human Attribute and Visual Feature Recognition Module

4.1. Construction and Local Deployment of the Age and Gender Recognition Model

To select an appropriate recognition method, a comparative evaluation was conducted using six representative models: LBP, 4c2f-CNN, RoR-34 + SD, Two-level CNN, MiVOLO-D1 Face, and FaceXFormer. These models represent six methodological categories, including traditional handcrafted-feature methods, early convolutional neural networks, deep convolutional neural networks, multitask deep learning, multitask Transformers, and unified Transformer frameworks. Their performance was evaluated on age-group and gender recognition tasks, with the comparative results presented in Table 1. The comparison focused primarily on recognition accuracy and inference time, while the architectural characteristics and suitability of the candidate methods for local deployment were considered during model selection. The test set consisted of 800 AI-generated upper-body portrait images, including 400 male and 400 female characters. The images covered variations in age-related appearance, hairstyles, and clothing characteristics to provide diverse visual samples for comparing age-group and gender recognition models. All experiments were conducted using an NVIDIA RTX 4080 GPU. Inference time was measured as the average processing time per image under the same experimental conditions.

Table 1. Performance comparison of different human attribute recognition methods.

Method Category	Method/Model	Gender Accuracy	Age-Group Accuracy	Inference Time (ms/Image)
Traditional handcrafted-feature method	LBP	85.4%	72.8%	2.1
Early convolutional neural network method	4c2f-CNN	91.2%	80.6%	3.4
Deep convolutional neural network method	RoR-34 + SD	94.7%	86.9%	6.8
Multitask deep learning method	Two-level CNN	95.3%	88.4%	8.1
Multitask Transformer method	MiVOLO-D1 Face	96.5%	90.8%	10.6
Unified Transformer framework	FaceXFormer	97.1%	92.6%	12.9

Note: All six models were evaluated under the same experimental conditions using a test set of 800 samples (400 male and 400 female) on an NVIDIA RTX 4080 GPU. The reported results include gender-recognition accuracy, age-group recognition accuracy, and average inference time per image.

As shown in Table 1, age-group recognition provides greater differentiation among the evaluated methods than gender recognition. Traditional approaches and shallow CNNs maintain a generally usable level of performance for gender recognition, but exhibit a more pronounced decline in age-group recognition. In contrast, deep CNNs, multitask learning methods, and Transformer-based approaches demonstrate more substantial advantages for age-group classification. In terms of inference time, traditional methods and shallow CNNs generally demonstrate faster processing speeds, whereas deep CNNs, multitask learning models, and Transformer-based approaches achieve higher recognition accuracy at the cost of increased computational time. For an on-site interactive installation, method selection must therefore consider overall system-level performance rather than recognition accuracy alone, particularly whether the selected model can provide stable outputs for subsequent structured encoding and generation control.

Based on the comparative evaluation, the Two-level CNN multitask framework was selected for joint age-group and gender recognition. Although Transformer-based models achieved higher recognition accuracy, the Two-level CNN attained gender and age-group recognition accuracies of 95.3% and 88.4%, respectively, with an average inference time of 8.1 ms per image. In comparison, MiVOLO-D1 Face and FaceXFormer required 10.6 ms and 12.9 ms per image, respectively. Considering the requirements of local deployment and the role of attribute recognition as an input stage for subsequent semantic mapping and image generation, the Two-level CNN was considered a suitable compromise between recognition accuracy, processing efficiency, and system integration requirements. Moreover, both age-group and gender information are extracted from the same facial image and therefore share low-level visual representations. Joint modelling reduces redundant computation and is naturally compatible with a multitask classification framework, while also facilitating subsequent structured encoding, visual-semantic mapping, and conditional image generation.

For age-group and gender recognition, the system first preprocesses the input image and applies a face-detection module to locate valid facial regions. If no face is detected, an empty result is returned. When multiple faces are present, the primary face is selected according to the size of the detected facial region or its proximity to the centre of the image. The selected face is then processed by the joint attribute recognition model to predict the original age value and gender category. The gender prediction is directly mapped to either a male or female label, whereas the predicted age is further mapped to one of four predefined age groups: child, teenager, adult, and older adult. The final output consists of the combined gender and age-group labels.

The age-group and gender-related appearance labels used in this study are task-specific outputs derived from visual recognition models. Their predictions may be affected by differences in facial appearance, lighting, pose, occlusion, and the demographic composition of the underlying training data. In addition, the binary appearance labels used by the current model do not constitute a comprehensive representation of gender identity. These outputs are therefore employed only as visual appearance constraints for virtual character generation and should not be interpreted as definitive personal identity attributes. Potential recognition biases and their downstream effects remain limitations of the current framework.

4.2. Multidimensional Modelling and Recognition of Clothing Visual Features

In this study, clothing visual features constitute an important component of the human appearance input. Unlike age group and gender, which primarily serve as character-level constraints, clothing colour and clothing category are directly incorporated into the subsequent visual-semantic mapping process and are used to control the dominant colour system and contour-related structural characteristics of the fish visual prototypes, respectively. Existing studies have shown that clothing colour analysis is more appropriately performed through clothing-region extraction, colour-space transformation, and dominant-colour clustering [29], whereas clothing category recognition generally relies on image classification, object detection, or fine-grained clothing attribute recognition models [8].

4.2.1. Clothing Colour Recognition

To extract reusable and mappable colour labels that can be directly incorporated into the generation-control pipeline, this study defines 11 standardized clothing colours: black, white, grey, red, orange, yellow, green, blue, purple, pink, and brown. Clothing colour recognition is implemented through clothing-region extraction, K-means dominant-colour clustering, HSV colour-space transformation, and standardized colour classification. The HSV colour space separates chromatic information from brightness, facilitating the classification of the extracted dominant colour into standardized colour labels. This approach is highly compatible with the overall system pipeline because it produces stable, interpretable, and discrete colour labels that can be directly mapped to the dominant body colour, fin-edge colour, and local texture colour of the corresponding fish visual prototype.

For clothing colour recognition, the extracted clothing region is resized to 64 × 64 pixels, and K-means clustering is performed on the BGR pixel values with K = 3, k-means++ initialization, n_init = 1, max_iter = 300, and a convergence tolerance of 10⁻⁴, without a fixed random seed. The centroid of the largest cluster is selected as the dominant colour and converted into the HSV colour space using OpenCV, where H ranges from 0 to 360°, while S and V are normalized to [0, 1]. The dominant colour is then classified into one of 11 predefined categories using the HSV thresholds presented in Table 2. The resulting colour label is used for subsequent structured attribute encoding and visual-semantic mapping.

Table 2. HSV threshold rules for the 11 clothing colour categories.

Colour Category	HSV Threshold Conditions
Black	V < 0.18
White	S < 0.12 and V > 0.85
Grey	S < 0.20
Red	H ≥ 345° or H < 15°; also 300° ≤ H < 345° and V < 0.45
Orange	15° ≤ H < 35°
Yellow	35° ≤ H < 65°, excluding the brown condition
Green	65° ≤ H < 170°, excluding the brown condition
Blue	170° ≤ H < 255°
Purple	255° ≤ H < 300°
Pink	300° ≤ H < 345° and V ≥ 0.45
Brown	35° ≤ H < 65°, V < 0.55, S < 0.75; or 65° ≤ H < 85°, V < 0.45

Note: H is expressed in degrees, while S and V are normalized to [0, 1]. The rules are applied sequentially, with black, white, and grey evaluated before chromatic colours. Additional saturation and brightness conditions are used to distinguish neighbouring colour categories.

4.2.2. Clothing Category Recognition

Existing approaches to clothing category recognition can generally be divided into four categories. The first comprises deep learning methods based on holistic classification of clothing regions, such as FashionNet and its subsequent extensions, which are primarily designed for category prediction on cropped garment images [8,30]. The second category includes joint learning approaches for clothing categories and attributes, such as M2Fashion, which simultaneously model clothing categories and additional visual attributes and therefore provide strong representational capability [31]. The third category comprises single-stage object detection methods within the YOLO family. Chang and Zhang [32] demonstrated the application of YOLOv5s to clothing pattern recognition, while the more recent YOLO11 framework provides object detection capabilities that can be adapted to clothing category recognition through task-specific training [10]. The fourth category is represented by fine-grained clothing detection and attribute recognition frameworks such as Fashionpedia. Although such approaches provide rich category and attribute taxonomies, they also require substantially greater annotation and deployment costs [33]. Models such as FashionNet, SEResNeXt50 + OC, and M2Fashion have demonstrated strong performance in clothing classification or joint clothing-attribute learning; however, they are primarily designed for cropped clothing images or fine-grained category and attribute prediction, making them more suitable for clothing recognition tasks themselves than for the system requirements considered in this study.

Based on a comprehensive comparison, this study adopts a YOLO11-based single-stage object detection approach for clothing category recognition. As one of the representative real-time detection paradigms, YOLO11 provides end-to-end detection capability and is well aligned with the task characteristics of this study, which involve on-site human image acquisition and recognition within a predefined set of 12 clothing categories. The method provides an appropriate balance among recognition accuracy, computational efficiency, system compatibility, and feasibility of local deployment. Accordingly, the clothing visual feature recognition module is organized into two parallel yet complementary technical pathways. The colour recognition pathway generates standardized dominant-colour labels as inputs for subsequent mapping to the dominant colour characteristics of fish prototypes, whereas the category recognition pathway produces basic clothing category labels for subsequent mapping to fish contour and structural features. The outputs of both pathways are finally unified at the structured encoding layer to form a standardized representation of clothing visual features, which serves as one of the primary input layers for the overall cross-domain visual-semantic mapping and conditional image generation pipeline.

5. Cross-Domain Visual-Semantic Mapping and Conditional Virtual Human Generation

5.1. Construction of the Cross-Domain Visual-Semantic Mapping Mechanism

5.1.1. Selection and Definition of Yangtze River Fish Visual Prototypes

In this study, six representative fish species associated with the Yangtze River Basin were selected as visual prototypes to examine the feasibility of the proposed cross-domain visual-semantic mapping strategy. The selection was guided by four considerations. First, all selected species are relevant to the Yangtze River aquatic theme of the application scenario, ensuring consistency between the visual prototypes and the intended ecological and cultural context. Second, the selected species exhibit distinguishable visual characteristics in body morphology, colouration, surface patterns, and local structures, providing diverse feature references for semantic translation. Third, priority was given to species with regional representativeness or a degree of public recognizability, making the resulting visual associations more accessible to readers and exhibition audiences. Fourth, the selected visual characteristics needed to be identifiable from available morphological references and expressible as controllable descriptors for subsequent image generation. Rather than aiming for comprehensive taxonomic coverage, this task-oriented selection provides a manageable and visually differentiated prototype set for evaluating the proposed mapping mechanism.

According to the Bulletin on the Status of Aquatic Biological Resources and Habitats in the Yangtze River Basin (2025), a total of 351 native fish species were recorded in the Yangtze River Basin during 2021–2025 [34]. Considering the substantial diversity of fish species in the basin, it is neither necessary nor practical to include every species when evaluating a task-specific visual-semantic mapping framework. Therefore, six species were selected as representative examples: Chinese sturgeon, Yangtze sturgeon, Chinese sucker, mandarin fish, yellow catfish, and Japanese grenadier anchovy. These species differ in their overall contours, colour distributions, surface patterns, and distinctive morphological features, allowing the study to examine the mapping of different visual attributes within a limited and manageable prototype set. The number of selected species reflects the scope of the experimental demonstration rather than a biological classification requirement or a fixed limitation of the proposed framework.

The six selected species provide different morphological and chromatic references for visual-semantic mapping. The Chinese sturgeon (Acipenser sinensis) exhibits an elongated body and distinctive snout, providing references for extended contours and restrained colour schemes [35]. The Yangtze sturgeon (Acipenser dabryanus) is characterized by a comparatively shorter snout and prominent bony scutes, supporting the extraction of compact structural and scute-related texture descriptors [36]. The Chinese sucker (Myxocyprinus asiaticus) exhibits a prominent dorsal fin and distinctive colour patterns that vary with developmental stage and sex, providing references for decorative patterns and contrasting chromatic features [37]. The mandarin fish (Siniperca chuatsi) has a deep, laterally compressed body, irregular dark markings, and a pronounced mouth, contributing deep-bodied contours and patterned textures [38]. The yellow catfish (Tachysurus fulvidraco) provides references for barbel-related details and warm-toned visual interpretations [39]. The Japanese grenadier anchovy (Coilia nasus) is characterized by a slender, tapering body and elongated pectoral-fin rays, supporting streamlined and lightweight morphological descriptors [40]. These documented biological characteristics provide the reference basis for feature extraction, whereas descriptors such as solemn, graceful, decorative, and robust represent artistic interpretations used in the virtual character design process rather than formal biological characteristics.

The proposed mapping strategy is not restricted to the six selected fish species. Its general processing logic consists of human attribute encoding, visual feature translation, and structured generation-condition construction. When introducing additional fish species or adapting the framework to another visual theme, the core processing sequence can be retained while the visual prototype library, feature descriptors, and corresponding mapping entries are updated. For example, new species can be incorporated by defining their representative colour ranges, surface patterns, morphological characteristics, and associated textual descriptors. In this way, the six-species prototype library serves as an application-specific demonstration rather than a fixed boundary of the mapping strategy. The present study validates the feasibility of this approach within the selected Yangtze River fish-themed scenario and provides a methodological reference for extending it to other visual prototypes.

5.1.2. Functional Roles of Mapping Variables

In the visual-semantic mapping module, the four attribute variables are extracted from the same human image but serve different functions within the mapping mechanism. Therefore, they are divided into two groups: primary mapping variables and constraint variables. Based on the standardized attribute record x defined in Equation (1), the variables are further partitioned according to their functional roles as

$$R_m=\{c,k\},\hspace{1em}\hspace{1em}{R}_c=\{a,g\}$$

(3)

where R_m denotes the set of primary mapping variables that determines the visual direction of the fish prototype, whereas R_c denotes the set of constraint variables that controls the age-related characteristics and basic appearance of the anthropomorphic character.

According to the variable partition defined in Equation (3), Clothing colour and clothing category are assigned to the primary mapping variable set because they exhibit the most direct visual correspondences with the dominant colour range, local texture colours, decorative colour patches, contours, and structural characteristics of the fish visual prototypes. For example, a person wearing a tight-fitting black long-sleeved garment can be associated with the dark colouration and compact elongated body form of the Chinese sturgeon. Accordingly, clothing colour controls the chromatic direction of the mapping process, whereas clothing category controls the morphological direction. Together, these two variables constitute the primary mapping layer.

Age group and gender, by contrast, are primarily used to constrain the age and gender characteristics of the final mermaid-like character and do not directly determine the selection of fish visual prototypes. They are therefore categorized as constraint variables. In the subsequent image generation stage, exact numerical age values are not required because current generative models cannot reliably produce images that precisely correspond to a specific chronological age. For this reason, age is represented using four discrete stages: child, teenager, adult, and older adult. Gender plays a similar role but imposes a stronger categorical constraint. Only two outputs, male and female, are retained, and according to the training logic of the mermaid character model, the terms “merman” and “mermaid” are used as generation trigger words for male and female characters, respectively. These constraints define the basic gender-related appearance of the generated character and improve the stability and controllability of the generation process.

5.1.3. Construction of Visual-Semantic Mapping Rules

After the hierarchical roles of the input variables have been defined, the core task of the mapping module is to transform the standardized recognition results into visual-semantic fields that can be incorporated into the descriptive representation of the generated mermaid-like character. Rather than directly using fish species names to drive image generation, the proposed mapping mechanism decomposes the process into three parallel rule chains: character-style semantic mapping, colour semantic mapping, and structural semantic mapping. The outputs of these three mapping pathways are then integrated into a unified textual conditioning description. This design is adopted because diffusion models may respond inconsistently to abstract species names, particularly when the training data do not contain sufficiently accurate or comprehensive visual examples of the corresponding species. In contrast, decomposable descriptors related to colour, contour, texture, tail morphology, and anthropomorphic character style provide more explicit and controllable generation conditions.

Using the standardized input attribute record x defined in Equation (1), the visual-semantic mapping output is represented as

$$y=(q,t,s)=F(x)=(f_q(a,g),f_t(c),f_s(k))$$

(4)

where q, t, and s denote anthropomorphic character-style semantics, fish colour semantics, and fish structural semantics, respectively. Correspondingly, f_q maps age group and gender to anthropomorphic character style and appearance direction, f_t maps clothing colour to the dominant body colour and texture colours of the fish prototype, and f_s maps clothing category to body morphology, tail structure, and upper-body structural characteristics.

In the present implementation, these functions represent predefined, rule-based semantic transformations rather than trainable predictive models or optimization functions designed to minimize semantic drift. Based on the mapping relationships presented in Table 3, Table 4 and Table 5, they convert standardized human attributes into character-style, colour, and structural descriptors, which are subsequently assembled into structured generation conditions. This process establishes explicit and traceable correspondences between input attributes and generation descriptions.

Table 3. Visual-style semantics of Yangtze River fish prototypes and their associations with gender and age group.

Yangtze River Fish Species	Extractable Visual-Style Semantics	Gender Association	Age-Group Association
Chinese sturgeon	solemn, archaic, imposing, elongated	no clear gender preference	teenager/adult/older adult
Yangtze sturgeon	composed, robust, rugged, compact	male-oriented	adult/older adult
Chinese sucker	dynamic, vivid, decorative, graceful	female-oriented	child/teenager
Mandarin fish	deep-bodied, patterned, angular, distinctive	male-oriented	teenager/adult
Yellow catfish	soft, rounded, compact, gentle	no clear gender preference	child/older adult
Japanese grenadier anchovy	slender, lightweight, dynamic, concise	female-oriented	teenager

Note: Gender and age-group associations represent soft semantic preferences for character generation rather than deterministic rules for fish-prototype selection. These associations are artistic design preferences and should not be interpreted as biological age- or sex-specific characteristics of the fish species.

Table 4. Semantic mapping between clothing colours and fish colour characteristics.

Clothing Colour (11 Basic Colours)	Corresponding Fish Colour Range/Texture Colour	Colour-Mapping Details
Black	Dominant colour range of Yangtze sturgeon	Dark grey/dark blue low-saturation cool-toned base; no gradient or highlight; saturation ≤ 0.2
Grey	Dominant colour range of Chinese sturgeon	Silver-grey/light-grey low-saturation cool-toned base; uniform overall colouration; saturation ≤ 0.3
Blue	Dominant colour range of Chinese sturgeon	Grey-blue low-saturation cool-toned base; fins and scute edges outlined in analogous hues; no highly saturated colours
Red	Dominant colour range of Chinese sucker	Highly saturated red-orange dominant colour; core colour of oblique lateral bands; saturation ≥ 0.6; purple-toned gradients permitted
Purple	Texture colour of Chinese sucker	Highly saturated purple texture; red-purple gradient base; gradient accent along caudal-fin edges; saturation ≥ 0.6
Pink	Auxiliary/outline colour of Chinese sucker	Red-pink gradient accent; used for fin and texture-edge outlining; highly saturated light tone used only for decoration
Orange	Dominant colour range of Chinese sucker	Red-orange gradient as the dominant colour; auxiliary colour for oblique lateral bands and head-contour outlining; saturation ≥ 0.6
Yellow	Dominant colour range of yellow catfish	Warm ginger-yellow base; monochromatic matte appearance without gradients or highlights; saturation 0.3–0.5
Brown	Dominant colour range of yellow catfish	Warm yellow-brown base; used for outlining barbel-like structures around the head; monochromatic matte appearance; saturation 0.3–0.5
Green	Dominant colour range of mandarin fish	Dark green/light-brown low-saturation dark-toned base; background colour for dark patches and transverse patterns; saturation ≤ 0.3
White	Dominant colour range of Japanese grenadier anchovy	Silver-white/bright-silver light-toned base with highlights; strong reflective highlights on the tail; metallic appearance; brightness ≥ 0.8

Note: The saturation and brightness values indicate the semantic ranges used in the colour-mapping rules. These mappings characterize dominant body colour, texture colour, saturation, and highlight features rather than direct pixel-level colour replacement.

Table 5. Semantic Mapping Between Clothing Categories and Fish Structural Characteristics.

Clothing Category (12 Basic Types)	Extractable Visual Characteristics of Yangtze River Fish	Core Structural Semantics
T-shirt	Chinese sturgeon: elongated and extended body form with a straight trunk and no prominent protruding structures; Japanese grenadier anchovy: smooth body surface without excessive texture	extended + minimal
Short-sleeved top	Chinese sturgeon: elongated, flowing body lines with broad fins naturally integrated with the body; Japanese grenadier anchovy: slender trunk lines without prominent structures	extended + slender
Vest	Chinese sturgeon: compact trunk without protrusions and a continuous line from the elongated snout to the tail; mandarin fish: sharp, narrow fins with a concise structure and minimal decoration	compact + concise
Shirt	Yangtze sturgeon: compact and robust body mass with concentrated dorsal scute textures; Chinese sturgeon: moderately extended contour with subtle scute texture	compact + subtle texture
Long-sleeved top	Yangtze sturgeon: short, robust, and compact body with basic dorsal scute structures; yellow catfish: short, rounded trunk with smooth, non-angular contours	compact + rounded
Sweater	Yangtze sturgeon: visually heavy body mass with slightly relief-like dorsal scute textures; yellow catfish: short, rounded contour with a soft matte surface	robust + soft texture
Dress	Chinese sucker: upward-extending high dorsal fin, oblique lateral decorative patterns, and an extended body contour; Japanese grenadier anchovy: slender trunk and flowing streamlined tail	decorative + extended/slender
Skirt	Chinese sucker: extended body contour, gradient oblique lateral patterns, and broad flowing fins; Japanese grenadier anchovy: flowing streamlined tail lines	extended + decorative + flowing
Jacket	Mandarin fish: deep, laterally compressed, sharp contour with pointed projecting fins; Yangtze sturgeon: robust body mass with rigid dorsal scute structures	sharp + rigid/robust
Trench coat	Mandarin fish: deep, laterally compressed, vertically elongated contour with angular fin and head structures; Chinese sturgeon: elongated, extended, imposing contour with broad fins	elongated + sharp/extended
Hoodie	Yellow catfish: short, robust, rounded and compact contour, barbel-like decorative head structures, and smooth non-angular lines; Chinese sucker: subtle decorative textures and softly rounded fins	rounded + soft + lightly decorative
Suit jacket	Japanese grenadier anchovy: extremely narrow and streamlined contour, smooth untextured surface, and highlighted streamlined tail; mandarin fish: clean and sharp contour with minimal decorative elements	streamlined + slender/sharp + concise

Note: Clothing categories are mapped to fish-related structural semantics at three levels: overall body morphology, tail configuration, and local upper-body features. The resulting descriptors are used as semantic conditions for subsequent character generation.

A key characteristic of the proposed mapping mechanism is the functional separation between human attribute constraints and cross-domain visual feature translation. Rather than treating all recognized attributes as equivalent prompt elements, the mechanism assigns distinct semantic roles to age group, gender, clothing colour, and clothing category. This allows each attribute to be processed through a corresponding mapping function and subsequently integrated into a structured generation description. For instance, an input record of “teenager, female, blue, dress” can be translated into a young female character constraint, blue-grey fish-inspired colour descriptors, and decorative, extended or slender morphological descriptors derived from the visual prototype library. The intermediate representation makes the origin and intended function of each semantic field identifiable. Moreover, because the mapping logic is separated from the visual prototype data, the framework can accommodate other visual themes by updating the prototype library and its associated mapping entries without changing the core semantic processing procedure.

In the proposed mapping rules, age group and gender are treated as soft semantic constraints rather than deterministic mapping variables. As shown in Table 3, each fish prototype is associated with a set of extractable visual-style characteristics, which are further adapted to different age groups and genders through a soft matching strategy. For example, characters in the child group preferentially adopt descriptors such as rounded, compact, soft, and lightweight; those in the teenager group preferentially adopt dynamic, elongated, vivid, and concise characteristics; adult characters emphasize composed, balanced, extended, and robust visual qualities; and older adult characters tend toward archaic, solemn, calm, and restrained characteristics.

Gender further modulates the visual orientation of these character-style descriptors. Female characters tend to emphasize graceful, decorative, lightweight, and flowing characteristics, whereas male characters tend to strengthen robust, upright, sharp, and orderly visual qualities. These age- and gender-related character-style semantics are subsequently combined with the colour and structural semantic fields, enabling the generated characters to exhibit greater visual coherence, aesthetic richness, and morphological diversity while preserving the overall logic of the mapping mechanism.

Colour semantic mapping is driven by clothing colour and is not intended as a simple colour replacement operation. Instead, the input clothing colour is translated into multiple fish-related chromatic attributes, including dominant body colour, texture colour, saturation, and highlight characteristics. Through this mapping mechanism, clothing colours are transformed into hierarchical colour semantics derived from the visual characteristics of Yangtze River aquatic organisms, as summarized in Table 4. Consequently, the generated images can preserve the user’s appearance-related attributes while simultaneously establishing visual associations with fish body colouration and the ecological atmosphere of the aquatic environment.

Structural semantic mapping is driven by clothing category. Its primary purpose is to translate garment silhouette and structural characteristics into the body morphology, tail configuration, and local upper-body features of the generated mermaid-like character. Based on the clothing-to-fish correspondence rules summarized in Table 5, clothing categories are translated into fish-prototype structures at three levels. The first level concerns overall body morphology, such as elongated, compact, vertically extended, or rounded forms. The second level concerns tail morphology, including lightweight streamlined tails, upward-flowing tails, and robust convergent tails. The third level concerns local upper-body structures, such as dorsal scute-like textures, banded decorative patterns, barbel-like edges, or smooth forms without prominent protruding structures.

To ensure that the proposed mapping rules can be consistently invoked within the system, the visual-semantic mapping process is organized into a sequential rule-based workflow. The system first reads the age-group and gender variables to generate character-style semantics. It then processes clothing colour to derive the dominant fish colour range, texture colours, and associated chromatic-control semantics. Finally, clothing category is used to generate semantic descriptors for tail morphology, local upper-body structures, and overall body contour. The outputs from these three semantic levels are subsequently integrated into a unified visual-semantic representation, which is then passed to the following stage for the construction of textual conditioning descriptions.

5.1.4. Representation of Mapping Results and Generation Interface Output

After completing the three types of mapping—character-style semantics, colour semantics, and structural semantics—the system first organizes the resulting information into a unified structured representation to ensure a stable data interface among the front-end recognition layer, semantic mapping layer, and generation control layer.

Diffusion models generally have limited capability to accurately interpret abstract fish species names, and the same species name may lead to substantial variations when combined with different prompts. In contrast, explicit descriptive fields, such as dominant colour, texture, contour, tail morphology, decorative features, and age-related character characteristics, can be interpreted more reliably by the generative model. Therefore, the final output of the mapping layer is represented as a set of generation-description fields with a predefined order and clearly defined semantic boundaries.

The mapping output y, as defined in Equation (4), is further organized into a generation-interface representation:

$$P=(P_a,P_g,P_c,P_k,P_f)$$

(5)

where P_a, P_g, P_c, P_k, and P_f denote the age-group generation interface, gender generation interface, clothing-colour generation interface, clothing-category generation interface, and the visual-semantic feature field set, respectively. The first four components preserve the front-end recognition results directly at the generation layer, whereas P_f contains the fish visual prototype features obtained through the rule-based translation described in the preceding section.

Based on the generation-interface representation defined in Equation (5), the final structured description passed to the generation module is formulated as

$$D=P_a\oplus P_g\oplus P_c\oplus P_k\oplus P_f$$

(6)

where ⊕ denotes sequential concatenation. This ordering is designed to support the interface logic of generation control. Age group and gender are placed at the beginning to define the basic anthropomorphic direction of the generated character. Clothing colour and clothing category are positioned in the middle to preserve the clothing-related appearance information of the input individual. The feature fields are placed at the end to provide detailed descriptions of the dominant colour range, texture, contour, tail morphology, and other characteristics translated from the corresponding fish visual prototype.

To illustrate the complete semantic transformation process, consider an input attribute record x = (teenager, female, blue, dress). According to Equation (4), the character-style mapping function f_q(a,g) produces the appearance constraints of a young female mermaid. The colour mapping function f_t(c) translates the clothing colour blue into low-saturation grey-blue chromatic descriptors associated with the Chinese sturgeon, as defined in Table 4. Meanwhile, the structural mapping function f_s(k) translates the clothing category dress into decorative, extended, and slender morphological descriptors, drawing on the visual characteristics of the Chinese sucker and Japanese grenadier anchovy in Table 5. These three mapping outputs form the intermediate semantic representation y = (q,t,s). Following Equation (5), the generation-interface fields are organized as P_a = “young people”, P_g = “female mermaid”, P_c = “blue”, P_k = “dress”, and P_f = “low-saturation grey-blue colouration, decorative fin patterns, extended slender silhouette, flowing streamlined tail”. According to Equation (6), these fields are sequentially integrated to construct the final generation description D, which can be expressed as: “young female mermaid, blue dress, low-saturation grey-blue fish-inspired colouration, decorative fin patterns, extended slender silhouette, flowing streamlined tail”. This example illustrates how the recognized human attributes are transformed into interpretable intermediate semantic fields and subsequently organized into generation conditions, establishing explicit correspondences between the input attributes, mapping functions, and final generation description.

5.2. Conditional Generation Model Design and Control Strategy

5.2.1. Design of the Local Diffusion-Based Generation Framework

After human attribute recognition, visual-semantic mapping, and generation-interface construction are completed, the system proceeds to the image generation stage. The locally deployed generation system is organized into three components: the model layer, scheduling layer, and output layer. The model layer consists of a latent-space diffusion backbone and a mermaid-character prior module, which jointly generate the target image from textual conditions. The scheduling layer converts the structured descriptions produced by the front-end modules into executable generation conditions and is responsible for model invocation, parameter transfer, and node-order control. The output layer performs image decoding and final visualization. The overall conditional generation process can therefore be formulated as

$$I=G(\xi ,D;{\theta }_b,{\theta }_r)$$

(7)

where I denotes the output image, ξ represents the initial latent-space noise, D denotes the structured generation description defined in Equation (6), θ_b represents the parameters of the base generative model, and θ_r denotes the parameters of the mermaid-character prior module.

The proposed framework adopts a locally deployed diffusion-based generation paradigm, in which the target mermaid-like image is progressively reconstructed from random noise under the guidance of structured conditions. A key advantage of this framework is its ability to simultaneously incorporate multiple sources of conditioning information, including age group, gender, clothing colour, clothing category, and fish visual prototype features. These heterogeneous conditions are jointly integrated within a unified generation process to control character morphology, colour organization, and local structural characteristics.

The generation framework operates in a latent representation space rather than directly synthesizing images in pixel space. Earlier latent diffusion methods, such as the approach developed by Rombach et al. [41], demonstrated the computational advantages of performing generative modelling in compressed visual representations. This work provides relevant background for latent-space image generation. However, the specific backbone employed in the present study is FLUX.1 [dev], which adopts a rectified-flow Transformer architecture, a generative modelling approach supported by previous research on high-resolution image synthesis [42], rather than the conventional UNet-based latent diffusion formulation. The locally deployed workflow combines this generative backbone with structured textual conditions and a character-specific LoRA module, enabling the semantic mapping outputs to be incorporated into the final image synthesis process.

FLUX.1 [dev], developed by Black Forest Labs, is selected as the core generative model in the proposed system. According to its official model documentation [43], FLUX.1 [dev] is a 12-billion-parameter rectified-flow Transformer designed for text-to-image generation. The model provides a text-conditioned generation backbone that can be incorporated into a locally deployed, node-based workflow. In this study, FLUX.1 [dev] is used together with structured generation descriptions and a dedicated character-specific LoRA module to generate mermaid-like virtual characters. The contribution of the present work lies in the task-specific construction and organization of the conditioning information, rather than in modifying the internal architecture of FLUX.1 [dev].

For local implementation, a node-based workflow orchestration framework is constructed in which textual condition organization, base-model invocation, character-prior loading, sampling execution, and image decoding are encapsulated as independently adjustable nodes. These nodes are sequentially connected by the scheduling layer to form a complete generation pipeline. This design provides three main advantages. First, clearly defined module boundaries enable the front-end recognition module, semantic mapping module, and back-end generation module to be decoupled, thereby improving system maintainability. Second, the parameters of individual nodes can be adjusted independently, facilitating experimental comparisons of different sampling strategies and character-prior configurations. Third, the node-based architecture is inherently suitable for automated operation in a local deployment environment, allowing recognition, mapping, generation, and visualization to be integrated into an end-to-end workflow. This reduces manual intervention while improving the stability and repeatability of the system in on-site interactive applications.

5.2.2. Diffusion-Based Mermaid Image Generation and Mermaid Model Training

This study adopts a latent-space diffusion-based generation process, in which the mermaid description prompts produced by the semantic mapping module are injected into the diffusion model to achieve iterative image generation from random noise to the target output. The F.1-dev-fp8 series weights are employed as the backbone of the denoising network, while a dual-encoder architecture is adopted for textual conditioning to jointly incorporate short-text and long-text semantics into the conditioning space. An empty-latent initialization strategy is used to directly construct the initial latent representation at a predefined resolution of 864 × 1536, which satisfies the 9:16 aspect-ratio requirements of most display devices. The Euler sampler is used together with a simple scheduling strategy, with the number of sampling steps set to 25, guidance to 3.5, and denoise strength to 1.0. A sampling configuration of 25 steps provides an appropriate balance between generation efficiency and visual detail. A guidance value of 3.5 maintains effective adherence to the textual conditions while preserving a certain degree of generative flexibility. The simple scheduling strategy defines the progression of sampling steps during image generation and is used as part of the predefined inference configuration. The training logic and workflow of the Mermaid Character LoRA module are illustrated in Figure 2 and Figure 3, respectively.

Figure 2. Training logic of the Mermaid Character LoRA module. A character-specific LoRA module is trained on the base generative model using curated mermaid images to enhance the structural stability of the human–fish composite character while retaining controllable generation parameters.

Figure 3. Training workflow of the Mermaid Character LoRA module. The workflow includes dataset construction and distribution control, image annotation and prompt refinement, training-parameter configuration, iterative LoRA optimization, and model evaluation and selection.

In addition to the base image generation model, a dedicated Mermaid Character LoRA module was trained using a ComfyUI-based workflow and the F.1-dev-fp8 base model. The training dataset consisted of 400 AI-generated mermaid images covering diverse artistic styles, poses, age-related appearances, and gender-related appearances. Images were selected to ensure complete full-body representations, coherent human–fish transitions, and structurally intact fish tails, while close-up images and those with obvious morphological defects were excluded. The training images were automatically captioned and subsequently subjected to manual screening.

The training resolution was set to 864 × 1536 pixels, with a learning rate of 8 × 10⁻⁵, a cosine learning rate scheduler with warmup, LoRA rank and alpha both set to 8, and gradient accumulation set to 4. Training was conducted for 5000 optimizer-update steps. Each training image was repeated 10 times per epoch over five epochs, resulting in 20,000 sample presentations. A LoRA checkpoint was saved at the end of each epoch, producing five candidate models. These models were subsequently evaluated through visual assessment and objective metrics, with particular emphasis on structural completeness and feature fidelity. The best-performing LoRA model was selected for integration into the subsequent conditional image generation pipeline.

At the final presentation stage, the generated mermaid image is displayed through an integrated visualization interface. The primary visual output is a personalized mermaid character matched to the individual attributes of the user. In terms of colour, contour, texture, and other visual dimensions, the generated character preserves fundamental user-specific characteristics, including gender, age group, and clothing-related attributes, while simultaneously incorporating the visual prototype features of fish species associated with the Yangtze River Basin. This integration enables users to directly perceive the visual relationship between their own appearance characteristics and Yangtze River aquatic organisms, thereby enhancing visual resonance and interactive engagement.

6. Experimental Results and Analysis

6.1. Results and Analysis of Human Attribute Recognition

To evaluate the performance of the system in practical artistic interactive installation scenarios, 500 on-site interaction trials were conducted. These trials did not use a public image dataset or a pre-established collection of human photographs. More than 100 visitors participated in the interactive experience, with some participating multiple times. Individual identities were not registered, and each interaction was counted as a separate trial. The camera was typically positioned at a height of approximately 1.4 m and adjusted according to the exhibition environment. Lighting conditions and camera angles varied across deployment locations, with the primary requirement being that the participant’s face was clearly visible and the upper body was fully captured. Camera frames were temporarily processed for attribute recognition without storing or uploading the original facial images. Only textual recognition results and intermediate processing records were retained. For each trial, the system generated a structured four-tuple containing age group, gender, clothing colour, and clothing category. Four evaluation metrics were adopted: complete output rate, individual attribute recognition accuracy, all-field accuracy, and average recognition time.

Among the 500 interaction trials, the system successfully produced complete four-field outputs in 488 trials, corresponding to a complete output rate of 97.6%. This result indicates that, under most test conditions, the front-end recognition pipeline can reliably transform human images into structured attribute representations. In terms of individual attribute recognition, gender achieved the highest accuracy of 97.8%, followed by clothing colour at 94.6% and age-group recognition at 91.4%. Clothing category recognition achieved a comparatively lower accuracy of 89.8%. When all four attributes were required to be correctly recognized simultaneously, the all-field accuracy reached 76.2%. The average recognition time was 51.6 ms per sample. Under an NVIDIA RTX 4080 hardware environment, this processing speed indicates that the recognition module provides sufficient real-time responsiveness to satisfy the basic front-end processing requirements of an interactive installation.

For the mermaid-like virtual human generation task investigated in this study, these results further demonstrate that the recognition layer is sufficiently reliable to serve as the input stage for subsequent structured encoding and visual-semantic mapping, thereby providing an experimental basis for evaluating mapping performance and analysing the generated results in the following sections.

6.2. Results and Analysis of Visual-Semantic Mapping

To evaluate the executability and stability of the visual-semantic mapping module within the overall generation pipeline, the 488 samples for which the preceding human attribute recognition module successfully produced complete four-field outputs were used as inputs for experimental evaluation. The primary objective of this module is to reliably transform the standardized attribute record x, as defined in Equation (1), into visual-semantic fields and generation-description interfaces that can be interpreted by the diffusion model, thereby establishing a continuous and interpretable data flow across the front-end recognition layer, semantic mapping layer, and back-end generation layer.

Five evaluation metrics were defined: mapping completion rate, field completeness rate, colour-mapping rule matching rate, category-mapping rule matching rate, and average mapping time. The mapping completion rate measures whether the system can successfully generate a complete mapping result for each input sample. The field completeness rate evaluates whether all five generation fields, P_a, P_g, P_c, P_k, and P_f, are present without missing values. The colour-mapping rule matching rate measures whether the recognized clothing colour is correctly translated into the dominant colour range and texture semantics of the corresponding fish prototype. The category-mapping rule matching rate evaluates whether the clothing category is correctly translated into body contour, tail morphology, and local structural semantics. Average mapping time is used to assess the execution efficiency of the module in real-time interactive scenarios.

As shown in Table 6, the visual-semantic mapping module achieved high completion and rule-matching performance. Its overall stability was higher than that of the front-end human attribute recognition module. This difference can be attributed to the fact that the mapping stage is essentially a rule-driven semantic transformation process rather than an open-ended perceptual inference task based on deep visual models. At this stage, the input attribute records have already been restricted to predefined fields and label sets. Therefore, the task of the mapping module is no longer to extract information from complex visual images, but rather to translate discrete labels according to explicitly defined cross-domain mapping rules. This transition from perceptual recognition to symbolic rule-based transformation provides inherent advantages in terms of completion rate, field integrity, and computational efficiency. The average mapping time was 6.8 ms per sample, indicating that the visual-semantic mapping module introduces relatively little processing overhead compared with the subsequent image generation stage. Overall, the proposed mapping module can reliably transform human attribute recognition results into structured outputs comprising character-style semantics, colour semantics, structural semantics, and generation-interface fields, while maintaining high mapping completion and rule-matching rates. These results demonstrate that the mapping module can efficiently provide structured semantic conditions for the subsequent conditional image generation process within the proposed staged interaction framework.

Table 6. Overall Performance of the Visual-Semantic Mapping Module.

Metric	Value
Number of input samples	488
Successfully mapped samples	486
Mapping completion rate	99.6%
Field completeness rate	98.8% (482/488)
Colour-mapping rule matching rate	98.4% (480/488)
Category-mapping rule matching rate	97.1% (474/488)
Average mapping time	6.8 ms

6.3. Results and Analysis of Conditional Generation

To evaluate whether the proposed pipeline of human attribute recognition, visual-semantic mapping, and conditional image generation can reliably produce mermaid-like images that meet the expected requirements, the 486 structured description records successfully generated by the visual-semantic mapping module in the previous stage were used as inputs for evaluating the conditional generation module. According to the system design described above, the conditional generation stage directly receives the structured description D defined in Equation (6). In this sequence, age group and gender constrain the anthropomorphic characteristics of the generated character, clothing colour and clothing category preserve the clothing-related visual information of the input individual, and the feature fields derived from visual-semantic mapping provide additional descriptions of the dominant colour range, texture, contour, tail morphology, and other visual details.

Considering the practical objectives of the artistic interactive installation, generation success was evaluated through manual inspection based on two predefined criteria: structural completeness and attribute preservation. Structural completeness required a recognizable full-body mermaid-like character with a coherent human upper body and fish tail, without obvious anatomical defects such as extra limbs, duplicated legs, or missing tails. Attribute preservation required the generated character to remain consistent with the recognized gender, approximate age group, clothing colour, and clothing-related visual semantics specified in the generation description. An output was classified as successful only when both criteria were satisfied. This evaluation primarily assessed whether the generated images met the functional and visual requirements of the installation, rather than their aesthetic superiority. Both criteria were evaluated through manual visual inspection using a binary pass/fail judgment. The corresponding rates were calculated as the proportion of generated images satisfying each criterion relative to the total number of evaluated images.

Among the 486 input descriptions, 460 samples produced relatively complete mermaid-like structures, corresponding to a structural completeness rate of 94.7%. A total of 452 samples maintained satisfactory consistency in age group, gender, and clothing-related visual semantics, resulting in an attribute preservation rate of 93.0%. When both structural completeness and attribute preservation were required simultaneously, 438 of the 486 generated images satisfied both criteria, corresponding to a conditional generation success rate of 90.1%. Considering the complete processing pipeline, 438 of the original 500 on-site interaction trials resulted in generated images satisfying the two evaluation criteria, yielding an operational end-to-end success rate of 87.6%. The difference reflects 12 trials without complete attribute recognition outputs, two unsuccessful semantic mappings, and 48 generated images that failed to meet the combined evaluation criteria. The end-to-end operational success rate therefore describes the proportion of initial interactions producing acceptable final outputs, rather than independently verified correctness of all original human attributes throughout the pipeline. The average generation time was 36.8 s per image under the current local deployment configuration. Considering the hardware constraints and the requirement for fully local operation in on-site exhibition environments, the generation workflow was configured to balance processing efficiency with image quality and structural consistency. The proposed installation prioritizes the generation of relatively stable and high-quality personalized character images rather than real-time image synthesis. Therefore, a certain waiting period is an expected part of the staged interaction process. With more powerful computing hardware, further optimization of inference strategies, and continued advances in generative AI technologies, the generation time may be further reduced in future implementations.

Further analysis of the unsuccessful generation results revealed three main failure categories. Among the 486 generated images, 48 failed to satisfy the combined criteria of structural completeness and attribute preservation. Specifically, 14 images failed only the structural completeness criterion, 22 failed only the attribute preservation criterion, and 12 failed both criteria. These results indicate that the remaining limitations primarily concern mermaid structural integrity and the consistent preservation of recognized human attributes. Representative failure cases are illustrated in Figure 4.

Figure 4. Representative failure cases in mermaid-like virtual character generation: (a) structural incompleteness; (b) attribute inconsistency; and (c) combined structural and attribute failures.

The failure cases illustrated in Figure 4 indicate that unsuccessful generations may originate from different stages of the proposed pipeline. Attribute inconsistencies may be associated with errors in the initial recognition of age group, gender, clothing colour, or clothing category, which can subsequently affect semantic mapping and the resulting generation descriptions. In contrast, structural defects, such as additional limbs, incomplete fish tails, and unnatural transitions between the human upper body and fish tail, are more directly related to limitations of the image generation process in synthesizing complex human–fish composite anatomy. Although the dedicated Mermaid Character LoRA module was introduced to improve structural consistency, it cannot completely eliminate such generation defects. The retained textual recognition and mapping records provide a basis for distinguishing upstream attribute errors from inconsistencies arising during image generation. However, the current qualitative analysis does not independently establish the precise cause of every failure or isolate the contribution of the LoRA module from that of the base generative model. Future work will further improve structural stability and attribute preservation through optimization of the recognition, semantic mapping, and conditional generation processes.

6.4. Comparative and Ablation Experiments

To further investigate the contributions of visual-semantic mapping and the dedicated Mermaid Character LoRA module to the proposed generation framework, comparative and ablation experiments were conducted using four generation configurations based on FLUX.1. The experiments adopted a unified human attribute input, “teenager, female, blue, dress”, and two prompting strategies. The basic prompt was “young female mermaid, blue dress”, while the structured description generated through the proposed semantic mapping mechanism was “young female mermaid, blue dress, low-saturation grey-blue fish-inspired colouration, decorative fin patterns, extended slender silhouette, flowing streamlined tail”. The four experimental configurations were defined as follows: (A) FLUX.1 with the basic prompt; (B) FLUX.1 with the basic prompt and the dedicated Mermaid Character LoRA; (C) FLUX.1 with the semantic mapping description; and (D) FLUX.1 with the semantic mapping description and the dedicated Mermaid Character LoRA. Each configuration generated 30 images, resulting in a total of 120 images for comparative evaluation. These 120 images were generated under a common predefined attribute combination for configuration-level comparison. They do not represent 120 independent human participants, and the comparative results should be interpreted separately from the sequential evaluation of the 500 on-site interaction trials.

The generated images were manually evaluated using four indicators: structural completeness rate, attribute preservation rate, fish-related semantic consistency, and overall generation success rate. Structural completeness assessed whether the generated character exhibited a coherent human upper body and fish tail without obvious anatomical defects. Attribute preservation evaluated the consistency of the generated character with the specified age group, gender-related appearance, clothing colour, and clothing characteristics. Fish-related semantic consistency examined whether the generated images reflected the intended chromatic, decorative, and morphological features defined by the semantic mapping mechanism. This indicator was evaluated only for configurations C and D, as configurations A and B did not incorporate the corresponding fish-related semantic mapping descriptions. To ensure a consistent evaluation criterion across all four configurations, overall generation success was determined by the simultaneous satisfaction of structural completeness, attribute consistency, and basic visual acceptability, without obvious defects that would adversely affect the artistic presentation. The experimental results are summarized in Table 7. Representative visual results are shown in Figure 5.

Table 7. Comparative and ablation results under different generation configurations.

Generation Configuration	Structural Completeness Rate (%)	Attribute Preservation Rate (%)	Fish-Related Semantic Consistency (%)	Overall Generation Success Rate (%)
A. FLUX.1 + Basic Prompt	13.33% (4/30)	36.67% (11/30)	—	6.67% (2/30)
B. FLUX.1 + Basic Prompt + LoRA	90% (27/30)	73.33% (22/30)	—	66.67% (20/30)
C. FLUX.1 + Semantic Mapping	36.67% (11/30)	20% (6/30)	83.3% (25/30)	13.3% (4/30)
D. FLUX.1 + Semantic Mapping + LoRA	96.67% (29/30)	93.33% (28/30)	93.33% (28/30)	90% (27/30)

Note: Each configuration generated 30 images. The symbol “—” indicates that fish-related semantic consistency was not evaluated for configurations without the proposed semantic mapping mechanism.

Figure 5. Representative visual results of the four generation configurations under the same human attribute input: (a) FLUX.1 with the basic prompt; (b) FLUX.1 with the basic prompt and Mermaid Character LoRA; (c) FLUX.1 with the semantic mapping description; and (d) FLUX.1 with the semantic mapping description and Mermaid Character LoRA.

As shown in Table 7 and Figure 5, the basic-prompt configuration (A) exhibited limited performance in generating complete mermaid-like structures, with a structural completeness rate of 13.33% and an attribute preservation rate of 36.67%. After introducing the dedicated Mermaid Character LoRA, configuration B achieved a structural completeness rate of 90.00% and an attribute preservation rate of 73.33%, with an overall generation success rate of 66.67%. These results suggest that task-specific LoRA adaptation played an important role in improving the structural integrity and attribute consistency of the generated characters. Configuration C, which incorporated semantic mapping without LoRA, achieved a fish-related semantic consistency of 83.33%, but its structural completeness and attribute preservation rates remained relatively low, at 36.67% and 20.00%, respectively. This indicates that detailed semantic descriptions alone were insufficient to ensure stable generation of coherent human–fish composite structures.

The complete configuration (D) achieved the highest reported performance across the evaluated indicators, with structural completeness of 96.67%, attribute preservation of 93.33%, fish-related semantic consistency of 93.33%, and overall generation success of 90.00%. Compared with configuration B, the complete framework achieved improvements of 6.67, 20.00, and 23.33 percentage points in structural completeness, attribute preservation, and overall generation success, respectively. Furthermore, fish-related semantic consistency increased from 83.33% in configuration C to 93.33% in configuration D, representing an improvement of 10.00 percentage points. These findings suggest that the visual-semantic mapping mechanism and dedicated LoRA adaptation may serve complementary functions: the former provides structured visual guidance based on human attributes and fish-related features, while the latter supports the generation of coherent mermaid-like structures. However, the experiment involved only one human attribute combination and a limited number of generated images. Therefore, the results should be interpreted as preliminary evidence of the effectiveness of the combined configuration under the investigated conditions, rather than proof of its general superiority across diverse inputs or generation models.

7. Discussion and Conclusions

7.1. Discussion

The experimental findings provide further insight into the respective roles of visual-semantic mapping and the dedicated Mermaid Character LoRA module in conditional virtual character generation. As shown in Section 6.4, the complete configuration combining semantic mapping and LoRA achieved an overall generation success rate of 90.00%, compared with 66.67% for the configuration using basic prompting and LoRA alone. This difference suggests that the semantic mapping mechanism provides additional guidance for preserving human attributes and incorporating fish-related visual characteristics, while the dedicated LoRA supports the generation of coherent human–fish composite structures. However, these findings were obtained using a single representative attribute combination and a limited number of generated images, and therefore should not be interpreted as evidence of general superiority.

Compared with existing controllable generation approaches, the proposed framework addresses a different stage of the generation process. ControlNet proposed by Zhang et al. [21] and T2I-Adapter developed by Mou et al. [22] primarily focus on incorporating external spatial or visual conditions into pretrained generative models. In contrast, the present study emphasizes the preceding semantic representation stage, where recognized human attributes are translated into structured, task-specific visual descriptors. The methodological significance of this approach lies in establishing interpretable correspondences between human appearance attributes and fish-related visual characteristics, rather than modifying the internal architecture of the generative model. Accordingly, the proposed semantic mapping mechanism can be regarded as complementary to existing model-level conditioning methods, although its comparative performance against these approaches remains to be directly evaluated.

The failure analysis in Section 6.3 further reveals several limitations of the integrated generation process. Among the 48 unsuccessful images, 14 exhibited structural defects only, 22 failed to preserve the specified attributes, and 12 failed both criteria. These failures may originate from different stages of the pipeline. In the recognition stage, ambiguous clothing categories, illumination variations, pose changes, and partial occlusion may introduce inaccurate attribute labels. Such errors can subsequently affect semantic mapping and generation conditions. At the generation stage, the synthesis of complex human–fish composite structures remains challenging, particularly regarding the anatomical transition between the human upper body and fish tail, local proportional coordination, and the preservation of fine-grained visual details. Nevertheless, the present evaluation does not independently determine the precise cause of each failure.

From a methodological perspective, the experimental results suggest that structured visual-semantic translation can provide a practical intermediate mechanism for connecting human attribute recognition with domain-specific image generation. This is particularly relevant to digital art applications, where the objective is not only to produce visually complete characters but also to establish interpretable relationships between individual appearance and thematic visual elements. However, the current framework remains dependent on manually defined mapping rules and a visual prototype library comprising six representative fish species. Its applicability to other visual themes has not yet been experimentally established. In addition, the use of qualitative visual assessment, the limited conditions examined in the comparative experiments, and the possible non-independence of repeated on-site interaction trials constrain the interpretation of the reported results. The average image generation time of 36.8 s also presents a practical limitation for installations with high visitor throughput. Although the framework was developed for cultural tourism and science communication scenarios, its effects on visitor engagement and educational outcomes require further empirical validation.

7.2. Conclusions and Future Work

This study developed a controllable virtual human generation framework integrating human attribute recognition, structured encoding, visual-semantic mapping, and conditional image generation. Its principal methodological contribution lies in the interpretable translation of recognized human attributes into structured generation conditions that incorporate fish-inspired visual characteristics. The experimental results demonstrate the feasibility of this approach within the investigated application scenario and provide a task-oriented reference for personalized virtual character generation. Nevertheless, the current findings remain limited to the selected visual prototypes and experimental conditions, and broader applicability requires further validation.

Future work can be pursued in four main directions. First, the robustness of human attribute recognition under complex real-world conditions should be further improved, particularly for clothing category recognition, colour-boundary estimation, and stable prediction under multiple sources of visual interference, thereby reducing error propagation throughout the overall pipeline. Second, more adaptive semantic organization mechanisms can be introduced on top of the current rule-based mapping framework, together with an expanded library of fish visual prototypes and feature mappings, to improve transferability across different human attributes and biological prototypes. Third, the current framework, which primarily focuses on static image generation, can be further optimized and extended toward real-time generation, dynamic feedback, multimodal interaction, and mixed-reality (MR) presentation. Fourth, as the current experiments primarily evaluated system feasibility and image generation performance, visitor engagement, user satisfaction, and science communication outcomes have not yet been systematically assessed. Future on-site user studies will further evaluate these aspects in practical exhibition settings.