Expressive Dizi Synthesis: Unlimited Synthetic Datasets, Real–Synthetic Integration and Technique Labeling

Abstract

Digital modeling of traditional Chinese musical instruments significantly lags behind that of their Western counterparts, limiting advances in cultural preservation and related research. This paper addresses score-to-audio generation for the Chinese bamboo flute (dizi), aiming to synthesize expressive audio with human-like performance nuances directly from Musical Instrument Digital Interface (MIDI) scores. Two core challenges remain in existing score-to-audio synthesis methods: first, the scarcity of large-scale paired MIDI-audio training data for traditional Chinese instruments; second, the inability of standard MIDI to encode instrument-specific expressive techniques, such as vibrato, pitch bends, and ornamentations. To address these challenges, we propose a scalable workflow that generates large-scale synthetic MIDI-audio pairs through rule-based score randomization, automated digital audio workstation (DAW) rendering with commercial sample libraries, and standardized feature extraction. Performance technique information is directly encoded into the MIDI stream using out-of-range MIDI note numbers, achieving more effective conditioning than external control methods. Our system is built upon the Musical Instrument Digital Interface–Differentiable Digital Signal Processing (MIDI-DDSP) framework. Large-scale synthetic data is used for pre-training to establish timbral consistency, while real recordings from the University of Rochester Multi-Modal Music Performance (URMP) flute corpus are employed for fine adjustment to achieve expressive dynamic variations. Synthetic data alone yields stable but less expressive outputs, whereas real data alone risks overfitting. The combined strategy achieves a realistic timbre with controllable dynamics and performance techniques.

1. Introduction

Digital modeling of expressive musical instrument performance spans music information retrieval, computational musicology, and neural audio generation [1]. For Western instruments such as piano and orchestral winds, data-driven synthesis has matured substantially, supported by aligned datasets such as the University of Rochester Multi-Modal Music Performance (URMP) dataset [2] and the MIDI and Audio Edited for Synchronous TRacks and Organization (MAESTRO) dataset [3]. For traditional Chinese instruments, by contrast, paired symbolic–audio data and dedicated synthesis models remain scarce, hindering digital heritage preservation, systematic research, and flexible use in media production.

The Chinese bamboo flute (dizi) poses unique acoustical and representational challenges that cannot be resolved by simple timbre adaptation from the Western concert flute. Its dimo (bamboo membrane), membrane hole, tone-hole geometry, and bore structure shape the instrument’s resonance, tuning, and harmonic profile [4]; the nonlinear vibration of the dimo in particular generates the instrument’s characteristic high-frequency harmonic energy [5]. Combined with idiomatic performance techniques including tonguing, trills, mordents, appoggiaturas, and continuous pitch bends, these properties make expressive dizi synthesis a distinctly difficult task. Standard Musical Instrument Digital Interface (MIDI) workflows further complicate representation: while MIDI 1.0 supports pitch bend and control change messages [6], it defines no portable, standardized mapping for dizi-specific techniques, leaving articulation mappings vendor-dependent across sample libraries. The newer MIDI 2.0 orchestral articulation profile [7] improves general consistency but provides no dizi-specific taxonomy, paired training data, or native representation for data-driven synthesis.

Early end-to-end score-to-audio systems connected symbolic performance modeling directly to waveform generation. PerformanceNet mapped piano-roll inputs to spectrograms for multiple instruments [8], and Deep Performer combined expressive timing prediction, spectrogram synthesis, and waveform reconstruction for violin and piano [9]. Differentiable Digital Signal Processing (DDSP) later introduced interpretable signal-processing modules into neural networks [10], and MIDI-DDSP built on this framework with a hierarchical note–performance–synthesis architecture enabling explicit control over dynamics, vibrato, and articulation [11]. All these methods, however, were designed and validated primarily for Western instruments and rely heavily on high-quality aligned symbolic–audio recordings.

More recent work has explored diffusion models, instrument-specific representations, and neural audio codecs for score-to-audio synthesis. Multi-instrument spectrogram diffusion generated diverse instrumental timbres from MIDI input [12]; instrument-specific designs improved articulation modeling for acoustic guitar [13] and pitch-bend control for violin [14]. For piano, integrated architectures jointly modeled expression and audio generation [15], and MIDI-VALLE explored neural-codec language modeling for expressive synthesis [16]. CoSaRef further investigated annotation-free MIDI-to-audio synthesis via concatenative synthesis with generative refinement [17]. While these approaches demonstrate steady progress, existing models remain constrained when applied to non-Western instruments due to three unaddressed barriers: the absence of aligned large-scale symbolic-audio corpora, the lack of standardized representations for instrument-specific playing techniques, and the lack of dedicated cross-instrument transfer frameworks tailored to traditional aerophones.

Two core research gaps remain. First, no large-scale synchronized MIDI–audio dataset exists for dizi synthesis. Public datasets cover Western instruments [2,3], and symbolic corpora such as GigaMIDI include expressive performance indicators but lack aligned dizi audio or technique annotations [18]; building such a corpus manually requires extensive performer time and specialist expertise. Second, no standardized, model-ready representation exists for dizi-specific performance techniques. Pitch bends can be encoded via standard MIDI messages, but complex ornaments and articulation categories rely on ad hoc, vendor-specific mappings, leading to inconsistent conditioning signals across systems.

To address these gaps, we build an expressive dizi score-to-audio synthesis system on the MIDI-DDSP backbone. Among available frameworks, MIDI-DDSP’s hierarchical architecture preserves explicit symbolic control over dynamics and articulation, which aligns with our goal of building an interpretable, technique-labeled pipeline and implementing cross-instrument expressive transfer. A preliminary controlled comparison with the multi-instrument spectrogram diffusion (MSD) baseline on a URMP flute excerpt confirms that MIDI-DDSP achieves substantially higher note-level alignment and more reliable symbolic control under identical conditions (

Table 1. Backbone comparison on the controlled URMP flute benchmark.

Model	Note F1	Note F1 + Offset	Onset MAE (ms)	Offset MAE (ms)	Frame F0 MAE (cents)	Voiced Coverage
MIDI-DDSP	0.87	0.83	7.62	35.37	20	0.97
Music-Spectrogram-Diffusion	0.25	0.16	17.45	59.41	209	0.74

). We focus our contributions on three complementary components: scalable synthetic data construction, dizi-specific performance technique labeling, and real–synthetic transfer learning.

The main contributions of this work are summarized as follows:

• Scalable synthetic data generation pipeline. Rule-based score generation and automated batch rendering in the REAPER digital audio workstation (DAW) produce large-scale, time-aligned MIDI–audio pairs from a commercial dizi sample library, reducing reliance on manually recorded paired data.
• Out-of-range performance technique labeling scheme. MIDI note numbers outside the dizi’s natural playable register are assigned to dizi-specific techniques (trills, mordents, appoggiaturas), embedding technique labels directly into the note sequence without a separate auxiliary input stream.
• Two-stage synthetic–real transfer learning strategy. Timbre synthesis components are first pre-trained on synthetic dizi data to establish stable timbral representations; the expression module is then adapted using real URMP flute recordings to transfer general aerophone dynamics and performance variation under limited data conditions, preserving native dizi timbre while improving expressive realism.

To clarify the independent role of each component, we define four progressive experimental configurations, summarized in

Table 2. Functional configurations of the proposed synthesis framework.

Configuration	Synthetic Data Pretraining	Technique-Aware Conditioning	Cross-Instrument Expression Transfer	Primary Function
MIDI-DDSP (Baseline)	−	−	−	Score-conditioned synthesis
+Synthetic Data Pretraining	√	−	−	Scalable timbre modeling
+Technique-Aware Conditioning	√	√	−	Technique-aware control
Full Framework	√	√	√	Expressive synthesis with real-data transfer

Note: Bold-font text represents the complete and full set of the proposed synthesis framework.

, corresponding to the incremental integration of the three proposed modules.

The remainder of this paper is structured as follows. Section 2 describes the synthetic data generation workflow and out-of-range technique labeling scheme. Section 3 presents the cross-instrument transfer rationale and two-stage training strategy. Section 4 reports experimental setup and results, including both objective metrics and subjective listening tests. Section 5 discusses findings and limitations, and Section 6 concludes the paper.

2. Unlimited Synthetic Data Workflow with Performance Technique Labeling

To overcome dataset scarcity for traditional Chinese instruments, we design a scalable workflow generating paired score–audio data. The pipeline consists of four sequential stages: (i) rule-based symbolic score generation, (ii) batch audio rendering, (iii) MIDI–audio feature extraction, and (iv) extended performance technique encoding. This pipeline combines reproducibility, extensibility, and expressive control for deep neural synthesis.

As visualized in Figure 1, configurable inputs for score generation include pitch range, key, scale and beats per minute (BPM). Rule-based score generation outputs MIDI files, which are batch-rendered to WAV audio within the Reaper digital audio workstation (DAW). Subsequent feature extraction computes fundamental frequency (F0) via the CREPE [19] algorithm and A-weighted loudness, producing note-aligned features fed into the MIDI-DDSP training framework.

2.1. Rule-Based Score Generation

The first stage constructs symbolic musical sequences emulating idiomatic dizi patterns. We use a rule-based algorithm generating unlimited symbolic sequences with configurable parameters: key, mode, pitch range, tempo. This enables scalable generation while maintaining musically meaningful constraints.

Rhythmic structures follow a fixed metric grid for temporal regularity and rendering stability. Note durations sample from a weighted distribution of common rhythmic values, promoting stylistic plausibility without excessive complexity. Pitch transitions stay within one octave, reflecting typical dizi practice and maintaining physical playability. Velocity values sample from a Gaussian distribution, providing controlled loudness fluctuations.

All generated sequences obey strict numerical constraints: pitch intervals favor stepwise motion without exceeding a major sixth; durations sample from a weighted distribution biased toward quarter and eighth notes; and MIDI velocities sample from a Gaussian distribution (μ = 85, σ = 12). Phrases are constrained to 4–16 notes to form self-contained melodic units.

Operating symbolically enables high reusability and flexibility. Generation rules adapt easily to alternative scales, rhythmic profiles, or tempo distributions, facilitating systematic experimentation. Symbolic generation decouples musical structure from acoustic realization, allowing large-scale controllable dataset construction.

2.2. Batch Rendering of Audio

Generated symbolic scores render into audio waveforms using Reaper v7.22 DAW with custom ReaScript automation. This ensures consistent, reproducible MIDI-to-audio conversion. Flexible virtual instrument or sample library substitution enables systematic investigation of data quality and timbral variation effects on model performance.

Training corpus fidelity matters. High-quality rendered audio preserves timbral characteristics and expressive detail. Low-quality or inconsistent sources introduce artifacts that hinder learning. The pipeline enforces controlled rendering: 48 kHz sampling, 24-bit resolution, monophonic output.

Batch rendering is implemented via custom ReaScript Lua scripts, which automatically traverse the target MIDI directory, initialize the designated virtual instrument, preheat the plugin engine, render each clip with unified project settings and export WAV files to a categorized output folder with matched filenames. The dizi sample library is used as the rendering sound source, which contains multi-layer professionally recorded dizi samples covering the full register and common articulations.

Automated rendering imports MIDI files in batch, assigns them to a predefined instrument track, initializes the virtual instrument plugin, and exports audio under identical settings. This enables large-scale homogeneous dataset generation while minimizing variability. Rendering identical scores with different virtual instruments allows controlled analysis of timbral fidelity and dataset composition effects.

2.3. Feature Extraction from MIDI-Audio Pairs

Each MIDI–audio pair processes into frame-level features for training performance-aware synthesis models like MIDI-DDSP. Audio signals downsample to 16 kHz mono, reducing computational cost while preserving perceptually relevant information. Signals segment into fixed-length frames with anti-alias filtering for spectral stability.

Convolutional Representation for Pitch Estimation (CREPE) estimates fundamental frequency (f₀) trajectories, capturing melodic and expressive pitch variations. Perceptual loudness and signal power (in decibels) represent dynamic characteristics. These features encode core acoustic dimensions: pitch, dynamics, temporal evolution.

MIDI events align to the same frame grid, yielding onset/offset indicators, active note indices, and time-varying velocity curves. Metadata (instrument identity, file identifiers) attaches to each instance. All features serialize into TFRecord, TensorFlow’s standard binary serialization format, enabling compact storage and efficient training loading. This representation captures melodic contour, dynamic shaping, and structural timing while maintaining compatibility with scalable synthesis pipelines.

2.4. Out-of-Range MIDI Performance Technique Labeling

Dizi expressivity relies on ornamentations (tonguing, vibrato) not captured by standard MIDI. We use out-of-range MIDI mapping: note numbers outside the natural range (G4–D7) represent extended techniques (

Table 3. Representative mapping of out-of-range MIDI note numbers to extended techniques for the dizi.

MIDI Note	Technique	Description
D1	Tonguing	attack variants
E1	Flutter onset	tremulous tongue start
F#1	Upper passing	step ornaments
A2	Fast trill (2nd)	rapid alternation

Note: The sharp symbol (#) stands for the raised semitone of the corresponding MIDI note.

).

As visualized in Figure 2, the upper staff displays melodic notes within dizi’s native pitch range (G4–D7). The lower staff demonstrates out-of-range MIDI events assigned to typical dizi ornaments, including ascending rapid scalar passages, appoggiatura, mordent, and flick ornaments. This mapping embeds idiomatic performance controls inside standard MIDI sequences; it delivers explicit symbolic supervision for score-to-audio models and retains compatibility with mainstream DAW environments.

This integrates with DAWs/plugins (remapping assigned notes to articulatory controls). It creates a digital notation paradigm encoding idiomatic techniques alongside pitch/rhythm—enabling automatic generation and human editing. It provides explicit supervision for training score-to-audio models, supporting fine-grained ornamentation control.

Extending symbolic representations to include performance-level control balances interpretability and modeling flexibility. Explicit technique encoding lets the synthesis model associate symbolic events with measurable acoustic phenomena: transient energy distribution, spectral noise components, pitch deviation patterns. This design fits hybrid neural-DSP architectures, where symbolic controls map to physically meaningful synthesis parameters.

2.5. Summary of Workflow

Four stages establish a pipeline from symbolic abstraction to expressive audio rendering. The rule-based generator supplies unlimited symbolic material. Batch rendering ensures timbral fidelity. Feature extraction aligns symbolic and acoustic representations. Extended MIDI mapping encodes ornamentation beyond standard notation. This workflow provides scalable training data and a framework for integrating traditional instrumental techniques into neural audio synthesis.

Large-scale synthetic data play a dual role: they provide controllable, noise-free supervision while enabling systematic coverage of pitch ranges, dynamic levels, and articulation patterns difficult to obtain from real performances. Synthetic audio may lack micro-level expressive irregularities, but its consistency and reproducibility suit learning stable timbral representations and fundamental acoustic mappings.

3. Cross-Instrument Transfer with Real and Synthetic Data Integration

3.1. Complementary Roles

Synthetic data provides stable, controllable timbral priors. Real recordings recover subtle dynamics and natural phenomena (breathing noises). Our scalable score-to-audio workflow generates large-scale high-fidelity MIDI–WAV pairs, ensuring adequate coverage of pitch, rhythm, and scores. However, rendered sources lack breathing noise and performance details (phrasing, airflow artifacts), capping expressivity for models trained solely on them. Real recordings from the URMP dataset [8], though limited in quantity, contain authentic breathing and dynamic patterns—providing critical expressive priors.

We design a staged fusion strategy: first, pre-train the timbre modeling (synthesis) module with synthetic data to obtain stable pitch-to-spectrum mappings and harmonic structures; then, fine-tune the expression generation module primarily with real data, injecting authentic flute performance dynamics and noise into the learned timbral representation. During training, we normalize synthetic audio loudness to match the real flute corpus, reducing domain mismatch and facilitating transfer. This hybrid strategy preserves synthetic data’s broad coverage and controllability while inheriting real recordings’ irreplaceable performance details, balancing timbre fidelity and expressive naturalness.

Subsequent sections quantitatively and perceptually evaluate this scheme, analyzing its transfer performance and limitations under cross-instrument conditions.

3.2. Cross-Instrumental Analysis: URMP-Flute and Chinese QuDi

The choice to utilize Western flute recordings as a supplementary data source for dizi synthesis stems from both pragmatic constraints and shared organological principles. Large-scale acoustic datasets for the qudi are currently unavailable in the public domain. This data scarcity severely limits the direct optimization of neural acoustic models. The Western concert flute presents a viable surrogate for capturing respiratory dynamics and onset transient behaviors. Both instruments are side-blown aerophones relying on an air jet striking an embouchure hole to excite the resonant air column. This shared excitation mechanism results in comparable macroscopic dynamic contours and baseline harmonic envelopes.

However, recognizing the physical and acoustic divergences between the two instruments is necessary for understanding the limitations of this transfer approach. The most prominent structural distinction is the dimo of the dizi. This thin bamboo membrane is placed over a dedicated hole in the instrument body. As the internal acoustic pressure increases, the dimo exhibits highly nonlinear vibrations. These vibrations introduce pronounced buzzing artifacts in the high-frequency range and modulate the harmonic spectrum dynamically. The Western flute lacks this coupling mechanism and consequently produces a much smoother and more stable spectral profile.

Furthermore, the internal bore of the qudi is typically cylindrical but retains natural bamboo irregularities. The modern flute employs a highly optimized parabolic headjoint and a cylindrical body made of metal or dense wood. These differences in material and geometry influence the radiation characteristics and the cutoff frequency of the respective instruments. Performers also employ different articulation strategies. Western flute pedagogy often emphasizes clean tonguing and consistent vibrato. Dizi performance practice relies heavily on complex pitch bends, rapid grace notes, and variable air pressure techniques to convey regional stylistic flavors.

By fine-tuning the Expression Generator on flute data, the model primarily learns the macroscopic respiratory envelopes and breath-induced noise floors common to edge-blown aerophones. The synthesis module is pretrained on synthetic dizi data and is tasked with retaining the baseline timbral identity. The transferred expressive cues are approximations. They do not perfectly encapsulate the nonlinear dimo turbulence or the micro timing of idiomatic bamboo flute ornamentations. We treat the integration of flute data as a practical regularization strategy rather than a definitive solution for dizi modeling.

3.3. MIDI-DDSP Adaptation for Spectral Features

As validated in the preliminary backbone comparison in the Introduction (Table 1), MIDI-DDSP achieves better symbolic alignment and controllability than music spectrogram diffusion models. Therefore, we select this architecture as our backbone and adapt it for dizi-oriented synthesis tasks. MIDI-DDSP, originally designed for Western instruments such as those in the URMP dataset, faces challenges when applied to the qudi. The timbre of the qudi is shaped by bamboo resonance and membrane vibration, resulting in fluctuating harmonic structure and pronounced noise components such as breath and tonguing. These characteristics are not fully captured by the default harmonic and noise branches. In addition, performance-related variations introduce further complexity beyond standard pitch conditioning.

Two targeted modifications are applied to the original MIDI-DDSP architecture: first, the harmonic-noise balance branch is reweighted to enhance high-frequency noise components corresponding to dimo vibration; second, an HNR calculation module is embedded into the feature inference pipeline for real-time spectral balance evaluation. All input acoustic features (F0, A-weighted loudness) are standardized to 128-dimensional fixed-size vectors before being fed into the two generator branches.

To better characterize these properties, we focus on the harmonic-to-noise ratio (HNR), which quantifies the balance between harmonic and noise energy in the signal:

$$\mathrm{HNR}(t) = 10{\log }_{10}{\displaystyle \frac{P_{\mathrm{h}\mathrm{a}\mathrm{r}\mathrm{m}}\left(t\right)}{P_{\mathrm{n}\mathrm{o}\mathrm{i}\mathrm{s}\mathrm{e}}\left(t\right)}},$$

(1)

HNR provides a compact descriptor of noise-related behavior that is directly relevant to wind instrument synthesis. It is used in our analysis to examine whether the generated audio exhibits a realistic balance between harmonic structure and noise components, particularly in the presence of breath and articulation effects.

By incorporating this descriptor into the evaluation, we aim to better understand how the adapted MIDI-DDSP framework captures the spectral characteristics of the qudi.

3.4. AI Tool Usage Statement

During the preparation of this manuscript, generative AI (GPT-5) was used for language polishing and wording refinement to improve the readability and fluency of the academic text. All AI-generated content has been thoroughly reviewed, revised, and validated by the authors. All authors take full responsibility for the entire content of the manuscript.

4. Experiments and Results

The experiments evaluate the proposed framework from three core perspectives: technique-aware modeling, cross-instrument expression transfer, and synthetic–real transfer learning.

4.1. Performance Technique Modeling

We evaluate the proposed out-of-range MIDI mapping for technique-aware synthesis, embedding dizi-specific performance cues directly into MIDI sequences.

Figure 3 compares spectrograms of generated audio against reference recordings. The model accurately reproduces subtle spectral fluctuations from ornaments and rapid note transitions, verifying the mapping delivers usable symbolic control for expressive synthesis.

4.2. Cross-Instrument Expression Transfer

We explore whether URMP flute recordings can supply transferable expressive features for dizi synthesis. The Expression Generator undergoes fine adjustment after initialization on synthetic dizi data.

As shown in Figure 4, the transferred model exhibits smoother dynamic transitions and richer local temporal variation than the synthetic-only configuration, particularly in sustained notes and ornamented passages. These observations suggest that real flute recordings provide useful expressive cues that are difficult to obtain from rendered synthetic data alone.

In the right sub-plot of Figure 4, green boxes mark regions with enhanced temporal dynamics and ornament-related fine-scale fluctuations introduced by cross-instrument transfer, which are absent in the synthetic-only output on the left.

These visual differences reflect the physical divergences analyzed in Section 3.2. While the transferred model captures macro-level respiratory dynamics, high-frequency energy modulations induced by dimo vibration are not fully replicated, confirming that cross-instrument transfer acts as a pragmatic expressive prior rather than an exact acoustic substitution.

The transfer should nevertheless be interpreted as a pragmatic cross-instrument strategy rather than a direct substitution for dizi recordings. The shared excitation mechanism of side-blown aerophones provides a basis for transferring macroscopic expressive characteristics, whereas dimo-induced spectral modulation and idiomatic dizi ornamentation remain instrument-specific.

4.3. Transfer Learning with Synthetic and Real Data

To combine the scalability of synthetic data with the expressive variability of real recordings, we employed a two-stage training strategy. In the first stage, the model was pretrained on synthetic dizi MIDI–audio pairs to establish stable timbral representations. In the second stage, real URMP flute recordings were used to fine-tune the expressive components.

As illustrated in Figure 5, the upper sub-figure depicts the pre-training workflow with three distinct training steps using synthetic dizi data. The lower sub-figure illustrates the subsequent adaptation stage on real flute material, where only the Expression Generator receives parameter updates and all synthesis-related modules remain frozen.

The synthetic pretraining stage used 40,000 steps for the Synthesis Generator, 5000 steps for the Expression Generator, and 10,000 steps for the DDSP Inference Module. During the fine-adjustment stage, only the Expression Generator was updated (5000 steps), while the Synthesis Generator and DDSP Inference Module remained frozen. Training used Adam with a learning rate of 0.0003 and a batch size of 8 on an NVIDIA GeForce RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA). An exponential learning-rate decay factor of 0.99 per 1000 steps was applied throughout training.

This asymmetric training strategy separates timbral learning from expressive refinement: synthetic data provides broad and consistent coverage for timbral modeling, whereas real recordings supply performance-related variation. The resulting model therefore retains the synthetic timbral prior while incorporating expressive characteristics from real performance data.

4.4. Subjective Listening Evaluation

We conducted a listening test to assess perceptual quality, focusing on aspects that are not fully captured by objective metrics, such as naturalness and expressive plausibility.

A total of 50 participants took part in the evaluation, including 27 professional musicians from art troupes, 5 conservatory-trained specialists, and 18 informed music enthusiasts. All tests were completed in a quiet monitoring room (background noise < 30 dB) using Sennheiser HD600 headphones (Sennheiser electronic GmbH & Co. KG, Wedemark, Germany). Each clip lasted 8–15 s, and each session required approximately 12 min under a single-blind randomized protocol. Informed consent was obtained from all participants prior to testing, using the template provided in Supplementary File S1.

The evaluation consisted of three tasks. The first task focused on timbral realism, assessing how closely the synthesized audio resembled a real dizi performance. The second task examined expressive performance, including dynamics, articulation, and the perceived naturalness of musical phrasing. The third task evaluated the effect of combining synthetic and real data, by comparing different training configurations in terms of their expressive alignment with a reference performance.

These three components jointly aim to capture both timbral fidelity and higher-level expressive characteristics, providing a perceptual complement to the objective metrics reported in Section 4.5.

4.4.1. DAW vs. Model Rendering

Participants compared pairs of audio clips generated from identical MIDI sequences. One clip was rendered in Reaper using a professional commercial sample library built from recorded acoustic qudi performances. The other was generated using our synthesis model. Listeners were asked to select which clip sounded closer to a real qudi performance. Because the underlying MIDI was identical in both conditions, any observed preference reflects differences in synthesis quality, not musical content.

Of the 50 valid responses, 34 participants preferred the model output (68.0%), and 16 preferred the DAW rendering (32.0%). We performed McNemar’s test to evaluate whether this difference was statistically reliable. The test showed no significant difference between the two conditions (χ² = 0.08, p = 0.781). The 95% confidence interval for the model preference rate was [54.0%, 79.5%].

These results show that listeners could not reliably distinguish between our model and the professional DAW sample library. The proposed model therefore achieves perceptual realism comparable to the reference DAW rendering rather than demonstrating a statistically significant preference over it.

4.4.2. Model Variant Ratings

The participants rated two model variants (synthetic-only vs. jointly trained) on a five-point scale for the following criteria:

• timbral realism;
• expressive naturalness.

The jointly trained model received higher average ratings on both dimensions, with a more noticeable improvement in expressive naturalness (Figure 6). This suggests that real-data fine adjustment contributes primarily to dynamic shaping and phrasing rather than coarse timbral characteristics.

4.4.3. Expressive Alignment with Flute Reference

To evaluate expressive consistency, participants were given a real flute recording as a reference. They then compared two synthesized dizi outputs and selected the one whose expressive dynamics more closely matched the reference.

Of the 50 responses, 88% favored the jointly trained model and 12% favored the synthetic-only model (Figure 7). A chi-square goodness-of-fit test showed that the distribution differs significantly from chance (χ² = 28.88, p < 0.001).

This result suggests that the observed preference is unlikely to be due to random variation. Within this experimental setting, the jointly trained model is more often perceived as better matching the expressive contour of the reference performance. This supports the effectiveness of incorporating real performance data for improving expressive modeling.

4.5. Objective Evaluation Metrics

To complement the subjective evaluation, we computed objective metrics on held-out samples, including mel-cepstral distortion (MCD) for spectral similarity, Fréchet Audio Distance (FAD) for distribution-level similarity, and harmonic-to-noise ratio (HNR) for noise-related characteristics. FAD was computed using embeddings extracted from VGGish.

Table 4. Objective evaluation results.

Model	Training Data	MCD (dB)	FAD	HNR (dB)
MIDI-DDSP (Baseline)	flute only	5.12 ± 0.30	3.21	18.5 ± 2.1
Ours	dizi only	4.75 ± 0.25	2.85	22.3 ± 1.8
	dizi + flute (joint)	4.21 ± 0.30	2.34	19.6 ± 2.0

summarizes the objective evaluation results. The proposed jointly trained model achieves lower MCD and FAD than both the baseline and the synthetic-only model, indicating improved spectral fidelity and closer alignment with the distribution of real recordings. In addition, its HNR values are closer to those observed in real audio, suggesting a more realistic balance between harmonic components and noise-related features such as breath and turbulence. The synthetic-only model exhibits higher HNR values, indicating reduced noise components. This is consistent with the tendency of synthetic training to produce over-smoothed outputs lacking realistic breath noise.

These trends are consistent with the subjective listening results. However, as objective metrics cannot fully capture expressive quality, they are interpreted here as complementary evidence rather than a substitute for perceptual evaluation.

5. Discussion and Limitations

This study presents an integrated pipeline designed for expressive dizi synthesis, specifically addressing the scarce availability of paired audio-symbolic data. Subjective listening evaluations and objective acoustic analyses demonstrate that combining synthetic datasets with cross-instrument real recordings achieves greater timbral stability and enhanced expressive dynamics compared to single-source training setups.

The framework integrates three primary components: an automated, scalable workflow for synthetic MIDI-audio generation; a standardized labeling mechanism utilizing out-of-range MIDI signals for dizi-specific performance techniques; and a two-stage transfer learning strategy that combines synthetic timbre pre-training with real audio fine-tuning. Together, these elements mitigate data scarcity while providing a standardized symbolic representation for native playing techniques in traditional Chinese wind instrument synthesis.

From a practical perspective, this research offers a data-efficient framework adaptable to other under-resourced traditional wind instruments. Integrating technique-specific symbolic annotations with cross-aerophone transfer learning demonstrates how cross-instrument acoustics can supplement limited native performance data.

Several methodological constraints remain. Relying on Western flute recordings for expressive fine-tuning captures shared aerophone dynamics, yet it does not fully replicate the dimo-driven harmonic structure characteristic of the dizi. Additionally, while the evaluation panel of 50 participants yielded statistically reliable trends under controlled listening conditions, its scale remains limited regarding broad demographic generalization. Future work will expand both the participant base and objective metric coverage. Future research will incorporate dedicated dizi recordings alongside advanced generative architectures to further refine synthesis quality.

6. Conclusions

This study addresses the digital modeling of Chinese traditional instruments, focusing on the dizi as a representative case. We propose a scalable data-generation workflow capable of producing unlimited synthetic MIDI-audio pairs, overcoming inherent data scarcity that limits learning-based synthesis methods for non-Western instruments. The synthetic pipeline enables systematic coverage of pitch ranges, articulations, and performance variations.

To capture the expressive richness of dizi performance practice, we introduce a technique-labeling scheme encoding ornamentations and performance techniques using MIDI note numbers outside the conventional playable range. This strategy offers a compact yet expressive symbolic representation bridging abstract musical scores and instrument-specific performance gestures.

Building upon the MIDI-DDSP framework, our approach integrates synthetic and real datasets. Synthetic data ensures timbral consistency, controllability, and broad coverage. Real performance recordings (URMP flute corpus) inject authentic expressive dynamics, timing irregularities, and nuanced articulation patterns. This hybrid training strategy enables effective transfer of expressive characteristics while maintaining stable timbral modeling.

Subjective listening evaluations confirm that the system produces audio more closely resembling real instrumental performance than conventional DAW-based rendering. The system benefits from cross-instrument training using Western flute data, suggesting shared acoustic principles among wind instruments can be exploited to improve synthesis quality for underrepresented instruments.

The framework extends beyond the dizi. Future work may incorporate larger-scale real recordings, more expressive symbolic representations, and emerging generative paradigms such as diffusion-based models or neural codec architectures. By combining controllable synthetic data generation, explicit encoding of performance techniques, and acoustically motivated model design, this work provides a practical methodology for expressive sound synthesis under data-limited conditions.