Enhancing Typhlo Music Therapy with Personalized Action Rules: A Data-Driven Approach
Abstract
1. Introduction
2. Background
2.1. Music Therapy for Visually Impaired Individuals
2.2. Action Rules in Therapy
2.3. Data-Driven Approaches in Therapeutic Settings
3. Materials and Methods
3.1. Collaboration with the Domain Expert
3.2. Data Overview
3.2.1. Stable Classification Attributes
3.2.2. Flexible Classification Attributes
3.2.3. Decision and Meta-Decision Attributes
- Physiological reactions: sweating, breathing, and reddening.
- Motorics: changing body position, tightening of muscles, movement of legs, movement of hands, and performing co-movements.
- Concentration of attention: expressing the desire to listen to music for longer, suggesting a desire to shorten the time to listen to music, and attention to acoustic phenomena occurring outside the music being listened to.
- Experience of music: humming the melody of the presented music, performing the rhythm of presented music, rocking for presented music, and responding to changes in music.
- Communication: communicating with words, communicating by gesticulation, and communicating with a mimic.
- Blindism: rocking, head shaking, hands waving (not in front of eyes), hands waving in front of eyes, and eye rubbing.
- Expression: the expression or manifestation of emotions and feelings, using gestures, using mimicry, verbalizing (expressing something with words), and vocalizing (expressing something, not necessarily with words).
3.3. Music Feature Extraction
3.3.1. Data Retrieval and Basic Information Extraction
3.3.2. Audio Feature Extraction
- Measure onset strength,
- Estimate tempo from onset autocorrelation,
- Select peaks in onset strength consistent with the estimated tempo.
- is the frequency (in Hz) of bin k—i.e., what pitch this bin represents.
- is the spectral centroid at time t, representing the center of spectral energy.
- measures how far each bin is from the center, with squaring used to ensure positive values and emphasize larger deviations.
- is the normalized spectral magnitude at bin k and time t, indicating how much energy is present relative to the total.
3.3.3. Extracting Musical Keys from Chroma Features
- Method: The chroma vector is converted from a string representation to a list of floats. The pitch class with the highest value is then identified using the np.argmax function.
- Limitations: While this method provides insight into the most prominent note in the music, it does not necessarily reflect the overall key of the piece, as it does not consider the harmonic context.
- Method:
- –
- Major and Minor Scale Templates: We defined templates for all 23 major and minor scales. Each template is a chroma vector with 1s indicating the presence of notes in the scale and 0s otherwise.
- –
- Correlation Calculation: For each chroma vector, we calculate the correlation with each major and minor scale template using the np.correlate function. The scale with the highest correlation is considered the estimated key. Since there are relative scales, such as the C major scale and A minor scale being equivalent, we record multiple scales with the highest correlation if they are equal.
- Benefits: This method provides a more accurate estimation of the musical key by taking into account the overall distribution of pitch classes in the chroma vector.
Examples
- Title: Für Elise
- Estimated Scale: C major or A minor
- Most Frequent Pitch: E
- Title: Moonlight Sonata
- Estimated Scale: E Major or C# Minor
- Most Frequent Pitch: G#
3.4. Data Cleaning and Preparation
3.4.1. Handling Missing Values
3.4.2. Discretization of Continuous Audio Features
3.4.3. Generate Classification Rules
3.5. Action-Rule Generation
- Calculate Attribute Correlations: We begin by calculating the correlations between the flexible classification attributes in the dataset. Pearson’s correlation coefficient is used for continuous attributes, Cramer’s V for categorical attributes, and the Correlation Ratio for mixed attribute types. These associations are calculated using the Dython Python library.
- Perform Agglomerative Clustering: Using the calculated correlations, we create a distance matrix, defined as:We then perform agglomerative clustering with single linkage to generate a dendrogram, which helps us identify clusters of flexible attributes. An example dendrogram is provided in Figure 2.
- Generate Action Rules for Clusters: For each cluster of attributes derived from the dendrogram, we generate action rules. This process can be executed in parallel for efficiency, utilizing the Python Ray library. First, classification rules are generated using the Rough Sets Exploration System (RSES) tool. These classification rules identify patterns and relationships between attributes and decision outcomes. Our custom program then uses these classification rules as input to generate action rules, which suggest changes in flexible attributes that can lead to desired transitions between decision classes. This process is illustrated further in Figure 3 and discussed in Section 3.5.3.
- Combine Rules from Clusters: We create all possible combinations of action rules from each cluster, considering their support. The rule combination process enhances efficiency by introducing depth-controlled exploration and applying confidence and support thresholds for pruning. This ensures only the most relevant and supported action rules are retained.
- Evaluate and Select Optimal Partition: Finally, we evaluate the sets of action rules generated at each dendrogram level by calculating their F-scores. The level with the highest F-score is selected as the optimal partition, containing the final set of action rules. While F-score is used for this study, other metrics like lightness, coverage, or the number of rules can also be considered for determining the best level.
3.5.1. Calculate Attribute Correlations
- Pearson’s Correlation Coefficient for continuous attributes.
- Cramer’s V for categorical attributes.
- Correlation Ratio for mixed types of attributes.
3.5.2. Perform Agglomerative Clustering
3.5.3. Generate Action Rules for Clusters
3.5.4. Combine Rules from Clusters
3.5.5. Evaluate and Select Optimal Partition
4. Results
4.1. Validating Movement as a Driver of Motoric Improvement
4.2. Tonal Preferences and Their Impact on Attention
4.3. Exploring the Role of Tonal and Mixed Sounds
4.4. Effects of Increased Harmonic Richness on Expression and Communication
4.5. Loudness Adjustment and Communication Improvement
Rule # | Stable Attributes | Flexible Attribute Changes | Decision Change | Support | Confidence |
---|---|---|---|---|---|
(3) | DI = N, US-b = C, Sex = M, SP = N, Sib = 2, IQ = A, TVD = B, PDe = N, OD = N, ID = N, ICF = K, PD = N, FC = F, HD = N | Movement of Hands: None → Frequent | MOTORICS: Moderate → Good | 16 | 0.888 |
(4) | Sex = M, Childbirth = N, TVD = L, PD = N, HD = N, Family = P, DI = N, ID = N, PDe = N, PPR = V, Sib = 2, IQ = C, ESD-b = C, OD = N, FC = F, SP = N, ICF = K | Muscles Tightening = None → Frequent | MOTORICS = Poor → Moderate | 16 | 0.623 |
(5) | PDe = N, ICF = K, ID = N, DI = N, OD = N, TVD = L, IQ = A, FC = F, HD = N, SP = N, PD = N, Childbirth = A, US-b = A, Family = F, ESD-e = C → N | chroma_E_quantile_binned = Low → High | CONC_ATT = None → Ssmall | 10 | 0.714 |
(6) | SP = N, DI = N, PDe = N, US-b = A, OD = N, FC = F, HD = N, PD = N, Sex = M, TVD = B, Family = R, ESD-b = C, ID = N, MD-b = N, ICF = K, IQ = A | chroma_A#_quantile_binned = Medium → Low | CONC_ATT = Small → Moderate | 8 | 0.533 |
(7) | HD = N, FC = F, IQ = A, Sib = 1, PD = N, ICF = K, DI = N, SP = N, US-b = A, Family = F, Childbirth = A, ID = N | Tonal_Noise = Tonal → Mixed | CONC_ATT = Moderate → High | 6 | 1.000 |
(8) | PD = N, Sex = M, ID = N, HD = N, IQ = A, OD = N, FC = F, SP = N, US-b = A, PDe = N, DI = N, ICF = K, ESD-e = C → C | Timbre_Complexity = High → Low, Tonal_Noise = Tonal → Noisy, Movement of Hands = None → Frequent | MOTORICS = Poor → Good | 6 | 0.600 |
(9) | ICF = K, Sib = 2, PD = N, Age = 10, PDe = N, ID = N, FC = F, Grade = 4, MD-b = D, HD = N, Childbirth = N, ESD-b = C, SP = N, OD = N, IQ = A, DI = N, Sex = M | Harmonic_Richness = Medium → High | EXPRESSION = Poor → Moderate | 6 | 1.000 |
(10) | ESD-b = C, Childbirth = N, OD = N, FC = F, ICF = K, Sib = 1, MD-b = D, PD = Y, DI = N, HD = N, Sex = F | chroma_F_quantile_binned = Low → Medium, chroma_A#_quantile_binned = Low → High, Loudness = High → Medium, Harmonic_Richness = Medium → High | COMMUNICATION = Very Poor → Moderate | 4 | 0.500 |
(11) | Sex = M, FC = F, ID = N, OD = N, SP = N, PDe = N, PD = N, ICF = K, DI = N, TVD = B, Sib = 2, HD = N, IQ = A, ESD-e = C → C, CM = N → R | chroma_F_quantile_binned = Low → Medium, chroma_D_quantile_binned = High → Medium, chroma_A#_quantile_binned = Low → High, chroma_B_quantile_binned = High → Low, chroma_F#_quantile_binned = Medium → Low, Loudness = High → Medium, Brightness = Medium → High | COMMUNICATION = Poor → Moderate | 4 | 1.000 |
5. Discussion
5.1. Impact of Action Rules on Therapeutic Outcomes
5.2. Comparison with Existing Literature
5.3. Strengths and Limitations
5.4. Implications for Practice
5.5. Future Research Directions
5.6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Alvin, J.; Andrews, J. Music Therapy; Hutchinson London: London, UK, 1975. [Google Scholar]
- Bunt, L.; Stige, B. Music Therapy: An Art Beyond Words; Routledge: London, UK, 2014. [Google Scholar]
- Cylulko, P. Typhlo music therapy interventions supporting the motor development of a child with visual disability. Interdiscip. Context Spec. Pedagog. 2018, 22, 147–159. [Google Scholar] [CrossRef]
- Aggarwal, C.C. Recommender Systems; Springer: Berlin/Heidelberg, Germany, 2016; Volume 1. [Google Scholar]
- Schafer, J.B.; Konstan, J.; Riedl, J. Recommender Systems in E-Commerce. In Proceedings of the 1st ACM Conference on Electronic Commerce, ACM, Denver, CO, USA, 3–5 November 1999; pp. 158–166. [Google Scholar] [CrossRef]
- Christensen, I.A.; Schiaffino, S. Entertainment Recommender Systems for Group of Users. Expert Syst. Appl. 2011, 38, 14127–14135. [Google Scholar] [CrossRef]
- Blanco-Fernandez, Y.; Pazos-Arias, J.J.; Gil-Solla, A.; Ramos-Cabrer, M.; Lopez-Nores, M. Providing Entertainment by Content-Based Filtering and Semantic Reasoning in Intelligent Recommender Systems. IEEE Trans. Consum. Electron. 2008, 54, 727–735. [Google Scholar] [CrossRef]
- Ivanova, M.; Raś, Z.W. Recommendation Systems in Healthcare. In Recommender Systems for Medicine and Music; Raś, Z.W., Wieczorkowska, A., Tsumoto, S., Eds.; Studies in Computational Intelligence; Springer: Cham, Switzerland, 2021; Volume 946, pp. 1–11. [Google Scholar] [CrossRef]
- Tran, T.N.T.; Felfernig, A.; Trattner, C.; Holzinger, A. Recommender Systems in the Healthcare Domain: State-of-the-Art and Research Issues. J. Intell. Inf. Syst. 2021, 57, 171–201. [Google Scholar] [CrossRef]
- Ragone, G.; Good, J.; Howland, K. How technology applied to music-therapy and sound-based activities addresses motor and social skills in autistic children. Multimodal Technol. Interact. 2021, 5, 11. [Google Scholar] [CrossRef]
- Weller, C.M.; Baker, F.A. The role of music therapy in physical rehabilitation: A systematic literature review. Nord. J. Music Ther. 2011, 20, 43–61. [Google Scholar] [CrossRef]
- Lu, G.; Jia, R.; Liang, D.; Yu, J.; Wu, Z.; Chen, C. Effects of music therapy on anxiety: A meta-analysis of randomized controlled trials. Psychiatry Res. 2021, 304, 114137. [Google Scholar] [CrossRef] [PubMed]
- Wong, H.; Lopez-Nahas, V.; Molassiotis, A. Effects of music therapy on anxiety in ventilator-dependent patients. Heart Lung 2001, 30, 376–387. [Google Scholar] [CrossRef] [PubMed]
- Nayak, S.; Wheeler, B.L.; Shiflett, S.C.; Agostinelli, S. Effect of music therapy on mood and social interaction among individuals with acute traumatic brain injury and stroke. Rehabil. Psychol. 2000, 45, 274–283. [Google Scholar] [CrossRef]
- Cylulko, P. Therapy and upbringing of visually impaired children by the use of music. In Essays on Education Through Art Time Passing and Time Enduring; Samoraj, M., Ed.; University of Warsaw: Warsaw, Poland, 2002; pp. 256–260. [Google Scholar]
- Cylulko, P.; Gladyszewska-Cylulko, J. A Model of Typhlo Music Therapy in Educational and Rehabilitation Work with Visually Impaired Persons. In Recommender Systems for Medicine and Music; Ras, Z.W., Wieczorkowska, A., Tsumoto, S., Eds.; Studies in Computational Intelligence; Springer: Cham, Switzerland, 2021; Volume 946, pp. 223–236. [Google Scholar] [CrossRef]
- Ras, Z.W.; Wieczorkowska, A. Action-rules: How to increase profit of a company. In Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery, Lyon, France, 13–16 September 2020; Springer: Berlin/Heidelberg, Germany, 2000; pp. 587–592. [Google Scholar] [CrossRef]
- Ras, Z.; Dardzinska, A. From Data to Classification Rules and Actions. Int. J. Intell. Syst. 2011, 26, 572–590. [Google Scholar] [CrossRef]
- Mardini, M.T.; Raś, Z.W. Extraction of actionable knowledge to reduce hospital readmissions through patients personalization. Inf. Sci. 2019, 485, 1–17. [Google Scholar] [CrossRef]
- Tarnowska, K.A.; Raś, Z.W.; Jastreboff, P.J. A Data-Driven Approach to Clinical Decision Support in Tinnitus Retraining Therapy. Front. Neuroinform. 2022, 16, 934433. [Google Scholar] [CrossRef] [PubMed]
- Kleć, M.; Wieczorkowska, A. Music and Healthcare Recommendation Systems. In Recommender Systems for Medicine and Music; Ras, Z.W., Wieczorkowska, A., Tsumoto, S., Eds.; Studies in Computational Intelligence; Springer: Cham, Switzerland, 2021; Volume 946, pp. 187–195. [Google Scholar] [CrossRef]
- Shin, I.H.; Cha, J.; Cheon, G.W.; Lee, C.; Lee, S.Y.; Yoon, H.J.; Kim, H.C. Automatic Stress-Relieving Music Recommendation System Based on Photoplethysmography-Derived Heart Rate Variability Analysis. In Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Chicago, IL, USA, 26–30 August 2014; pp. 6402–6405. [Google Scholar] [CrossRef]
- Ellis, D.P.W. Beat Tracking by Dynamic Programming. J. New Music Res. 2007, 36, 51–60. [Google Scholar] [CrossRef]
- McFee, B.; McVicar, M.; Faronbi, D.; Roman, I.; Gover, M.; Balke, S.; Seyfarth, S.; Malek, A.; Raffel, C.; Lostanlen, V.; et al. librosa/librosa: 0.11.0. 2025. Available online: https://zenodo.org/records/15006942 (accessed on 31 May 2024).
- Constantinescu, C.; Brad, R. An Overview on Sound Features in Time and Frequency Domain. Int. J. Adv. Stat. IT C Econ. Life Sci. 2023, 13, 45–58. [Google Scholar] [CrossRef]
- Klapuri, A.; Davy, M. Signal Processing Methods for Music Transcription; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2007. [Google Scholar]
- Jiang, D.N.; Lu, L.; Zhang, H.J.; Tao, J.H.; Cai, L.H. Music Type Classification by Spectral Contrast Feature. In Proceedings of the IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, 26–29 August 2002; Volume 1, pp. 113–116. [Google Scholar] [CrossRef]
- Dubnov, S. Generalization of Spectral Flatness Measure for Non-Gaussian Linear Processes. IEEE Signal Process. Lett. 2004, 11, 698–701. [Google Scholar] [CrossRef]
- Tzanetakis, G.; Cook, P. Musical Genre Classification of Audio Signals. IEEE Trans. Speech Audio Process. 2002, 10, 293–302. [Google Scholar] [CrossRef]
- Ellis, D.P.W. Chroma Feature Analysis and Synthesis. 2007. Available online: https://www.ee.columbia.edu/~dpwe/resources/matlab/chroma-ansyn/ (accessed on 31 May 2024).
- Benedict, A.C.; Ras, Z.W. Distributed Action-Rule Discovery Based on Attribute Correlation and Vertical Data Partitioning. Appl. Sci. 2024, 14, 1270. [Google Scholar] [CrossRef]
- Benedict, A.; Ras, Z.W. Towards Scalable Action Rule Discovery: A Structured Vertical Partitioning Method. J. Intell. Inf. Syst. 2025, 1–31. [Google Scholar] [CrossRef]
- Athanasopoulos, G.; Eerola, T.; Lahdelma, I.; Kaliakatsos-Papakostas, M. Harmonic organisation conveys both universal and culture-specific cues for emotional expression in music. PLoS ONE 2021, 16, e0244964. [Google Scholar] [CrossRef] [PubMed]
Attribute | Description |
---|---|
Age | Child’s age: 7, 8, 9, above (A) |
Sex | Male (M), Female (F) |
Place of Permanent Residence (PPR) | Large town (L), Medium town (M), Small town (S), Village (V) |
Information about Child’s Family (ICF) | Known (K), Unknown (U) |
Family | Full (F), Partial (P), Reconstructed (R), Missing (M) |
IQ | Average (A), Below Average (B), Above Average (C) |
Foster Care (FC) | Family (F), Institutional (I) |
Type of Visual Disability (TVD) | Blind (B), Residual Sight (R), Low Vision (L) |
Other Dysfunctions (OD) | Yes (Y), No (N) |
Hearing Disability (HD) | Yes (Y), No (N) |
Intellectual Disability (ID) | Yes (Y), No (N) |
Physical Disability (PD) | Yes (Y), No (N) |
Other Dysfunctions and Illnesses (DI) | Yes (Y), No (N) |
Siblings (Sib) | None (0), 1, 2, 3, More (M), No Data (?) |
Prenatal Development (PDe) | Normal (N), Abnormal (A), No Data (?) |
Childbirth | Natural (N), Caesarean (A), No Data (?) |
Grade in School (Grade) | 1, 2, 3, 4, 5, 6 |
School Program (SP) | Usual Core Curriculum (N), For Students with Moderate and Severe Mental Retardation (U) |
Attribute | Description |
---|---|
Emotional and Social Development-Beginning (ESD-b) | Correct (C), Not Correct (N) |
Emotional and Social Development-End (ESD-e) | Correct (C), Not Correct (N) |
Motor Development-Beginning (MD-b) | Normal (N), Delayed (D), Accelerated (A) |
Motor Development-End (MD-e) | Normal (N), Delayed (D), Accelerated (A) |
Using Speech-Beginning (US-b) | Average (A), Below Average (B), Above Average (C) |
Using Speech-End (US-e) | Average (A), Below Average (B), Above Average (C) |
Music | The specific pieces of music the child is listening to (m01 to m34) |
Meta-Decision Attribute | Possible Values |
---|---|
Physiological Reactions (PHYS REACT) | Big (B), Moderate (M), Small (S), None (N) |
Motorics | Good (G), Moderate (M), Poor (P), Very Poor (V) |
Concentration of Attention (CONC ATT) | High (H), Moderate (M), Small (S), None (N) |
Experience of Music (EXP MUSIC) | Big (B), Moderate (M), Small (S), Very Weak (V) |
Communication | Good (G), Moderate (M), Poor (P), Very Poor (V) |
Blindism | None (N), Small (S), Moderate (M), Big (B) |
Expression | Rich (R), Moderate (M), Poor (P), Very Poor (V) |
Decision Attribute | Possible Values |
---|---|
Sweating | High (H), Small (S), None (N) |
Breathing | Rapid (R), Moderate (M), Slow (S) |
Reddening | Big Red (B), Small Red (S), None (N) |
Changing Body Position (BODY POS) | Frequent (F), Occasional (O), None (N) |
Tightening of Muscles (TM) | Frequent (F), Occasional (O), None (N) |
Movement of Legs (ML) | Frequent (F), Occasional (O), None (N) |
Movement of Hands (MH) | Frequent (F), Occasional (O), None (N) |
Performing Co-Movements (PCM) | None (N), Occasional (O), Frequent (F) |
Expressing the desire to listen to music for longer (LLM) | Unambiguous (U), Ambiguous (A), None (N) |
Suggesting a desire to shorten the time to listen to music (SLM) | Unambiguous (U), Ambiguous (A), None (N) |
Attention to acoustic phenomena outside the music (AAP) | None (N), Occasional (O), Frequent (F) |
Nucing melody of the presented music (NM) | Frequent (F), Occasional (O), None (N) |
Performing the rhythm of presented music (PR) | Frequent (F), Occasional (O), None (N) |
Rocking for presented music (RPM) | Frequent (F), Occasional (O), None (N) |
Responding to changes in music (RCM) | Frequent (F), Occasional (O), None (N) |
Communicating with words (CW) | Often (O), Rare (R), None (N) |
Communication by gesticulation (CG) | Often (O), Rare (R), None (N) |
Communicating with a mimic (CM) | Often (O), Rare (R), None (N) |
Rocking | None (N), Occasional (O), Frequent (F) |
Head Shaking | None (N), Occasional (O), Frequent (F) |
Hands Waving (not in Front of Eyes) | None (N), Occasional (O), Frequent (F) |
Hands Waving in Front of Eyes (HWFE) | None (N), Occasional (O), Frequent (F) |
Eye Rubbing | None (N), Occasional (O), Frequent (F) |
Expression/manifestation of emotions and feelings (EEF) | Rich (R), Poor (P), None (N) |
Using gestures | Rich (R), Poor (P), None (N) |
Using mimicry | Rich (R), Poor (P), None (N) |
Vocalizing | Rich (R), Poor (P), None (N) |
Verbalization | Rich (R), Poor (P), None (N) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Benedict, A.; Ras, Z.W.; Cylulko, P.; Gladyszewska-Cylulko, J. Enhancing Typhlo Music Therapy with Personalized Action Rules: A Data-Driven Approach. Information 2025, 16, 666. https://doi.org/10.3390/info16080666
Benedict A, Ras ZW, Cylulko P, Gladyszewska-Cylulko J. Enhancing Typhlo Music Therapy with Personalized Action Rules: A Data-Driven Approach. Information. 2025; 16(8):666. https://doi.org/10.3390/info16080666
Chicago/Turabian StyleBenedict, Aileen, Zbigniew W. Ras, Pawel Cylulko, and Joanna Gladyszewska-Cylulko. 2025. "Enhancing Typhlo Music Therapy with Personalized Action Rules: A Data-Driven Approach" Information 16, no. 8: 666. https://doi.org/10.3390/info16080666
APA StyleBenedict, A., Ras, Z. W., Cylulko, P., & Gladyszewska-Cylulko, J. (2025). Enhancing Typhlo Music Therapy with Personalized Action Rules: A Data-Driven Approach. Information, 16(8), 666. https://doi.org/10.3390/info16080666