Adapting the Motivated Strategies for Learning Questionnaire to the Japanese Problem-Based Learning Context: A Validation Study

The COVID-19 pandemic has greatly changed medical education, and medical trainees’ self-regulation has become more emphasized. In Japan, the concept of self-regulated learning has not been fully applied in health profession education due to a lack of effective measurement tools. We aimed to validate the translated Japanese version of the Motivated Strategies for Learning Questionnaire in the context of Problem-Based Learning (J-MSLQ-PBL). The questionnaire employs a seven-point Likert-type scale with 81 items and is categorized into two sections: motivation and learning strategies. An exploratory factor analysis (EFA) was conducted by using Promax rotation to examine the factor structure of the scale, using the collected data from 112 Japanese medical students. Factor extraction was based on a scree plot investigation, and an item was accepted when the factor loading was ≥0.40. In the motivation section, the extracted factors from the EFA were well aligned with the subscales of the original MSLQ, including “Self-Efficacy for Learning and Performance”, “Task Value”, “Self-Efficacy for Learning and Performance”, “Test Anxiety”, “Extrinsic Goal Orientation”, and “Intrinsic Goal Orientation”. In the learning strategies, the extracted factors poorly matched the structure of the original subscales. This discrepancy could be explained by insufficient translation, the limited sample size from a single medical school, or cross-cultural differences in learning strategies between Western and Japanese medical students. Only the motivation part of the J-MSLQ-PBL should be implemented to measure the competency elements of self-regulated learning in Japan.


Introduction
COVID-19 has substantially changed medical education. Social distancing is strongly recommended during a pandemic, and in-person educational activities in medical schools and hospitals have been suspended; therefore, the modality of learning for medical students has changed to an online mode [1,2]. This results in fewer opportunities for trainees in medicine to meet their classmates and talk with the faculty and mentors at medical schools [2]. With the increased use of online learning during the pandemic, students spent less time in live learning opportunities and more time in self-directed learning activities. Medical students' autonomy and self-regulation should be more emphasized during and post-COVID-19 pandemic, as self-regulated learning is a fundamental process that allows students to adapt to the unusual situation (e.g., pandemic) and to develop planning, prevision, and monitoring of their learning activities and wellness [3].

Design and Participants
This was a secondary analysis study using a database, which was initially collected to examine the impact of self-regulated learning during the problem-based learning (PBL) course [11] at Jichi Medical University, Japan, on medical students' professional identity formation. Of all 124 third-year medical students invited to participate in this study, 112 agreed.

Measurements
The MSLQ, which is a seven-point Likert-scale survey, includes two sections: motivation and learning strategies [12]. The motivation section includes 31 items assessing three domains: goal orientation, self-belief, and test anxiety. The learning-strategies section includes 50 items assessing three domains: the use of cognitive strategies, metacognitive strategies, and resource management. The principal investigator of the research project (Y.M.) and the supervisor (A.L.J.) created a Japanese version of the MSLQ by translating all 81 items into Japanese, with backtranslation [13]. In the translation process, we adapted the item descriptions to the context of PBL to make the MSLQ suitable for assessing the participants' self-regulated learning competencies during the PBL course. We named this scale the J-MSLQ-PBL.

Context
The one-day PBL program for third-year medical students was divided into four segments: (1) an opening case discussion for the formulation of the self-study objectives, (2) a self-study period for objectives and preparation for subsequent group discussion, (3) a group discussion that included within-group information sharing, and (4) a 60 min wrap-up lecture from a specialist. This survey was conducted in the orientation phase of the PBL course.

The Theoretical Framework of Validation
According to Kane [14], the validation of a measurement method requires gathering evidence to examine the four key inferences: (1) the scoring of a single observation (scoring), (2) using the primary observation score to generate the whole test performance (generalization), (3) inferring the real-life performance from the test performance (extrapolation), and (4) interpreting this information to make a decision (implication). In addition, recent validation studies in health sciences education have commonly used Kane's framework for translating psychomimetic tools in English into other languages [15,16].

Analysis
We conducted an exploratory factor analysis (EFA) with the maximum likelihood method and Promax rotation to examine the factor structure of the Motivation and Learning Strategies Scales of the J-MSLQ. Factor extraction was based on parallel analysis, and an item was accepted when the factor loading was ≥0.40. Due to the small sample size, a confirmatory factor analysis was not conducted. The Kaiser-Meyer-Olkin (KMO) test was used to test the suitability of the scale for the sampling adequacy. Cronbach's alpha was calculated as a measure of internal reliability. All data analyses were conducted in R (Version 4.2.1) and R studio (2022.07.2 Build 576) with the packages psych (version 2.2.9) and GPArotation (version 2022.10-2).

Ethics
This study was approved by the Jichi Medical University Clinical Research Ethics Committee (reference number: 18-168). Informed consent was obtained from all participants. Tables 1 and 2 show the mean score, standard deviation, median, and first and third quartiles in items of the J-MSLQ-PBL, respectively. Q5, Q6, Q15, and Q31, which were categorized as "Self-Efficacy for Learning and Performance", tended to have low scores (less than three points). In contrast, Q4, Q17, and Q23, which were categorized as "Task Value", tended to have high scores (more than five points). In the items of the Learning Strategies Scale, trends in scores were not observed.

Factor Analysis
Because of the negative correlation with the total scale, Q3 of the Motivation Scale was excluded from the EFA. The KMO tests for the Motivation and Learning Strategies Scales were 0.788 and 0.754, respectively. Based on parallel analysis, the 30-item Motivation Scale and the 50-item Learning Strategies Scale suggested six and five factors, respectively. The factor loadings and proportions of variance explained by the factors are outlined in Tables 3 and 4. Table 3. Factor loadings and variance explained from an exploratory factor analysis of the Motivation Scale.

Self-Efficacy for
Learning and Performance  In the Motivation Scale (Figure 1), the first factor, which explained 12.2% of the total variance in the data, was labeled "Self-Efficacy for Learning and Performance" based on the high loadings of Q6, Q15, Q20, and Q31. The second factor, which explained 9.9%, was labeled "Task Value" based on the high loadings of Q17, Q23, Q26, and Q27. The third factor, which explained 8.5%, was labeled "Control of Learning Beliefs and Self-Efficacy for Learning and Performance" based on the high loadings of Q2, Q5, Q18, Q21, and Q29. The fourth factor, which explained 8.0%, was labeled "Extrinsic Goal Orientation" based on the high loadings of Q7, Q11, Q13, Q22, and Q30. The fifth factor, which explained 7.1%, was labeled "Test Anxiety" based on the high loadings of Q8, Q14, Q19, and Q28. The last factor, which explained 6.7%, was labeled "Intrinsic Goal Orientation" based on the high loadings of Q1, Q10, Q24, and Q25. In total, 52.4% of the total variability in the data was explained by the factor structure. In the Motivation Scale (Figure 1), the first factor, which explained 12.2% of the variance in the data, was labeled "Self-Efficacy for Learning and Performance" base the high loadings of Q6, Q15, Q20, and Q31. The second factor, which explained 9.9%, labeled "Task Value" based on the high loadings of Q17, Q23, Q26, and Q27. The t factor, which explained 8.5%, was labeled "Control of Learning Beliefs and Self-Effi for Learning and Performance" based on the high loadings of Q2, Q5, Q18, Q21, and The fourth factor, which explained 8.0%, was labeled "Extrinsic Goal Orientation" b on the high loadings of Q7, Q11, Q13, Q22, and Q30. The fifth factor, which expla 7.1%, was labeled "Test Anxiety" based on the high loadings of Q8, Q14, Q19, and The last factor, which explained 6.7%, was labeled "Intrinsic Goal Orientation" base the high loadings of Q1, Q10, Q24, and Q25. In total, 52.4% of the total variability in data was explained by the factor structure.  In the Learning Strategies Scale (Figure 2), the first factor explained 10.9% of the total variance in the data. The second to fifth factors explained 8.9%, 8.5%, 6.4%, and 5.2%, respectively. In total, 39.9% of the total variability in the data was explained by the factor structure. There were no trends in the categories of items with high loadings in each factor; hence, labeling factors according to loadings was difficult.
In the Learning Strategies Scale (Figure 2), the first factor explain variance in the data. The second to fifth factors explained 8.9%, 8.5% spectively. In total, 39.9% of the total variability in the data was ex structure. There were no trends in the categories of items with high lo hence, labeling factors according to loadings was difficult.

Internal Reliability of the Motivation Scale and Subscales
After excluding the items with a factor loading less than 0.40, t of the overall Motivation Scale was 0.87 (26 items). The six subscales (four items), 0.81 (four items), 0.80 (four items), 0.79 (five items), 0.72 (four items) for "Self-Efficacy for Learning and Performance", "Task Learning Beliefs and Self-Efficacy for Learning and Performance", " tation", "Test Anxiety", and "Intrinsic Goal Orientation", respectiv ternal reliability of the Motivation Scale, as well as the subscales oth was adequate (α > 0.70).

Discussion
This study aimed to collect and examine validity evidence for of the MSLQ adapted to the PBL context regarding Kane's four steps o [14,15]. In addition, we found that the internal structure of the motiv MSLQ-PBL was consistent with the theory of self-regulated learning ing-strategies section did not align with the original structure of the

Scoring Inference
One of the potential reasons for the inconsistency of the learni between the J-MSLQ-PBL and the original MSLQ could be the insuffi cess for the J-MSLQ-PBL. While the J-MSLQ-PBL was developed usin method, the method could be inappropriate for some translations wh to sociocultural factors such as the learning culture, educational sys in cultural backgrounds. This could be a barrier to correctly translat in health-sciences education. The descriptions of the motivation-sec cise because the section focuses on the planning of goal setting for le

Internal Reliability of the Motivation Scale and Subscales
After excluding the items with a factor loading less than 0.40, the Cronbach's alpha of the overall Motivation Scale was 0.87 (26 items). The six subscales were as follows: 0.88 (four items), 0.81 (four items), 0.80 (four items), 0.79 (five items), 0.72 (four items), and 0.64 (four items) for "Self-Efficacy for Learning and Performance", "Task Value", "Control of Learning Beliefs and Self-Efficacy for Learning and Performance", "Extrinsic Goal Orientation", "Test Anxiety", and "Intrinsic Goal Orientation", respectively ( Table 3). The internal reliability of the Motivation Scale, as well as the subscales other than goal setting, was adequate (α > 0.70).

Discussion
This study aimed to collect and examine validity evidence for the Japanese version of the MSLQ adapted to the PBL context regarding Kane's four steps of validity arguments [14,15]. In addition, we found that the internal structure of the motivation section of the J-MSLQ-PBL was consistent with the theory of self-regulated learning; however, the learning-strategies section did not align with the original structure of the MSLQ.

Scoring Inference
One of the potential reasons for the inconsistency of the learning-strategies section between the J-MSLQ-PBL and the original MSLQ could be the insufficient translation process for the J-MSLQ-PBL. While the J-MSLQ-PBL was developed using the backtranslation method, the method could be inappropriate for some translations whose topic is sensitive to sociocultural factors such as the learning culture, educational system, and differences in cultural backgrounds. This could be a barrier to correctly translating assessment tools in health-sciences education. The descriptions of the motivation-section items were concise because the section focuses on the planning of goal setting for learning at an individual level. However, the descriptions of the learning-strategies section were more complicated because these subscales refer to applying strategies, monitoring performance, and reflecting on performance in the self-regulated learning process [17]. In addition, the learners' behavior related to the subscale on profound learning (e.g., critical and metacognitive thinking) and interpersonal learning (e.g., peer learning and helpseeking) could be susceptible to cultural influences. To deal with this issue, cross-cultural survey guidelines recommend using a team translation model that employs bilingual ex-perts to ensure proper translation and cross-cultural and linguistic equivalences between the two language survey versions [18].

Generalization Inference
The internal structure of the motivation section was consistent with the original MSLQ. In addition, Cook et al. [10] reported that the motivation section of the MSLQ was well-validated in medical residents by performing a correlation, reliability, and factor analysis. Furthermore, Miyabe also conducted a validation study of the Japanese version of the MSLQ with first-year nursing students [19]. The current study furthermore examined the validity evidence of the J-MSLQ-PBL and demonstrated that the scale's internal structure is consistent with the SRL theory. This indicates that the J-MSLQ-PBL for the PBL context can be generalizable to measure Japanese medical students' SRL competency elements for learning in PBL. However, this study was conducted in a single private medical school in Japan; thus, further validation study in another type of institution is needed to expand the generalizability. Furthermore, a more robust statistical analysis, such as a structural equation model using a larger sample size, would improve the generalizability of the J-MSLQ-PBL.

Implication Inference
Our study also showed that the results of EFA for the learning-strategies section did not align with the SRL theory. There are several possible reasons. Most of Japan's educational contents in preclinical medical education are didactic, even if PBL is partially included in the curriculum [20]. Thus, Japanese medical students have fewer opportunities to develop their skills in learning strategies than medical students in other countries [21]. In this sense, there was the possibility of a content validity issue; Japanese medical students perhaps could not understand the meaning of the descriptions in the learning-strategies sections because they are less experienced in using these skills. In addition, it has been reported that the group dynamics of Japanese medical students in PBL are inactive. Therefore, the students' interactions during PBL may differ from those in other English-speaking countries [22]. This could influence the response to the items of the help-seeking and peer-learning subscales, as it is considered that there is a social-interaction pattern within the learning environment that is unique to Japanese medical trainees [18,23].

Extrapolation Inference
Due to its limited coherence with theory, the learning-strategies section is not applicable for usage, as the validity evidence in both the literature and the present study for the section was insufficient. On the contrary, the motivation section was well-validated from both perspectives.
There are limitations to arguing the implications of the J-MSLQ-PBL. First, as discussed, there would be an issue of face validity due to the insufficient translation procedure; thus, the team translation model needs to be applied to consider the cultural influence on the translation process. In addition, integrating qualitative evidence, such as in-depth interviews to explore the students' perceptions of item descriptions, could help improve the face validity.

Conclusions
This study examined validity evidence for the Japanese version of the MSLQ adapted to the PBL context. The internal structure of the motivation section of the J-MSLQ-PBL was consistent with the theory of self-regulated learning; however, the learning-strategies section did not align with the original structure of the MSLQ. Thus, only the motivation section of the J-MSLQ-PBL can be implemented to measure competency elements in self-regulated learning in Japan. Additional work is needed to clarify the reason for the discrepancy in the learning-strategies section between the J-MSLQ-PBL and the MSLQ.