Metacognitive Rating Scale: A Study Applying a Korean Translation to Individuals with Schizophrenia

Metacognition is a higher-level cognition of identifying one’s own mental status, beliefs, and intentions. This research comprised a survey of 184 people with schizophrenia to verify the reliability of the metacognitive rating scale (MCRS) with the revised and supplemented metacognitions questionnaire (MCQ) to measure the dysfunctional metacognitive beliefs of people with schizophrenia by adding the concepts of anger and anxiety. This study analyzed the data using principal component analysis and the varimax method for exploratory factor analysis. To examine the reliability of the extracted factors, Cronbach’s α was used. According to the results, reliability was ensured for five factors: positive beliefs about worry, negative beliefs about uncontrollability and danger of worry, cognitive confidence, need for control, and cognitive self-consciousness. The negative beliefs about uncontrollability and danger of worry and the need for control on anger expression, which were both added in this research, exhibited the highest correlation (r = 0.727). The results suggest that the MCRS is a reliable tool to measure the metacognition of people with schizophrenia.


Introduction
Schizophrenia is a serious mental illness, for which hallucinations and delusions are key symptoms [1]. Schizophrenia is characterized by delusions with fixed false beliefs due to hallucinations and strong convictions; people with schizophrenia have difficulties discerning between events occurring in the external world and those occurring inside the patient's mind. As a result, symptom management is difficult, and social functioning, such as the ability to maintain a job, declines [2][3][4]. The cognitive bias causing, maintaining, or deteriorating the delusion symptom of schizophrenia is revealed as a selective disorder by the distortion of thoughts and handling such as memory accuracy and attention deficit rather than by the lack and limitation of mental ability [5,6].
Metacognition is a higher-order thinking process involving active control over oneself, allowing observation of one's flow of consciousness and recalling memories. While cognition is defined as an intelligent activity to handle a task, metacognition performs monitoring of cognitive activities [7]. Namely, metacognition means the psychological structure, knowledge, events, and processes related to the control, revision, and interpretation of the thought itself [8].
Metacognition plays a role in helping people with schizophrenia identify that their thoughts, emotions, and intentions regarding specific events are biased and foster an ability to integrate them. It calls attention to their own cognitive bias when they are exposed to a threatening or anxiety-causing situation [9][10][11]. The ability of people with schizophrenia to recognize cognitive bias that distorts information collection and cognition is called metacognitive beliefs [12]. Metacognitive beliefs can predict mental illness symptoms' frequency and effects, and dysfunctional metacognitive beliefs may cause more serious and negative effects [13,14].
In psychopathology, the self-regulatory executive function (S-REF) model, which explains metacognition, recognizes that dysfunctional metacognitive beliefs occur based on the people's type of response to negative thoughts and emotions, namely cognitive attentional syndrome (CAS) [15,16]. According to the S-REF model, dysfunctional metacognitive beliefs on worry and intrusive thoughts are related to the development and retention of hallucinations and delusions [17]. Above all, dysfunctional metacognitive beliefs are highly correlated with anger due to a belief that one's own thoughts cannot be controlled [18]. This does not indicate that better self-reflection was present in people with high metacognitive abilities.
To evaluate dysfunctional metacognitive beliefs using the S-REF model, the metacognitions questionnaire was originally developed with 65 items (MCQ) [19]; it was later shortened to 30 items (Metacognitions Questionnaire-30: MCQ-30) [8]. Similar to the original MCQ, the MCQ-30 consists of the following five factors that are distinguished conceptually despite their correlations: cognitive confidence (CC), positive beliefs about worry (POS), cognitive self-consciousness (CSC), negative beliefs about uncontrollability and danger of worry (NEG), and need for control (NC) [20]. The MCQ-30 consists of six items for each of the five factors and presents simple multidimensionally measured values on metacognition; thus, it is more economical and efficient than the original MCQ [8].
The research on the development of the MCQ [19] explained that thought control can produce adverse effects and make people reveal aggressive impulses; further, thought control is correlated with worry and anxiety vulnerability in the cognitive process [8,21]. Moeller [22] also reported that the five factors of the MCQ-30 are correlated and that anger is caused at the metacognitive level according to the research developing the metacognitive anger processing (MAP) scale, which explored the metacognitive factors of anger. Caselli et al. [23] observed the effects of metacognitive beliefs, rumination, and anger and found that metacognitive beliefs directly affect anger, separately from rumination. The risk of violently expressing anger in men and women with schizophrenia is 4.6 times and 23.3 times higher, respectively, compared to normal men and women [24]. Furthermore, Ringer and Lysaker [25] insist that the failure of people with schizophrenia to control anger expression reduces treatment results and responses. Despite previous research results, most studies on metacognition and anger have been conducted on college students and hospitalized patients with mixed clinical diagnoses.
To advance the existing literature, this research examined the correlations between metacognitive beliefs and anger in people with schizophrenia. Specifically, this study analyzed the reliability of the metacognitive rating scale (MCRS), which is a revised version of the MCQ-30 that is supplemented with anger and anxiety concepts, for people with schizophrenia. The specific purposes of this study are as follows: Firstly, to verify the reliability of the MCRS's factors. Secondly, to verify the mean differences according to the general characteristics of the MCRS.
Thirdly, to analyze the correlations of the MCRS's factors.

Materials and Methods
This research targeted people with schizophrenia and verified the reliability of the MCRS by adding anger and anxiety concepts, revising and supplementing the MCQ, which was developed to measure dysfunctional metacognitive beliefs in the general population (Appendix A).
The MCRS has 30 items, with six items for each of the five factors of the MCQ developed by Cartwright-Hatton and Wells [19]: positive beliefs about worry (POS), negative beliefs about uncontrollability and danger of worry (NEG), cognitive confidence (CC), need for control (NC) (anger expression), and cognitive self-consciousness (CSC).
The MCRS requires respondents to rate each item on a 5-point Likert scale, ranging from one point (strongly disagree) to five points (strongly agree).
Cartwright-Hatton approved the use of the MCQ for revision and supplementation in this research to verify its reliability. The MCQ was first translated into Korean. The researchers then reviewed whether further revision was necessary in terms of the translation's accuracy and adequacy of expression. Next, the researchers ensured that the translation did not change the meaning of each item. Finally, the researchers added anger and anxiety concepts reflecting Korean cultural features in consideration of cultural differences [26].
The data were collected after obtaining approval from the Korea University's Institutional Review Board (IRB No. 40525-202011-HR-068-03). The study's respondents were people with schizophrenia who were hospitalized in mental hospitals and agreed to voluntarily participate in the research after understanding the research purpose. Out of the 200 distributed questionnaires, 184 questionnaire responses were analyzed after excluding incomplete responses. To analyze the collected data, frequency analysis, descriptive statistics, factor analysis, independent samples t-test, and variance analysis were conducted using SPSS Statistics 26.0. (IBM Corp., Armonk, NY, USA)

Demographic Characteristics
In terms of demographic characteristics, 59.2% of the respondents were men and 40.8% were women. As for the level of education, 53.8% of the respondents dropped out of middle school or earlier and 46.2% graduated from middle school or received a higher level of education; thus, the level of education was low overall. Concerning the marital status, 58.7% of the respondents were unmarried, 15.2% were divorced, and 26.1% selected "other." Regarding the age, 44.6% of the respondents were 49 or under, 55.4% were 50 or over, and the mean age was 50.7 years. As for the residential type, 29.9% of the respondents lived alone, 35.9% lived with their family and friends, and 34.2% selected "other" (Table 1).

Analysis of Reliability
To analyze the MCRS's reliability and validity, exploratory factor analysis (EFA) was used to infer the factors (principal components) that could potentially account for the observed correlation. To simplify the related variables, principal component analysis was used as a factor extraction method to classify each factor's characteristics, and the method used to determine the number of factors was when the eigenvalue was 1 or higher. For the factor rotation method, the varimax method was used. To review the validity of the variables included in the extracted factors, Cronbach's α was used to analyze reliability.
If the eigenvalue was 1 or higher; if Cronbach's α was 0.6 or higher, reliability was confirmed [27] (Table 2). For POS (positive beliefs about worry), reliability was ensured as Cronbach's α was 0.643. For NEG (negative beliefs about uncontrollability and danger of worry), reliability was ensured; Cronbach's α was 0.821, the eigenvalue was greater than 1. For CC (cognitive confidence), reliability was ensured as Cronbach's α was 0.821. For NC (need for control) (anger expression), reliability was ensured as Cronbach's α was 0.696. For CSC (cognitive self-consciousness), reliability was ensured as Cronbach's α was 0.731. In sum, the reliability of the five factors was ensured.

Test for Differences between Means by Gender
There were no significant differences between the means of POS, NEG, CC, NC, and CSC by gender. Although POS, NEG, and CSC did not show significant differences, female respondents had relatively higher mean values than male respondents. CC and NC factors also did not exhibit significant differences; however, the male respondents had relatively higher mean values than the female respondents (Table 3).

Test for Differences between the Means by Level of Education
For t POS and NEC, the mean values were significantly different between the respondents who graduated from middle school graduates or received a higher level of education and the respondents who dropped out of middle school or earlier. Although no significant differences were shown for the mean values of CC, NC, and CSC by the level of education, the respondents who graduated from middle school graduates or received a higher level of education had relatively higher mean values than the respondents who dropped out of middle school or earlier (Table 3).

Test for Differences between the Means by Age
There were no significant differences between the means of POS, NEG, CC, NC, and CSC by age. However, the respondents who were 50 or over had relatively higher mean values of POS than the respondents who were 49 or under. The respondents who were 49 or under had relatively higher mean values on NEG, NC, and CSC than those who were 50 or over (Table 3).

Test for Differences between the Means by Marital Status
There were no significant differences in the means of POS, NEG, CC, NC, and CSC by marital status. However, the mean values for all the factors were highest for the respondents who were divorced, lower for the respondents who were unmarried, and even lower for the respondents who selected "other" (Table 4).

Test for Differences between the Means by Residential Type
There were no significant differences between the means of POS, NEG, NC, and CSC by residential type (Table 4).

Correlation Analysis
As a result of analyzing the correlations between the five factors, NEG and NC (anger expression) exhibited the highest correlation, with r = 0.727. CC and CSC exhibited the lowest correlation, with r = 0.397 (Table 5).

Discussion
Metacognition refers to the ability to see oneself objectively and control oneself [7]. The main symptoms of schizophrenia are delusions and hallucinations, and proper evaluation of dysfunctional metacognition is needed to ease symptoms and improve functions of people with schizophrenia. Anger expression is highly correlated with treatment results and responses to schizophrenia [25]. In this study of people with schizophrenia, we analyzed the reliability of the MCRS, which adds anger and anxiety concepts to the MCQ [19], a questionnaire designed to evaluate personal differences in assessing worry, invasive thoughts, and beliefs on cognitive function (Appendix A).
Exploratory factor analysis was performed by classifying five factors in the same way as the existing MCQ. The reliability of the items "Worrying helps me to avoid problems in the future" and "I need to worry in order to work well" was lower than that of the items "Worrying helps me to avoid problems in the future," "I need to worry in order to remain organized," "Worrying helps me to get things sorted out in my mind," and "Worrying helps me cope." The positive beliefs on worry reflected the thought that worrying helps solve problems or develop a plan to cope. When conducting exploratory factory analysis, the reliability of the item "Worrying helps me to solve problems" was 0.459. Therefore, inconsistent responses seemed to be elicited from respondents when worry was explained as a pessimistic thought. The reliability of the item "I need to worry in order to work well" was 0.370, thereby confirming that differences existed in the respondents' responses when worry was presented as carefully coping with a dangerous situation without mentioning worry.
Factor two-NEG-was reliable for all the six items: "My worrying is dangerous for me," "When you become addicted to worry, even the slightest amount of stress lead to anxiety," "My worrying thoughts persist no matter how I try to stop them," "I cannot ignore my worrying thoughts," "My worrying could make me go mad," and "When I start worrying, I cannot stop." The items represent the negative belief that worrying cannot be controlled and is dangerous. As each item's reliability was 0.743 or higher when conducting exploratory factor analysis, these items were reliable for measuring negative beliefs on worrying for people with schizophrenia.
Factor three-CC-was reliable for the following six items: "I have little confidence in my memory for words and names," "My memory can mislead me at times," "I have a poor memory," "I have little confidence in my memory for places," "I do not trust my memory," and "I have little confidence in my memory for actions." The lack of cognitive confidence indicates a lack of confidence in memory and concentration. As each item's reliability was 0.684 or higher when conducting exploratory factor analysis, these items were reliable for measuring the cognitive confidence of people with schizophrenia. However, this means there was a limitation in measuring cognitive confidence limited to personal subjective confidence in memory.
For factor four-NC (anger expression)-the items "Expressing anger is bad" and "I can express anger in words that are not aggressive" exhibited lower reliability than the following items: "I easily get angry when I feel irritated," "I always try to suppress my anger," "It is my weakness that I cannot control my anger," and "I cannot control expressions of anger." NC reflects negative beliefs including wrong beliefs, punishment, or responsibility. In this research, NC was revised and supplemented with items measuring subjective recognition of anger expression. The reliability of the factor was ensured at 0.696. In particular, the reliability of the item "If I could not control my thoughts, I would not be able to function" was 0.278, indicating that anger could be controlled differently what was stated in other items; as a result, the respondents were considered to have given inconsistent responses. Thus, there is a need to check the items related to anger expression on the NC factor through repeated research.
For factor five-CSC-the reliability of the following six items was ensured: "I think a lot about my thoughts," "I am aware of the way my mind works when I think through a problem," "I monitor my thoughts," "I am constantly aware of my thoughts," "I pay close attention to the way my mind works," and "I constantly examine my thoughts." CSC encompasses self-recognition that one cannot control worrying, including a thinking process committed to one's own thoughts. In this research, the reliability of the CSC factor was ensured at 0.731. Specifically, the reliability of the item "I am aware of the way my mind works when I think through a problem" was 0.439, indicating that the negative meaning of CSC on worrying or repeatedly occurring thoughts was slightly excluded unlike the other items. Therefore, the responses of the subjects seem to show inconsistencies.
In terms of general characteristics, the mean metacognition by gender, age, marital status, and residential type did not exhibit significant differences. However, the mean values for POS (p = 0.009) and NEG (p = 0.041) did exhibit significant differences by level of education: the respondents who graduated from middle school or received a higher level of education had significantly higher mean values than the respondents who dropped out of middle school or earlier. Given that the respondents' overall level of education was low and their mean age was 50, it would be impractical to conclude that a higher education level is related to higher metacognitive beliefs based on graduation from middle school. Nevertheless, a previous study [28] reported that self-handicapping behaviors negatively affecting academic accomplishment decrease as metacognitive beliefs increase. This explains that continuing learning, without abandoning a course of learning, is correlated with metacognitive beliefs.
The respondents' metacognition scores for each factor (range of 14-17 points) were generally higher than those (range of 8-12 points) found in a study by Wells and Cartwright-Hatton [8], which highlights the dysfunctional metacognitive beliefs of people with schizophrenia. However, the MCRS needs to be carefully compared considering that the original tools were revised and supplemented considering cultural differences. According to a study by Østefjells et al. [29], the people with schizophrenia had higher metacognition scores for dysfunctional metacognitive beliefs than the healthy control group in almost all the factors. Further, in another study, the people with mental illness symptoms had higher dysfunctional metacognitive beliefs than the healthy people [29]. Given these findings, determining the degree of dysfunctional metacognition is useful for understanding people with schizophrenia [30].
Further, the correlation coefficient between NEG and NC (anger expression) was 0.727, which was the highest correlation between the five factors; the lowest correlation was between CC and CSC, which was 0.397. Notably, the NC factor (anger expression) had high correlations with POS (r = 0.572), CC (r = 0.601), and CSC (r = 0.615). This is consistent with the previous research by Salguero et al. [18], which also found correlations between metacognitive beliefs and anger. In the end, correlation coefficients between the factors ranged between 0.397 and 0.727, indicating moderate and high correlations, and the correlations were significant. In addition, the total reliability of this research was 0.6 or higher, indicating that the MCRS was internally consistent.
This research has significance in that the reliability of the MCRS, a revised version of the MCQ that was supplemented by presenting the concepts of anger and anxiety, were ascertained with people with schizophrenia. Further, the study confirmed the high correlation between dysfunctional metacognitive beliefs and anger expression. Given the verification of the MCRS, follow-up research will be conducted to ensure that the reliability of the treatment, diagnosis, and rating of people with schizophrenia can be raised and also be used for future research.

Conclusions
This research was conducted to check the reliability of the MCRS, which revised and supplemented the MCQ to measure dysfunctional metacognition by adding the anger and anxiety concepts, in people with schizophrenia.
The MCRS consisted of five factors, with six items per factor. The MCRS was found to be reliable for measuring the metacognition of people with schizophrenia. The traditional standards of satisfactory reliability were met, but it is difficult to say that reliability was high. This research has significance as the first study to determine the metacognition of people with schizophrenia by adding anger and anxiety concepts.
There are several limitations to this study. Firstly, in this study we used EFA as an exploratory way of data analyses. Given the limitations associated with this method, future research should test replicability of the factorial structure using CFA methods. Secondly, although the reliability standards of the MCRS were generally met, they did not reach the optimal standards. The validity needs to be improved using the tools or reference instruments that confirm differences between the patient and control groups, and item correction may be required to improve internal consistency. Thirdly, this study targeted 184 people with schizophrenia hospitalized in mental hospitals, which limits the generalizability of the results. Future research should include people with schizophrenia under treatment and rehabilitation in various environments. Fourthly, the structure of the scales should be replicated in independent samples. Fifthly, mean comparisons with a healthy control group and possibly other patient samples should be provided. Sixthly, construct validity should be tested with other measurement instruments. Lastly, this study used a subjective measurement tool based on a self-reporting questionnaire. Thus, future research should develop and execute diverse measuring methods using objective third-party evaluation or physiological indicators. Informed Consent Statement: Informed consent was obtained from all participants involved in the study. Written informed consent was obtained from the participants to publish this paper.

Data Availability Statement:
The data presented in this study are available on request to the authors. Some variables are restricted to preserve the anonymity of study participants. This questionnaire is concerned with beliefs people have about their thinking. Listed below are a number of beliefs that people have expressed. Please read each item and say how much you generally agree with it using the appropriate number. Please respond to all the items. There are no right or wrong answers. Gender:_ Age: 1 Strongly disagree; 2 Disagree; 3 Unsure; 4 Agree; 5 Strongly agree.