INTRODUCTION
Along with the growing significance of patient-centered care, patient-reported outcomes (PROs), directly reported by the patient who experiences the symptoms, are receiving more attention in clinical research [1]. Particularly, PRO measures used to assess psychological symptoms experienced by patients are gaining importance in primary health care due to a significantly high prevalence of depressive symptoms in people with chronic illness. Among 41,344 outpatients from 83 studies, 27% experienced depression or depressive symptoms [2]. Outpatients from otolaryngology clinics ranked highest in prevalence (53%), followed by patients from dermatology (39%), possibly because of chronic medical conditions, such as chronic tinnitus and atopic dermatitis [2]. Considering the significant impact of untreated depressive symptoms on the health outcomes and quality of life of patients, reliable and efficient PRO instruments for early detection of depressive symptoms are greatly needed.
Launched in 2004, the Patient-Reported Outcome Measurement Information System® (PROMIS®) has been developed by the National Institute of Health to meet the need for reliable, efficient, and flexible instruments to measure PROs in clinical research and health care delivery settings [3]. The PROMIS was developed and expanded through a series of systematic procedures, including creating extensive candidate item banks and testing candidate items with a wide range of clinical samples and general population. Using the full item banks and short forms, the PROMIS assesses many different aspects of self-reported physical, mental, and social health among the general population and individuals living with chronic conditions. As of November 1, 2018, the PROMIS adult measures (measures for adults 18 years or older) consist of 1,894 items, 118 item banks, 116 short forms, 97 domains, and 7 profiles [4]. Since their first introduction to the clinical research community, the PROMIS measures have demonstrated excellent psychometric properties and have been translated into more than 20 different languages for worldwide use. Currently, the physical, mental, and social domains of the PROMIS are being translated and validated in Korea [5].
One of the main advantages of the PROMIS measures is that they utilize computerized adaptive testing (CAT) using item response theory (IRT). CAT enables clinicians and researchers to identify high-risk individuals with minimal burden on both patients and clinicians [6,7]. Thus, the PROMIS measures are "efficient (minimizes item number without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate)"[3].
Among the numerous domains of the PROMIS, the focus of the present study is the PROMIS item bank-emotional distress measures, which consists of depression (version 1.0), anxiety (version 1.0), and anger item banks (version 1.1) [3,8]. Depression bank (28 items) assesses changes in negative mood and positive affect, anxiety bank (29 items) focuses on fear, anxious misery, hyperarousal, and somatic symptoms, while anger bank (22 items) assesses angry mood, negative social cognition, aggression, and efforts used to control anger [9]. The convergent validity of the emotional distress scales using the Minnesota Multiphasic Personality Inventory-2-Restructured Form was examined and the scales demonstrated evidence of adequate convergent validity [10]. Short forms from the emotional distress item bank are included in the emerging measures section of the Diagnostic and Statistical Manual of Mental Disorders [11]. The emotional distress measures have been validated with various groups of people in different countries, including healthy adults in Brazil [12], patients with cancer in Israel [13], and general and clinical populations in the Netherlands [14].
The aim of the present study was to translate and validate the Korean version of the PROMIS emotional distress measures. When translating measures, it is important to ensure semantic, conceptual, and normative equivalence between the original and translated measures [15], in addition to establishing psychometric properties of the measures. For measures developed based on IRT models, it is also important to conduct IRT modeling assumptions and model fit testing for the translated version.
METHODS
1. Study Design
This methodological study was designed to develop the Korean version of the PROMIS emotional distress measures (i.e., depression, anxiety, and anger) for adults 18 years or older and evaluate their psychometric properties.
2. Instruments
This study used the PROMIS item bank-emotional distress measures, including depression, anxiety, and anger. The Depression-Short Form 8a (8 items), Anxiety Short Form 8a (8 items), and Anger Short Form 5a (5 items) were also used. These measures use a 7-day recall period and employ response scales with five options for frequency from "1 (Never)" to "5 (Always)" or "1 (Not at all)" to "5 (Very much)." Higher scores indicate higher levels of emotional distress and the standardized T-scores (Mean=50, Standard Deviation=10) for each measure can be obtained from the PROMIS Center (the HealthMeasures team).
3. Sample and Procedure
1) Translation of the Korean Version of the Patient-Reported Outcomes Measurement Information System
We received approval from the PROMIS Center (the HealthMeasures team) to translate the Korean version of the PROMIS item bank-emotional distress for adults aged 18 years or older. The translation was conducted using a standard approach based on the established Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology for accurate and culturally appropriate translation [16]. The Korean version of the PROMIS emotional distress measures for adults was developed through a series of steps, including forward translation, back translation, evaluation of the Korean version by three experts, and harmonization and quality assurance. Harmonization and quality assurance performed by the PROMIS Center (the HealthMeasures team) included an assessment of the equivalence of the final translation and proper documentation of the decision-making process.
Cognitive interview is a standardized interview method used to assess if there is discrepancy between the intended meaning of each item of the instrument and the respondent's interpretation of the items [17]. For a translated instrument, cognitive interview is used to explore if the meaning of the original item is retained after translation. We conducted individual cognitive interviews lasting 45 to 60 minutes with five adults in their 20s to 40s recruited through a convenience sampling strategy and analyzed participants' responses to identify the linguistic validity and acceptability of the items. Item history containing a step-by-step development and decision-making process was created for each item.
2) Evaluation of Psychometric Properties of the Measures
To examine the psychometric properties of the Korean version of the PROMIS emotional distress measures, we recruited a convenience sample of adults aged 18 years or older from universities and community workplaces located in three cities of South Korea. Adults who are able to read and understand Korean were eligible to participate in the study. After we explained the purpose and procedure of the study, confidentiality, and voluntary nature of the study, adults who agreed to participate in the study signed the consent form and completed the paper-and-pencil questionnaire. Participants completed the questionnaire in 15 minutes at their workplace, classroom, or a place of their convenience.
To conduct exploratory factor analysis (EFA), a sample size is considered to be stable if the number of subjects is 200 or more, or if the ratio between the number of subjects and items is 5 to 1 or more [18]. The Korean version of depression, anxiety, and anger item banks used in this study contain 28, 29, and 22 items, respectively, which meet the minimum sample size requirements.
The demographic characteristics of the study sample and each variable were summarized using descriptive statistics (i.e., means, standard deviations, frequencies, and ranges). The internal consistency of each instrument was established using Cronbach’s ⍺. The correlations between item bank and short form version, and among depression, anxiety, and anger measures were also assessed. EFA was conducted to examine construct validity. For statistical calculations, STATA version 15 (StataCorp LLC, College Station, TX, USA) was used.
RESULTS
1. Translation of the Korean Version of the Patient-Reported Outcomes Measurement Information System
All items of the PROMIS item bank-emotional distress measures were translated through a rigorous translation process, and semantic, conceptual, and normative equivalences between the original and Korean version were ensured. We documented the item history of the development and decision-making process for each item, and the sample item history is presented in Table 1.
Common issues that emerged during the development process were ambiguous expressions and linguistic and/or cultural inequivalence. For example, one of the items of the anger scale, "I had a bad temper," was interpreted by the participants in the cognitive interviews as "I had a bad temper before, but I do not anymore" in the Korean translation. Additionally, participants in the cognitive interviews reported that it was hard to understand another item of the anxiety scale, "I had difficulty calming down." since it was unclear "what needs to be calmed down." To clarify the meaning of the item, we added the words "anxious feelings" to the Korean version.
In addition, some words included in the original measures were deleted to make the sentence more suitable and natural for the Korean version. For example, we concluded that some words, such as "felt" and "my," were unnecessary in the Korean version since these words do not convey any specific meaning (e.g., my anger, my normal routine).
2. Evaluation of the Psychometric Properties of the Measures
A total of 210 participants completed the questionnaire from June to August 2016 and nine cases with missing data were excluded from the analysis. Of the 201 participants, 98 (48.8%) were male and 103 (51.2%) were female, with an average age of 34.01 years. The highest number of participants, 69 (34.7%), had a monthly household income of KRW 3.01-5 million, and the most prevalent level of education, with 119 participants (59.2%), was college graduate (Table 2).
Table 3 shows the Cronbach’s ⍺ of each measure. The reliability of the measures for the 28-item depression, 29-item anxiety, and 22-item anger banks was .97, .96, and .95, respectively; and for the 8-item depression, 8-item anxiety, and 5-item anger short forms was .92, .91, and .87, respectively. All these measures had high internal consistency. High correlations between item banks and short forms were observed for all depression, anxiety, and anger measures ranging from 0.938 to 0.944. Strong relationships were observed between depression, anxiety, and anger measures (0.64≤r≤0.77: p<.001). Table 4 shows the mean score of each measure. The T-scores were 49.42±8.33 for depression, 46.74±8.88 for anxiety, and 45.55±10.24 for anger.
Kaiser-Meyer-Olkin (KMO) test and Bartlett's test of Sphericity were performed to investigate the correlation between items to investigate whether they were suitable to be analyzed using the factor analysis method. In this study, the KMO for depression, anxiety, and anger was 0.95, 0.94, and 0.93, respectively. These were also recognized by Bartlett’s test of sphericity with a significance level of .001.
The first and second eigenvalues of depression, anxiety, and anger were 15.44, 14.10, and 10.93 and 1.05, 1.31, and 0.33, respectively. Unidimensionality is supported if the variability of the first factor accounts for more than 20% and the ratio of the magnitude of eigenvalues between factors is greater than four [19]. The ratios of the first and second factor eigenvalues were greater than four in our data.
The first factors of depression, anxiety, and anger explained 80.5%, 75.0%, and 81.1% of the variance, respectively. Except for the first factor, there was one other factor for depression and two factors for anxiety with eigenvalue greater than one: however, these factors accounted for relatively small percentages of the variance (5.5%, 7.0%, 5.6%).
The factor loadings of the depression, anxiety, and anger scale items are provided in Table 5. No rotations were performed because a one factor solution was accepted. All items of depression, anxiety and anger had loadings >.40 on the first factor. EDANX27 item (I felt something awful would happen) had a slightly lower factor loading (0.44).
DISCUSSION
The purpose of the present study was to translate and validate the Korean version of the PROMIS emotional distress measures (depression, anxiety, and anger), and test their psychometric properties for use in clinical and research settings. The Korean version, developed through a standardized and rigorous translation, harmonization, and quality assurance process, demonstrated acceptable psychometric properties and unidimensionality.
In this study, conceptual equivalence of the Korean version of the PROMIS emotional distress measures was assured through the FACIT translation methodology, a rigorous multi-step translation process [16]. This methodology ensures linguistic validation after the translation process and helps evaluate the suitability of the translated version for self-administration [17].
Cronbach’s ⍺ s of the PROMIS emotional distress measures showed high internal consistency, ranging from .91 (anger) to .97 (depression). All the short forms also demonstrated acceptable reliability. The correlations among the item banks and short forms of the depression, anxiety, and anger scales were also very high (r≥.94), indicating the applicability of short forms in research and clinical settings. Early identification of depressive and anxiety symptoms in primary healthcare settings is particularly important in Korea, where most people with psychological symptoms visit primary healthcare clinics because of the stigma associated with mental health problems [20,21]. In addition, early identification and treatment of mental disorders are closely associated with better patient outcomes and prognoses [22].
The EFA showed that all items of the PROMIS emotional distress measures loaded on one dominant factor and accounted for 75.01-81.10% of the total variance. In future, a confirmatory factor analysis would be needed to confirm the unidimensionality of the scale. Among depression, anxiety, and anger scales, all items had factor loadings >0.4. One item on the anxiety scale had a factor loading <0.5. The item EDANX27 “I felt something awful would happen” from the anxiety item bank is not included in the anxiety short form. However, the item was shown to be one of the most sensitive to small differences in latent anxiety in the Brazilian population at the Outpatient University Hospital [12]. Participants of the present study were not recruited from clinical settings but from a community. Therefore, the factor loadings may have been relatively low because the item was not appropriate for measuring anxiety. This item needs to be considered when conducting future studies on communities or clinical groups.
Clinicians have reported that PRO measures are useful in assessing patients’ views regarding changes in their health status and prompting difficult and sensitive conversations about topics, such as depression, anxiety, and suicidal ideation [23-25]. Furthermore, a previous study reported that compared to patients with cancer in the usual care group, those in the intervention group using PROs as routine measure experienced more days with lower psychological stress [26].
Thus, the Korean version of the PROMIS emotional distress measures could be applied in clinical settings to identify high-priority symptoms and unaddressed concerns of patients, screen high-risk populations who may be overlooked, and ultimately contribute to improving health outcomes. Currently, efforts are being made to incorporate the Korean version of the PROMIS emotional distress measures into the electronic medical record system of tertiary hospitals, and the PROMIS National Center in Korea has been established (with Ju Hee Cho and Heeseung Choi as Korean representatives). CAT based on IRT modeling may be used for this measure. Using the short forms or a CAT may make it easier to measure patients' emotional distress in clinical and research settings with minimal burden and time.
The findings of the present study should be interpreted with caution because the study recruited a convenience sample of adults from the community. Future studies should investigate the advantages of and barriers to using the Korean version of the PROMIS emotional distress measures in clinical settings. In addition, cross-sectional and longitudinal studies testing these measures across diverse populations are needed.
CONCLUSION
The Korean version of the PROMIS emotional distress measures demonstrated acceptable psychometric properties and unidimensionality. It is expected that the Korean version of the PROMIS emotional distress measures can be used extensively in both clinical and research settings and may ultimately contribute to the advancement of patientcentered care. In addition to the current use of the measures in the electronic medical record system of tertiary hospitals, the measures could be widely used in the community, such as primary health clinics, university counseling centers, and mental health welfare centers. These easyto-use measures for assessing emotional distress will allow early detection and timely intervention of mental health problems.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
