This methodological study aimed to translate the Korean version of the Patient-Reported Outcomes Measurement Information System® item bank-emotional distress (i.e., depression, anxiety, and anger) for adults and evaluate its psychometric properties.
Translation involved forward translation, back translation, expert review, harmonization and quality assurance, and cognitive testing. Psychometric properties were evaluated with a community sample of 201 adults. To assess the unidimensionality of the measures, we used factor analysis to examine construct validity.
All items of the Patient-Reported Outcomes Measurement Information System item bank-emotional distress measures were translated through a rigorous translation process, and semantic, conceptual, and normative equivalences between the original and Korean version were ensured. Exploratory factor analysis revealed that all items loaded on one dominant factor. The Korean Patient-Reported Outcomes Measurement Information System measures demonstrated acceptable psychometric properties and unidimensionality.
The Korean version of the Patient-Reported Outcomes Measurement Information System item bank-emotional distress (i.e., depression, anxiety, and anger) for adults is reliable and valid. Hence, it may be utilized extensively in clinical and research settings and contribute to patient-centered care.
Along with the growing significance of patient-centered care, patient-reported outcomes (PROs), directly reported by the patient who experiences the symptoms, are receiving more attention in clinical research [
Launched in 2004, the Patient-Reported Outcome Measurement Information System® (PROMIS®) has been developed by the National Institute of Health to meet the need for reliable, efficient, and flexible instruments to measure PROs in clinical research and health care delivery settings [
One of the main advantages of the PROMIS measures is that they utilize computerized adaptive testing (CAT) using item response theory (IRT). CAT enables clinicians and researchers to identify high-risk individuals with minimal burden on both patients and clinicians [
Among the numerous domains of the PROMIS, the focus of the present study is the PROMIS item bank-emotional distress measures, which consists of depression (version 1.0), anxiety (version 1.0), and anger item banks (version 1.1) [
The aim of the present study was to translate and validate the Korean version of the PROMIS emotional distress measures. When translating measures, it is important to ensure semantic, conceptual, and normative equivalence between the original and translated measures [
This methodological study was designed to develop the Korean version of the PROMIS emotional distress measures (i.e., depression, anxiety, and anger) for adults 18 years or older and evaluate their psychometric properties.
This study used the PROMIS item bank-emotional distress measures, including depression, anxiety, and anger. The Depression-Short Form 8a (8 items), Anxiety Short Form 8a (8 items), and Anger Short Form 5a (5 items) were also used. These measures use a 7-day recall period and employ response scales with five options for frequency from "1 (Never)" to "5 (Always)" or "1 (Not at all)" to "5 (Very much)." Higher scores indicate higher levels of emotional distress and the standardized T-scores (Mean=50, Standard Deviation=10) for each measure can be obtained from the PROMIS Center (the HealthMeasures team).
We received approval from the PROMIS Center (the HealthMeasures team) to translate the Korean version of the PROMIS item bank-emotional distress for adults aged 18 years or older. The translation was conducted using a standard approach based on the established Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology for accurate and culturally appropriate translation [
Cognitive interview is a standardized interview method used to assess if there is discrepancy between the intended meaning of each item of the instrument and the respondent's interpretation of the items [
To examine the psychometric properties of the Korean version of the PROMIS emotional distress measures, we recruited a convenience sample of adults aged 18 years or older from universities and community workplaces located in three cities of South Korea. Adults who are able to read and understand Korean were eligible to participate in the study. After we explained the purpose and procedure of the study, confidentiality, and voluntary nature of the study, adults who agreed to participate in the study signed the consent form and completed the paper-and-pencil questionnaire. Participants completed the questionnaire in 15 minutes at their workplace, classroom, or a place of their convenience.
To conduct exploratory factor analysis (EFA), a sample size is considered to be stable if the number of subjects is 200 or more, or if the ratio between the number of subjects and items is 5 to 1 or more [
The demographic characteristics of the study sample and each variable were summarized using descriptive statistics (i.e., means, standard deviations, frequencies, and ranges). The internal consistency of each instrument was established using Cronbach’s ⍺. The correlations between item bank and short form version, and among depression, anxiety, and anger measures were also assessed. EFA was conducted to examine construct validity. For statistical calculations, STATA version 15 (StataCorp LLC, College Station, TX, USA) was used.
This study was reviewed and approved by the institutional review board of the university (IRB No. 1512/001-015).
All items of the PROMIS item bank-emotional distress measures were translated through a rigorous translation process, and semantic, conceptual, and normative equivalences between the original and Korean version were ensured. We documented the item history of the development and decision-making process for each item, and the sample item history is presented in
Common issues that emerged during the development process were ambiguous expressions and linguistic and/or cultural inequivalence. For example, one of the items of the anger scale, "I had a bad temper," was interpreted by the participants in the cognitive interviews as "I had a bad temper before, but I do not anymore" in the Korean translation. Additionally, participants in the cognitive interviews reported that it was hard to understand another item of the anxiety scale, "I had difficulty calming down." since it was unclear "what needs to be calmed down." To clarify the meaning of the item, we added the words "anxious feelings" to the Korean version.
In addition, some words included in the original measures were deleted to make the sentence more suitable and natural for the Korean version. For example, we concluded that some words, such as "felt" and "my," were unnecessary in the Korean version since these words do not convey any specific meaning (e.g., my anger, my normal routine).
A total of 210 participants completed the questionnaire from June to August 2016 and nine cases with missing data were excluded from the analysis. Of the 201 participants, 98 (48.8%) were male and 103 (51.2%) were female, with an average age of 34.01 years. The highest number of participants, 69 (34.7%), had a monthly household income of KRW 3.01-5 million, and the most prevalent level of education, with 119 participants (59.2%), was college graduate (
Kaiser-Meyer-Olkin (KMO) test and Bartlett's test of Sphericity were performed to investigate the correlation between items to investigate whether they were suitable to be analyzed using the factor analysis method. In this study, the KMO for depression, anxiety, and anger was 0.95, 0.94, and 0.93, respectively. These were also recognized by Bartlett’s test of sphericity with a significance level of .001.
The first and second eigenvalues of depression, anxiety, and anger were 15.44, 14.10, and 10.93 and 1.05, 1.31, and 0.33, respectively. Unidimensionality is supported if the variability of the first factor accounts for more than 20% and the ratio of the magnitude of eigenvalues between factors is greater than four [
The first factors of depression, anxiety, and anger explained 80.5%, 75.0%, and 81.1% of the variance, respectively. Except for the first factor, there was one other factor for depression and two factors for anxiety with eigenvalue greater than one: however, these factors accounted for relatively small percentages of the variance (5.5%, 7.0%, 5.6%).
The factor loadings of the depression, anxiety, and anger scale items are provided in
The purpose of the present study was to translate and validate the Korean version of the PROMIS emotional distress measures (depression, anxiety, and anger), and test their psychometric properties for use in clinical and research settings. The Korean version, developed through a standardized and rigorous translation, harmonization, and quality assurance process, demonstrated acceptable psychometric properties and unidimensionality.
In this study, conceptual equivalence of the Korean version of the PROMIS emotional distress measures was assured through the FACIT translation methodology, a rigorous multi-step translation process [
Cronbach’s ⍺ s of the PROMIS emotional distress measures showed high internal consistency, ranging from .91 (anger) to .97 (depression). All the short forms also demonstrated acceptable reliability. The correlations among the item banks and short forms of the depression, anxiety, and anger scales were also very high (r≥.94), indicating the applicability of short forms in research and clinical settings. Early identification of depressive and anxiety symptoms in primary healthcare settings is particularly important in Korea, where most people with psychological symptoms visit primary healthcare clinics because of the stigma associated with mental health problems [
The EFA showed that all items of the PROMIS emotional distress measures loaded on one dominant factor and accounted for 75.01-81.10% of the total variance. In future, a confirmatory factor analysis would be needed to confirm the unidimensionality of the scale. Among depression, anxiety, and anger scales, all items had factor loadings >0.4. One item on the anxiety scale had a factor loading <0.5. The item EDANX27 “I felt something awful would happen” from the anxiety item bank is not included in the anxiety short form. However, the item was shown to be one of the most sensitive to small differences in latent anxiety in the Brazilian population at the Outpatient University Hospital [
Clinicians have reported that PRO measures are useful in assessing patients’ views regarding changes in their health status and prompting difficult and sensitive conversations about topics, such as depression, anxiety, and suicidal ideation [
Thus, the Korean version of the PROMIS emotional distress measures could be applied in clinical settings to identify high-priority symptoms and unaddressed concerns of patients, screen high-risk populations who may be overlooked, and ultimately contribute to improving health outcomes. Currently, efforts are being made to incorporate the Korean version of the PROMIS emotional distress measures into the electronic medical record system of tertiary hospitals, and the PROMIS National Center in Korea has been established (with Ju Hee Cho and Heeseung Choi as Korean representatives). CAT based on IRT modeling may be used for this measure. Using the short forms or a CAT may make it easier to measure patients' emotional distress in clinical and research settings with minimal burden and time.
The findings of the present study should be interpreted with caution because the study recruited a convenience sample of adults from the community. Future studies should investigate the advantages of and barriers to using the Korean version of the PROMIS emotional distress measures in clinical settings. In addition, cross-sectional and longitudinal studies testing these measures across diverse populations are needed.
The Korean version of the PROMIS emotional distress measures demonstrated acceptable psychometric properties and unidimensionality. It is expected that the Korean version of the PROMIS emotional distress measures can be used extensively in both clinical and research settings and may ultimately contribute to the advancement of patientcentered care. In addition to the current use of the measures in the electronic medical record system of tertiary hospitals, the measures could be widely used in the community, such as primary health clinics, university counseling centers, and mental health welfare centers. These easyto-use measures for assessing emotional distress will allow early detection and timely intervention of mental health problems.
The authors declared no conflicts of interest.
Translation Process and an Example
Translation Process | Example |
---|---|
Source | I felt discouraged about the future |
1) Two simultaneous forward translations (two independent professional translators) | Translator 1: 나는 미래에 대한 좌절감을 느꼈다. |
Translator 2: 미래에 대해 낙담하게 되었다. | |
2) Reconciled single target language translation | 나는 미래에 대한 좌절감을 느꼈다. |
3) Back-translation (one professional translators) | I felt frustrated with the future. |
4) Back-translation review | Frustrate means "to induce feelings of discouragement in." |
So, the meaning difference between "discourage" and "frustrate" is acceptable. | |
5) Expert reviews (3 Reviewers) | |
Reviewer 1: healthcare professional | Reviewer 1: 미래에 대해 좌절감을 느꼈다. |
Reviewer 2: healthcare professional | Reviewer 2: 나는 미래에 대한 좌절감을 느낀다. |
Reviewer 3: linguist | Reviewer 3: 나는 미래에 대해 좌절감을 느꼈다. |
6) Pre-finalization review | We followed Rev 3's recommendation. |
7) Finalization | 나는 미래에 대해 좌절감을 느꼈다. |
8) Harmonization and quality assurance | "Frustrated" and "discouraged" do not have the same meaning. The definition for this item says that "discouraged" means "felt deprived of confidence, hope or spirit [about the future]." Consider an alternative like, "I lost confidence in the future" or "I felt that I have no confidence in the future." |
9) Post Harmonization Final | 나는 미래에 대해 자신감을 잃었다. |
10) Literal BT of post harmonization Final | I have lost confidence in my future. |
I felt no confidence for the future. | |
11) Cognitive testing | 나는 미래에 대해 자신감을 잃었다. |
12) Translation Finalized | 나는 미래에 대해 자신감을 잃었다. |
General Characteristics of Participants (
Characteristic | Categories | n (%) |
---|---|---|
Sex | Male | 98 (48.8) |
Female | 103 (51.2) | |
Age | 20s | 90 (45.0) |
30s | 52 (26.0) | |
40s | 36 (18.0) | |
50s | 20 (10.0) | |
60s | 2 (1.0) | |
Monthly income (10,000 won) | <200 | 17 (8.5) |
201~300 | 35 (17.6) | |
301~500 | 69 (34.7) | |
501~700 | 30 (15.1) | |
>700 | 48 (24.4) | |
Level of education | Middle school diploma | 4 (2.0) |
High school diploma | 56 (27.9) | |
Bachelor's degree | 119 (59.2) | |
≥Master's degree | 22 (10.9) |
Reliability (Cronbach's ⍺) (
Categories | Depression | Anxiety | Anger |
---|---|---|---|
Item bank score reliability | Bank 28 | Bank 29 | Bank 22 |
.97 | .96 | .95 | |
Short form score reliability | Short Form 8a | Smort Form 8a | Short Form 5a |
.92 | .91 | .87 |
The Mean Scores of Depression, Anxiety, and Anger (
Variables | Items | T-score |
---|---|---|
M±SD | ||
Depression (r=.94 |
Bank 28 | 49.42±8.33 |
Short Form 8a | 49.67±7.99 | |
Anxiety (r=.94 |
Bank 29 | 46.74±8.88 |
Short Form 8a | 46.88±8.26 | |
Anger (r=.94 |
Bank 22 | 45.55±10.24 |
Short Form 5a | 44.89±9.41 |
All correlations are significant at
Factor Loadings of the Items of Depression, Anxiety, and Anger
Depression items | Factor 1 | Anxiety items | Factor 1 | Anger items | Factor 1 |
---|---|---|---|---|---|
D1_EDDEP04 | 0.68 | Ax1_EDANX01 | 0.76 | Ag1_EDANG01 | 0.72 |
D2_EDDEP05 | 0.75 | Ax2_EDANX02 | 0.76 | Ag2_EDANG03 | 0.81 |
D3_EDDEP06 | 0.79 | Ax3_EDANX03 | 0.78 | Ag3_EDANG04 | 0.69 |
D4_EDDEP07 | 0.65 | Ax4_EDANX05 | 0.82 | Ag4_EDANG05 | 0.64 |
D5_EDDEP09 | 0.70 | Ax5_EDANX07 | 0.79 | Ag5_EDANG09 | 0.76 |
D6_EDDEP14 | 0.77 | Ax6_EDANX08 | 0.75 | Ag6_EDANG10 | 0.60 |
D7_EDDEP17 | 0.76 | Ax7_EDANX12 | 0.70 | Ag7_EDANG11 | 0.61 |
D8_EDDEP19 | 0.82 | Ax8_EDANX13 | 0.66 | Ag8_EDANG15 | 0.70 |
D9_EDDEP21 | 0.55 | Ax9_EDANX16 | 0.66 | Ag9_EDANG16 | 0.72 |
D10_EDDEP22 | 0.76 | Ax10_EDANX18 | 0.53 | Ag10_EDANG17 | 0.72 |
D11_EDDEP23 | 0.73 | Ax11_EDANX20 | 0.51 | Ag11_EDANG18 | 0.67 |
D12_EDDEP26 | 0.81 | Ax12_EDANX21 | 0.64 | Ag12_EDANG21 | 0.67 |
D13_EDDEP27 | 0.76 | Ax13_EDANX24 | 0.56 | Ag13_EDANG22 | 0.62 |
D14_EDDEP28 | 0.68 | Ax14_EDANX26 | 0.72 | Ag14_EDANG26 | 0.65 |
D15_EDDEP29 | 0.78 | Ax15_EDANX27 | 0.44 | Ag15_EDANG30 | 0.75 |
D16_EDDEP30 | 0.65 | Ax16_EDANX30 | 0.74 | Ag16_EDANG31 | 0.71 |
D17_EDDEP31 | 0.81 | Ax17_EDANX33 | 0.63 | Ag17_EDANG35 | 0.80 |
D18_EDDEP35 | 0.78 | Ax18_EDANX37 | 0.70 | Ag18_EDANG37 | 0.80 |
D19_EDDEP36 | 0.82 | Ax19_EDANX40 | 0.65 | Ag19_EDANG42 | 0.75 |
D20_EDDEP39 | 0.71 | Ax20_EDANX41 | 0.75 | Ag20_EDANG48 | 0.71 |
D21_EDDEP41 | 0.79 | Ax21_EDANX44 | 0.58 | Ag21_EDANG55 | 0.73 |
D22_EDDEP42 | 0.63 | Ax22_EDANX46 | 0.78 | Ag22_EDANG56 | 0.61 |
D23_EDDEP44 | 0.70 | Ax23_EDANX47 | 0.62 | ||
D24_EDDEP45 | 0.74 | Ax24_EDANX48 | 0.78 | ||
D25_EDDEP46 | 0.77 | Ax25_EDANX49 | 0.53 | ||
D26_EDDEP48 | 0.81 | Ax26_EDANX51 | 0.81 | ||
D27_EDDEP50 | 0.73 | Ax27_EDANX53 | 0.78 | ||
D28_EDDEP54 | 0.77 | Ax28_EDANX54 | 0.76 | ||
Ax29_EDANX55 | 0.83 | ||||
Eigen value | 15.44 | 14.10 | 10.93 | ||
Variance | 80.5% | 75.0% | 81.1% |