Public Domain, English and Japanese Scales for Measuring Self-Esteem, Anxiety, Depression, Understanding, Warmth, Altruism, Creativity, and Intellectuality: Validity and Reliability for use with Male and Female Adults in
Japan and the United States
Philip TROMOVITCH*
(Received July 9, 2014)
The present paper reports on the bilingual development of scales for assessing eight psychological constructs and personality traits. The constructs include: self-esteem, anxiety, depression, understanding, warmth, altruism, creativity, and intellectuality. The analyses employed two national samples of adults, one from Japan and one from the United States. Seven- to ten-item scales for the eight constructs were developed starting from ten- or eleven-item scales composed of International Personality Item Pool (IPIP) items that prior research had found to be valid and/or reliable in English. A process of elimination was used to retain the best items for valid and reliable use with male and female adults, of both countries, together or in combination. Findings regarding validity and reliability of the scales are provided. Future work may include developing versions in additional languages and reducing the size of each scale to six or seven items, while maintaining high reliability and validity across populations.
Key words: scale development, cross-cultural, multilingual, psychological assessment, self-esteem
1. Introduction
A large proportion of research in psychology and the social sciences involves measuring various personality, psychological, and other constructs and traits (hereafter: constructs). In most instances, for practical reasons this must be accomplished with the use of so-called 'paper and pencil' instruments, which today can be implemented via numerous computer-based technologies. In order to measure such constructs in a way that allows confidence to be placed in the results of the research, valid and reliable measures (hereafter:
scales) need to be developed. A great deal of work has gone into the development and validation of such scales, however, in the contemporary world many social scientists have to deal with two common obstacles.
First, many of these scales are proprietary or copyrighted, consequently researchers must obtain permission to use them and oftentimes must have sufficient funding to pay usage fees. Second, in many areas of research it is difficult to determine if a finding applies to humans in general, or instead is a result of the culture in which one's research participants grew up, thus necessitating research in multiple cultures which frequently means conducting the research in multiple languages.
Two lesser obstacles also exist. Many scales have been developed and tested for validity and reliability only on college student or other convenience samples; for such scales, it is not known if the scales are valid for use with other, broader populations.
Additionally, many scales are initially developed to
* Science and Engineering Research Institute, Doshisha University, Kyoto Telephone: +81-774-65-6671, E-mail:[email protected]
contain multiple subscales, after which many researchers extract and use only a single subscale needed for their specific project. The problem here is that some questions on a scale may be highly context- dependent, that is, the answers to such items are highly influenced by the questions that appeared before them (or around them), thus raising questions as to the validity of using subscales outside of the context of their development.
1.1 The IPIP Items
To begin to address some of the foregoing obstacles, the International Personality Item Pool (IPIP) was developed. The IPIP started in 1996 with over 1,300 individual items, written in Dutch. Most were translated into English, and after some exclusions and additions of new English items over 1,200 English items were in the IPIP -- and therefore in the public domain -- by 19991). By 2006 over 2,000 items were available2); in 2014 over 2,400 English items became available.
In order to greatly improve access and communication between researchers, Goldberg established a website3) with the items, information related to their use, and contact information for researchers working on or with the IPIP. The present research is a contribution to this ongoing work.
1.2 Developing Scales for Widespread Use
In order to increase the likelihood that a scale will be valid and reliable on a wide range of populations and sub-populations, scale development should be conducted either with multiple samples where each sample is drawn from a different sub-population (to ensure validity and reliability across many sub- populations), or scale development should be conducted using either a nationally representative sample or a national sample that has good coverage across a range of demographic backgrounds. For scales intended for use in more than one country, samples drawn from multiple countries are needed.
1.3 Developing (Sub)Scales that Maintain Validity and Reliability after Removal from the Context of their Development
In order to increase the likelihood that a subscale will be valid and reliable when taken out of the context of its original development, item selection must be designed so that context-independent items will be more likely to be selected or retained for the final scale.
With computer-based administration of scales, this is now an easy task, the scale developer must merely use computer-based survey administration software that allows randomization of the survey items. By randomizing the items for each participant during survey development, all survey items will appear in a wide variety of contexts. Those items that have high context-dependence will perform poorly and can be removed (e.g., their item-total correlations will be low, hence they will not be chosen for inclusion in the final version of a scale).
1.4 Scale Length
Many researchers prefer to use multi-item scales because they believe such scales have good reliability. On average, it may be true that longer scales have higher reliability, but it should be remembered that even single-item scales can have high reliability and that multi-item scales can have low reliability.
Nevertheless, using multi-item scales is a generally sound approach.
When trying to measure a construct that is difficult to precisely define (e.g., self-esteem), single- item scales may not be possible because the only practical way to measure the construct is to do so from multiple "angles", which necessitates the use of multiple items to assess different aspects or manifestations of the target construct.
Furthermore, it is often desirable to have a wide range of possible measurements for more precise delineation and separation between subjects along a continuum. For example, a single item with a 5-point
response only allows for 5 levels of delineation, whereas a three item scale using 5-point responses allows for 13 levels of delineation.
The present researcher prefers to have at least 10 levels of potential delineation, with 20 being desirable. Given that the IPIP items are traditionally scored using a 5-point Likert-type response, three items per scale would seem to be the absolute minimum.
However, when using 5-point response choices labeled as the IPIP items are (very accurate, moderately accurate, neither accurate nor inaccurate, moderately inaccurate, very inaccurate), some researchers may want to trichotomize their participants' responses (e.g., by merging the adjacent very and moderately categories); in this case five items becomes the minimum necessary for each scale (i.e., 11 levels of potential delineation). Using five items as a minimum is also desirable for assessing difficult to measure constructs, such as are being developed in the present research.
Although scales with more items may be more likely to have high reliability and scales with more diversity of items may appear to have higher face validity, having a lot of items burdens the participants in research, and in longer questionnaires limits the number of constructs that can be assessed. Thus, one goal of the current project is to develop scales that are as short as is practical, while meeting all other goals for the scales' usability, validity, and reliability across cultures and languages.
Given the above, and that all of the initial item sets (see below) contained at least ten items, the present researcher decided to make five items a minimum requirement for the final scales, with six or seven items being desirable. Since the long-term goal is to develop scales using the same items translated into multiple, additional languages, and with each new language/culture/country some items may not perform as desired, in the phase of the research presented here scales were allowed to be as long as ten items in length.
Thus, the long-term hope is to further reduce the number of items in each scale, based on more data from more countries, ultimately resulting in six-item or seven-item scales of the eight constructs that are valid and reliable across numerous Eastern and Western adult populations.
2. The Current Project
The purpose of the research presented in this article was to develop valid and reliable scales for eight constructs of interest (i.e., self-esteem, anxiety, depression, understanding, warmth, altruism, creativity,
& intellectuality), where the scales would be in the public domain and therefore freely usable by any researcher, where they would have good validity and reliability across many populations and sub-populations, and be valid and reliable when used in any of multiple language versions, whether used together or separately.
The present paper reports on the development of preliminary English and Japanese versions; the present author hopes to expand the research in the future to include additional Eastern and Western languages and reduce the number of items to six or seven per scale.
2.1 Selection of Eight sets of IPIP Items
Sets of IPIP items that had already been found to have good internal reliability (alpha > 0.7) as well as to be valid and/or reliable for measuring the eight constructs of interest were selected by the researcher.
Since these sets had already been shown to have at least some validity and reliability in some population, it was believed that there was a good likelihood they would perform well in the present research. When multiple sets of items could be chosen for a given construct, the present researcher assessed the face validity of the items within each set and selected the set that appeared to have greater face validity. In the event of a tie in terms of face validity, preference was given to sets that the
present researcher believed would be easier to translate into multiple languages.
The items selected for development of scales for assessing altruism, anxiety, depression, and intellectuality were based on item sets previously shown to correlate well with the corresponding scale in the NEO Personality Inventory (NEO-PI-R)4). Items sets for assessing creativity, understanding, and warmth were previously shown to correlate well with the corresponding scale in the Abridged Big Five- dimensional Circumplex (AB5C)5). Items for assessing self-esteem were those shown to perform well in comparison to the corresponding item set in the Personal Attributes Survey (PAS)6).
2.2 Scale Development Samples
Data was collected from two national samples;
one from the United States and one from Japan. The samples were national quota samples designed to have approximately equal numbers of males and females, and approximately equal numbers of people of various ages from 18 to 59 years of age. Although not part of the sampling design, demographic data collected from the samples suggested good coverage or representation across numerous variables such as the region they live in, the size of the area where they grew up, the socioeconomic status of their family when they were growing up, and the highest level of formal education they had completed (for more detailed information please contact the author).
Usable data was collected and analyzed from N = 476 Japanese males, N = 453 Japanese females, N = 487 U.S. males, and N = 478 U.S. females.
2.3 Statistical Methods
Processing of each of the eight scales was handled independently. All procedures were separately applied to seven datasets: the data from the four subgroups separately (Japanese males, Japanese females, U.S. males, U.S. females) as well as to the
combined data by country (Japanese adults, U.S.
adults), and to the total dataset (Japanese and U.S.
adults of both sexes; N = 1,900), to ensure that retained items would be widely applicable and valid for use across a broad range of populations and population mixes. All of the initial sets of items contained ten or eleven items; a process of elimination was used to eliminate weak or poorly performing items, as described next.
Initial item-level elimination: Items were eliminated if they had been left unanswered by more than 1% of the respondents (since this could result from an item being ambiguous or confusing), if their arithmetic mean was extreme (since this could cause floor or ceiling effects; 'extreme' was defined as less than 0.5 or greater than 3.5 on a 5-point scale running from 0 to 4), or if their standard deviation was very small (since this would limit variability and therefore the ability to distinguish between close but different levels of the construct; low variability was defined as less than 0.6), in any of the seven datasets.
Scale-level elimination: Item-total correlation analyses were computed. Item-total correlations measure each item's correlation with the total from the scale (excluding its own contribution), thereby providing a measure of how well the given item measures the construct assessed by the scale; high item- total correlations indicate high validity. Items were removed if they had a weak association with the total (defined as an item-total correlation r < 0.3) or if they were excessively correlated (defined as an item-total correlation r > 0.9). This process was repeated iteratively across all seven datasets, removing the worst performing item with each iteration.
Secondary item-level elimination: inter- correlation matrixes were computed for all scales for all datasets and examined for bloated specifics. Bloated specifics are pairs of items that are highly correlated with each other, usually because they are paraphrases of each other or because they are logical opposites (with
one item being reverse scored). Bloated specifics were defined as item pairs with an inter-correlation of 0.8 or higher. One item of each pair would be removed, with the selection being based on a holistic comparison of each item's likely contribution to its scale.
Within-scale Item Ranking: Although the ultimate goal is to reduce all of the scales to six or seven items in length, until future research in other languages and cultures has been completed it is prudent to retain all good items. Consequently rather than selecting the best six or seven items, the retained items for each scale were ranked starting from the most desirable item for use (maximum contribution to validly and reliably measuring the construct, with good psychometric properties). Since all retained items demonstrated good performance, even the last listed item for each scale is valid for use. Rankings were based on a holistic examination of the available data taking into consideration: face validity for measuring the construct, item-total correlations, valence (with the goal being to ensure all scales have at least two positive valence and two negative valence items), an assessment of likely translatability into other languages, means, standard deviations, and inter-correlations with other items in the scale.
3. Results
It was possible to construct valid and reliable scales for all eight constructs. Of the seventy initial items, only six failed to meet the retention criteria.
Item-total correlations of retained items ranged from a low of 0.32 to a high of 0.83, suggesting high validity for all retained items across all tested populations.
Cronbach's alpha for the eight scales, which measures internal reliability, ranged from 0.75 to 0.93 across the seven datasets, demonstrating high reliability across populations. The scales, together with key assessment statistics, appear in the Appendix.
3.1 IPIP Scale Usage
IPIP-based scales are typically administered with instructions such as: "Following is a list of phrases describing behaviors and personality traits. Please indicate how accurately each statement describes you as you generally are now." One item appears on each row and respondents indicate their response on a non- numbered scale containing the five labels (from left to right; often arranged as a matrix-type question): very accurate, moderately accurate, neither accurate nor inaccurate, moderately inaccurate, very inaccurate.
3.2 IPIP Scale Scoring
Scoring of the scales can be handled in a number of ways. Following are three possible methods.
(1) Traditional Scoring Method: The IPIP items are traditionally scored from 1 (very inaccurate) to 5 (very accurate), with negative valence items being reverse scored (i.e., from 5 to 1). A scale score is computed by summing the item scores. Although this is the traditional method when using IPIP items, this author prefers an alternative approach that (a) standardizes the range for scale scores, (b) has zero as the lowest possible score, and (c) that allows for easy handling of data that is missing at random (e.g., a respondent leaves one item unanswered; this can happen easily when using computer administered questionnaires since a respondent may click just outside of the area that will register the click, and move to the next question without noticing that their response was not recorded).
(2) Recommending Scoring Method: The individual items are scored from 0 (very inaccurate) to 4 (very accurate); reverse valence items are scored from 4 to 0. Scale scores are computed as the arithmetic mean of the item scores. Thus, scale scores are real numbers having a fixed range from 0 to 4 where the number of possible unique scores will be:
(#items x 4) + 1.
Optionally, researchers can use this procedure even for respondents who leave one item unanswered.
(3) Alternative Scoring Method: In populations where respondents are excessively reluctant, or eager, to label themselves with extreme scores, or where a comparison is being made where one group may tend toward reluctance and the other may tend toward eagerness, the validity of the comparison may be threatened if either of the above scoring methods are used since the distinction between the adjacent 'very' and 'moderate' responses may not be valid. In such cases, the adjacent 'very' and 'moderate' responses can receive the same score, effectively reducing the 5-point answers to 3-point scoring, while eliminating the effects of excessively reluctant/eager responding. In this case, individual items are scored as in (2) above, except that item, and therefore scale scores range from 0 to 2 and the number of possible unique scores drops to:
(#items x 2) + 1.
4. Conclusion
The eight scales appear to have good validity and reliability in adult males, females, Japanese respondents, and U.S. respondents. The scales are in the public domain and can be freely used by other researchers. Future research should conduct additional translations with the aim of increasing the number of language versions available, and with the further collection of validity and reliability data, the reduction of the scale lengths to six or seven items per scale.
Additionally, given that the creativity scale assesses what might be termed Intellectual Creativity, and given that the intellectuality scale measures a very similar construct, future research might examine combining these two scales into a single scale assessing the larger, combined construct.
Future research may also explore a variation on the recommended scoring method where the highest and lowest item scores on a given scale, for each respondent, are excluded before computing the scale score for that respondent, to determine if this increases scale validity and/or reliability by minimizing anomalous responses.
This research was supported in part by a grant- in-aid from the Science and Engineering Research Institute of Doshisha University.
References
1) L. R. Goldberg, "A Broad-bandwidth, Public Domain, Personality Inventory Measuring the Lower-level Facets of Several Five-factor Models". In I. Mervielde, I.
Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality Psychology in Europe, Vol. 7 (pp. 7-28). Tilburg, Netherlands: Tilburg University Press (1999).
2) L. R. Goldberg, J. A. Johnson, H. W. Eber, R. Hogan, M.
C. Ashton, C. R. Cloninger, & H. C. Gough, "The International Personality Item Pool and the Future of Public-domain Personality Measures", Journal of Research in Personality, 40, 84-96 (2006).
3) L. R. Goldberg, "International Personality Item Pool: A Scientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences". Available at http://ipip.ori.org/
updated July 18, 2014.
4) P. T. Costa Jr., & R. R. McCrae, Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) Professional Manual. Odessa, FL:
Psychological Assessment Resources (1992).
5) W. K. B. Hofstee, B. De Raad, & L. R. Goldberg,
"Integration of the Big-Five and Circumplex Approaches to Trait Structure", Journal of Personality and Social Psychology, 63, 146-163 (1992).
6) M. Rosenberg, Society and the Adolescent Self-image.
Princeton, NJ: Princeton University Press (1965).
Appendix. The Scales, Items (in both English and Japanese), and Key Statistics.
Scale (Cronbach's alpha range)
Item-Total Correlations
Valence Males Females Combined Item Self-esteem (.86 - .90)
-
.73 .71 .72 Dislike myself.63 .69 .66 自分自身に嫌悪感を持っている
+
.70 .67 .68 Feel comfortable with myself.61 .66 .64 自分自身に満足している
-
.73 .67 .70 Feel that I'm unable to deal with things .68 .67 .68 物事に対処する能力がないと思う+
.63 .57 .61 Just know that I will be a success.63 .60 .62 自分自身が成功するのは確実だ
-
.66 .64 .65 Feel that my life lacks direction.66 .68 .67 人生の目標を見失っていると感じる
+
.46 .52 .50 Know my strengths.63 .64 .63 自分の強みを知っている
-
.58 .50 .54 Am less capable than most people.67 .66 .67 他のほとんどの人に比べて能力がない
+
.47 .54 .51 Seldom feel blue.54 .61 .58 憂鬱になることはめったにない
-
.56 .50 .54 Question my ability to do my work properly.51 .42 .46 自分自身の適切に仕事や勉強をこなす能力を疑問視している
+
.49 .32 .41 Like to take responsibility for making decisions.43 .47 .46 責任を持って意思決定をするのがすきだ
Anxiety (.87 - .89)
+
.67 .71 .70 Get stressed out easily.71 .73 .72 簡単にストレスを感じる
-
.65 .62 .65 Am not easily bothered by things.71 .74 .73 ものごとに簡単に悩まされない
+
.62 .70 .68 Worry about things.61 .70 .66 心配性だ
-
.60 .65 .64 Am relaxed most of the time.57 .58 .58 だいたいにおいてリラックスしている
-
.51 .57 .56 Don't worry about things that have already happened.62 .66 .64 もう終わったことは心配しない
+
.61 .64 .62 Get caught up in my problems.63 .62 .62 自分のいろいろな問題が気になって、何事も手がつかない
+
.61 .53 .58 Am afraid of many things.67 .70 .68 多くのことに恐怖心を抱いている
+
.67 .71 .68 Fear for the worst.47 .58 .53 最悪の事態を恐れる
-
.46 .46 .46 Adapt easily to new situations.55 .47 .51 新しい状況に簡単に適応できる
-
.40 .45 .46 Am not easily disturbed by events.55 .60 .58 出来事に簡単にかき乱されない
Depression (.90 - .93)
+
.78 .82 .80 Often feel blue.78 .83 .80 しょっちゅう憂鬱になる
+
.81 .81 .81 Am often down in the dumps.77 .75 .76 しばしば落ち込む
-
.56 .64 .61 Seldom feel blue.71 .77 .74 憂鬱になることはめったにない
-
.68 .69 .69 Feel comfortable with myself.59 .66 .63 自分自身に満足している
+
.80 .79 .80 Dislike myself.74 .77 .76 自分自身に嫌悪感を持っている
+
.79 .75 .77 Have a low opinion of myself.49 .64 .57 自分自身を低く評価している
+
.69 .66 .67 Feel that my life lacks direction.67 .70 .68 人生の目標を見失っていると感じる
+
.74 .74 .74 Feel desperate.70 .72 .71 絶望的な気分だ
-
.67 .64 .66 Am very pleased with myself.60 .66 .63 自分自身が好きだ
+
.57 .66 .62 Have frequent mood swings.37 .48 .43 しょっちゅう機嫌が変わる
Understanding (.87 - .91)
-
.66 .70 .69 Can't be bothered with other's needs.59 .62 .61 他人を助ける時間はない
+
.70 .61 .67 Sympathize with others' feelings.63 .58 .61 他人の気持ちに共感する
-
.62 .58 .61 Take no time for others.60 .66 .63 他人のためには自分の時間を割かない
-
.52 .63 .58 Feel little concern for others.68 .63 .66 他人にあまり関心がない
-
.59 .58 .59 Am not interested in other people's problems.72 .60 .66 他人の問題に興味がない
+
.50 .60 .55 Like to be of service to others.68 .67 .67 他人を助けることが好きだ
-
.53 .59 .56 Am indifferent to the feelings of others.65 .63 .64 他人の気持ちに無関心である
+
.63 .60 .62 Respect others' feelings.61 .52 .56 他人の気持ちを尊重している
+
.59 .63 .62 Take others' interests into account.49 .54 .51 他人の興味を考慮している
+
.59 .50 .55 Appreciate the viewpoints of others.38 .54 .46 他人の意見をありがたく思う
Warmth (.85 - .91)
+
.75 .74 .75 Make others feel good.72 .69 .70 他人を気持ちよくする
+
.73 .76 .75 Make people feel welcome.64 .60 .61 他人を喜んで歓迎する
+
.67 .72 .70 Make people feel at ease.70 .61 .66 他人に安らぎを与える
+
.65 .72 .69 Take time out for others.60 .59 .60 他人のために時間を割く
-
.61 .55 .59 Am not really interested in others.57 .50 .54 他人にあまり興味がない
-
.40 .52 .46 Rarely smile.41 .42 .42 めったに笑顔を見せない
+
.67 .69 .69 Know how to comfort others.60 .59 .60 他人を慰める方法を知っている
+
.66 .66 .66 Am interested in people.62 .51 .57 他人に興味がある
+
.64 .60 .62 Show my gratitude.56 .59 .58 感謝の気持ちを表している
+
.51 .56 .54 Feel others' emotions.50 .43 .47 他人の感情を感じる
Altruism (.80 - .86)
+
.61 .65 .63 Love to help others.64 .62 .63 他人を助ける事が好きだ
-
.59 .56 .58 Turn my back on others.59 .57 .58 他人が困っていても気にならない
+
.64 .68 .66 Make people feel welcome.59 .54 .56 他人を喜んで歓迎する
-
.59 .59 .60 Take no time for others.56 .59 .58 他人のためには自分の時間を割かない
-
.43 .45 .45 Am indifferent to the feelings of others.60 .54 .58 他人の気持ちに無関心である
+
.54 .51 .51 Have a good word for everyone.52 .51 .52 他人が喜ぶような事を言う
-
.57 .40 .50 Make people feel uncomfortable.35 .36 .36 他人を不快にしてしまう
+
.41 .46 .44 Anticipate the needs of others.42 .39 .41 他人のニーズを察することができる
-
.54 .53 .54 Look down on others.38 .30 .34 他人を見下げている
Creativity (.75 - .82)
+
.46 .57 .52 Like to solve complex problems.56 .57 .57 複雑な問題を解決するのが好き
+
.52 .52 .52 Can easily link facts together.57 .53 .55 様々な事実の関連性を容易に把握できる
-
.51 .53 .52 Avoid philosophical discussions.44 .49 .47 哲学的な議論を避ける
-
.43 .53 .48 Am not interested in theoretical discussions.43 .51 .48 理論的な議論に興味がない
+
.43 .41 .41 Ask questions that nobody else does.49 .41 .46 誰も質問しないことを質問する
+
.43 .51 .48 Know the answers to many questions.55 .58 .57 多くの質問に対する答えを持っている
-
.54 .44 .49 Have difficulty understanding abstract ideas.35 .35 .35 抽象的な概念の理解に困難を感じる
Intellectuality (.81 - .86)
+
.60 .67 .63 Love to read challenging material.66 .66 .67 難しい物を読むことが好きだ
-
.53 .58 .55 Avoid difficult reading material.65 .64 .65 難しい物を読む事は避けている
+
.54 .57 .56 Like to solve complex problems.56 .64 .61 複雑な問題を解決するのが好き
+
.50 .53 .52 Can handle a lot of information.49 .55 .53 多くの情報を処理することができる
-
.53 .53 .53 Avoid philosophical discussions.51 .51 .51 哲学的な議論を避ける
+
.45 .46 .46 Enjoy thinking about things.56 .47 .52 考え事をするのが好きだ
+
.46 .50 .49 Have a rich vocabulary.48 .49 .48 ボキャブラリーが豊富だ
-
.44 .50 .47 Am not interested in theoretical discussions.48 .55 .52 理論的な議論に興味がない
-
.56 .52 .54 Have difficulty understanding abstract ideas.38 .33 .36 抽象的な概念の理解に困難を感じる
Notes: "Cronbach's alpha range" provides the lowest and highest alpha reliability statistic from the seven development samples of participants aged 18-59: Japanese men N = 476 , Japanese women N = 453, Japanese adults (the two prior groups combined) N = 929, U.S. men N = 487, U.S. women N = 478, U.S. adults N = 965, and adults (Japanese & U.S. adults combined) N = 1,900.
Valence indicates if the item needs to be reversed scored (-) or normally scored (+). Item-total correlations are based on the data from the country of data collection matching the language (Japan or the United States). Items that comprise a scale are listed in order with the "best" item listed first; researchers wishing to use scales with an equal number of items (e.g., six items per scale) can select the top items (e.g., the top six items listed for a given scale). All of the data and items in the table as well as additional data (e.g., means and standard deviations of the individual items) are available from the author as a Microsoft Excel (XLS) file.