Internal Consistency Reliability - Tutorial
At the most basic level, there are three methods that can be used to evaluate the internal consistency reliability of a scale: inter-item correlations, Cronbach's alpha, and corrected item-total correlations. More modern and sophisticated approaches to the estimation of internal consistency reliability exist and will be treated in an another tutorial.
Cronbach's alpha is arguably the most commonly used metric used to evaluate the internal consistency reliability associated with scores derived from a scale. If you ask most any researcher, he or she will likely tell you that Cronbach's alpha must be at least .70. Unfortunately, as pointed out by (Lance, Butts, & Michels, 2006), this often cited criterion, claimed to have been articulated by Nunnally, is actually misleading. I encourage you to read the Lance et al. (2006) paper. Essentially, Nunnaly and Bernstein (1994) state that .70 may be an acceptable minimum for a scale that is newly developed. By contrast, basic research should rely upon scales that yields scores with a minimum reliability of .80. In cases where important decisions are being made based on scores from a scale, a reliability in excess of .90 should be expected.
Ferketich (1991) recommended that corrected item-total correlations should range between .30 and .70 for a good scale. In my opinion, corrected item-total correlations in the low .20s can sometimes add reliable variance to a scale (see video below). In practice, negatively keyed items struggle to exhibit really good psychometric characteristics and often find themselves in the corrected item-total correlation range of .20 to .40.
References
● Ferketich, S. (1991). Focus on psychometrics: Aspects of item analysis. Research in Nursing & Health, 14, 165–168.
● Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods, 9(2), 202-220.
● Nunnaly, J. C., & Bernstein, I. H. (1994). Psychometric theory. Sydney: McGraw-Hill.
Video:
http://www.youtube.com/watch?v=2gHvHm2SE5s