Generalizability Theory as Construct-Related Evidence of the Validity of Job Performance Ratings
Abstract
Investigations of the construct-related evidence of the validity of performance ratings have been rare, perhaps because researchers are dissuaded by the considerable amount of evidence needed to show construct validity (Landy, 1986). It is argued that generalizability (G) theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972) is well-suited to investigations of construct-related evidence of validity because a single generalizability investigation may provide multiple inferences of validity. G theory permits the researcher to partition observed score variance into universe (“true”) score variance and multiple, distinct estimates of error variance. G theory was illustrated through the analysis of proficiency ratings of 256 Air Force jet engine mechanics. Mechanics were rated on three different rating forms by themselves, peers, and supervisors. Interpretation of G study variance components revealed suitable evidence of construct validity. Ratings within sources were reliable. Proficiency ratings showed strong convergence over rating forms, though not over rating sources. Raters showed adequate discriminant validity across rating dimensions. The expectation of convergence over sources was further questioned. © 1990, Taylor & Francis Group, LLC. All rights reserved.
Publication Title
Human Performance
Recommended Citation
Kraiger, K., & Teachout, M. (1990). Generalizability Theory as Construct-Related Evidence of the Validity of Job Performance Ratings. Human Performance, 3 (1), 19-35. https://doi.org/10.1207/s15327043hup0301_2
