Empiricism versus connoisseurship: establishing the appropriacy of texts in tests of academic reading

Abstract

This paper explores the potential of using information technology for the empirical measurement of contextual parameters identified in the socio-cognitive framework (the framework developed by CRELLA for test validation). Previously the level of difficulty represented by reading texts used in English Language tests has been estimated through expert judgement. The study, in contrast, employs computational tools for automated text analyses, demonstrating how relevant contextual features have been opened to empirical measurement by new technology. The techniques showcased in this paper allow for a more rigorous approach to reading test specification.