Construct Validity in Reading Tests

Designing a good test is one of the most important tasks in teaching a reading course. The objective of this article is to provide guidelines to help teachers to write valid reading comprehension tests. To achieve this, some important areas from the reading and testing literature must be considered.

The work of Goodman (1967) made important contributions to the way in which we understand reading today. He rejected the notion that reading is a precise process of perception and identification and described it as a psycholinguistic guessing game. He suggested that the reader begins a reading task already possessing certain information, which allows her/him to pick up important fragments of the text in order to reconstruct it. The information that readers already possess is characterized as background knowledge. Authors like Coady (1979) described the role that such knowledge played in EFL/ESL reading.

Schema theory

The importance of background knowledge in reading is also central to schema theory (Rumelhart 1980). This theory claims that reading a text implies an interaction between the reader's background knowledge and the text itself. The knowledge that is organized and stored in the reader's mind is called schemata. According to this theory, fluent readers relate their schemata with the new information present in text.

Many teachers now accept the view that reading is the result of a two-way communication between the reader and the text, achieved through the simultaneous interaction of bottom-up information processing (which involves word recognition, sound/spelling correspondence, etc.,) and top-down processing (which involves skills like prediction, inference, etc.)

Construct validity in testing

Validity is an important factor in designing good reading comprehension tests. Psychometricians distinguish between several types of validity. Savignon (1983) mentions five: face validity, content validity, predictive validity, concurrent validity, and construct validity.

Construct validity explores the adequacy of a test in relation to theory. In the case of reading comprehension, tests should reflect the theoretical assumptions under which reading teachers operate. For instance, if teachers view reading as an interactive process, tests have to be designed to give students opportunities to make use of their schematic knowledge.

Factors related to the construct validity of reading comprehension tests

Carrell (1983) distinguished between formal schemata (knowledge about the structural configuration of texts) and content schemata (knowledge about the subject matter of text). She found that a reader may fail to understand a text if it does not follow a formal schema (coherent organization) or if content schema was lacking. Carrell (1984) and Alderson and Urquhart (1988) documented the discipline-specific effect of content schemata in their work with students who found it difficult to read texts which did not relate to their area of study. Stefensen, Joagdev and Anderson (1979) found that a particular content schema may not exist for a reader if that schema is culturally specific; so the texts used in a test should be culturally accessible. Rivers (1968) suggested that the cultural link between culture and language must be maintained to allow for complete understanding. Thus, previous cultural and schematic knowledge is a factor in constructing reading comprehension tests.

Rumelhart (1980) claims that bottom-up and top-down processing occur simultaneously while reading. This reading process should not be neglected in tests (Eskey 1988). A test should not concentrate solely on asking students to draw inferences from a text. The test should also allow for the reader's use of lower level decoding skills. Briefly put, tests should contain items that test both, bottom-up and top-down elements.

Tests should provide sufficient room for students to use their background knowledge. Shing and Dunkel (1992) investigating EFL listening comprehension, suggest that the passage-dependent items (listening-bound items) and passage-independent items (information related to the listener's prior knowledge) should be balanced. Eliminating passage-independent items would leave only low-level questions on the tests, reducing the chance for the listeners to make inferences and predictions while processing the aural information. On the other hand, if passage-dependent items are skipped, the test will draw primarily upon background knowledge and not measure listening skills. This testing implication of Shing and Dunkel's work is applicable to reading-comprehension tests.

Guidelines for constructing valid reading comprehension tests

Based upon this body of research, I suggest the following guidelines for constructing reading tests.

1. Reading tests should take into account the students' content schemata by including texts about topics which have been dealt with previously in class.
2. Reading tests should not include texts which are tied to a culture (i.e., culture-specific).
3. Tests should contain passage-dependent, as well as passage-independent items.
4. The texts should be coherently organized to allow students to draw the organizational patterns from their formal schemata.
5. It is advisable to use texts which are semantically complete and authentic.
6. The tasks involved in answering test items should allow for higher-level interpretation skills, as well as low level recognition skills.

These guidelines reflect the latest developments in the area of reading research within the framework of schema theory. I hope these recommendations will help teachers appreciate the importance of construct validity in assessing reading comprehension.

Wilfredo Sequera is currently teaching reading courses at the Universidad Simon Bolivar in Caracas. He is also working on his EFL master thesis at the Universidad Pedagogica Experimental Libertador in Caracas.