Topic 2: Approaches to Language Testing

 APPROACHES TO LANGUAGE TESTING

(Sources: Heaton, J. B. 1990. Writing English Language Test. London: Longman. Page 15-24)

 

Language tests can be roughly classified according to four main approaches to testing: 1) the essay-translation approach; 2) the structuralist approach; 3) the integrative approach; and 4) the communicative approach. Although these approaches are listed here in chronological order, they should not be regarded as being strictly confined to certain periods in the development of language testing. Nor are the four approaches always mutually exclusive. A useful test will generally incorporate features of several of these approaches. Indeed, a test may have certain inherent weaknesses simply because it is limited to one approach, however attractive that approach may appear.

 

1.    The essay translation approach   
This approach is commonly referred to as the pre-scientific stage of language testing. No special skill or expertise in testing is required: the subjective judgement of the teacher is considered to be of paramount importance. Tests usually consist of essay writing, translation, and grammatical analysis (often in the form of comments about the language being learnt). The tests also have a heavy literary and cultural bias. Public examinations (e.g. secondary school leaving examinations) resulting from the essay-translation approach sometimes have an aural oral component at the upper intermediate and advanced levels -- though this has sometimes been regarded in the past as something additional and in no way an integral part of the syllabus or examination.

 

2.    The structuralist approach
This approach is characterised by the view that language learning is chiefly concerned with the systematic acquisition of a set of habits. It draws on the work of structural linguistics, in particular the importance of contrastive analysts and the need to measure the learner' s master of the separate elements of the target language: phonology, vocabulary and grammar. Such mastery is tested using words and sentences completely divorced from any context on the grounds that a larger sample of language forms can be covered in the test in a comparatively short time. The skills of listening, speaking, reading and writing are also separated from one another as much as possible because it is considered essential to test one thing at a time.

 

Such features of the structuralist approach are, of course, still valid for certain types of test and for certain purposes. For example, to desire to concentrate on the testees’ ability to write by attempting to seperate a composition test from reading (i.e. by making it wholly independent of the ability to read long and complicated instructions or verbal stimuli) is commendable in certain respects. Indeed, there are several features of this approach which merit consideration when constructing any good test.

 

The psychometric approach to measurement with its emphasis on reliability and objectivity forms an integral part of structuralist testing. Psychometrists have been able to show clearly that such traditional examinations as essay writing are highly subjective and unreliable. As a result, the need for statistical measures of reliability and validity is considered to be of the utmost importance in testing: hence the popularity of the multiple-choice item – a type of item which lends itself admirably to statistical analysis.

 

At this point, however, the danger of confusing methods of testing with approaches to testing should be stressed. The issue is not basically  a question of multiple-choice  testing versus communicative testing. There is still a limited use for multiple-choice items in many communicative tests, especially for reading and listening comprehension purposes. Exactly the same argument can be applied to the use of several other item types.

 

3.    The Integrative approach
This approach involves the testing of language in context and is thus concerned primarily with meaning and the total communicative effect of discourse. Consequently, integrative tests do not seek to separate language skills into neat divisions in order to improve test reliability: instead, they are often designed to assess the learner's ability to use two or more skills simultaneously. Thus, integrative tests are concerned with a global view of proficiency --- an underlying language competence or ‘grammar of expectancy’ (Oller, 1972), which it is argued every learner possesses regardless of the purpose for which the language is being learnt. Integrative testing involves ‘functional language’ (Cohen, 1980) but not the use of functional language. Integrative tests are best characterised by the use of doze testing and of dictation. Oral interviews, translation and essay writing are also included in many integrative tests --- a point frequently overlooked by those who take too narrow a view of integrative testing.

 

The principle of cloze testing is based on the Gestalt theory of ‘closure’ (closing gaps in patterns subconsciously). Thus, cloze tests measure the reader’s ability to decode ‘interrupted’ or ‘mutilated’ messages by making the most acceptable substitutions from all the contextual clues available. Every n^th word is deleted in a text (usually every fifth, sixth, or seventh word), and students have to complete each gap in the text, using the most appropriate word. The text used for the cloze test should be long enough to allow a reasonable number of deletions --- ideally 40 or 50 blanks. The more blanks contained in the text, the more reliable the cloze test will generally prove.

 

Dictation, another major type of integrative test, was previously regarded solely as a mean of measuring students’ skill of listening comprehension. Thus, the complex elements involved in tests of dictation were largely overlooked until fairly recently. The integrative skills involved in tests of dictation include auditory discrimination, the auditory memory span, spelling, the recognition of sound segments, a familiarity with the grammatical and lexical patterning of the language, and overall textual comprehension. Unfortunately, however, there is no reliable way of assessing the relative importance of the different abilities required, and each error in the dictation is usually penalised in exactly the same way.

 

Dictation tests can prove good predictors of global language ability even though same recent research has found that dictation tends to measure lower-order language skills such as straightforward comprehension rather than the higher-order skills such as inference(Cohen, 1980). The dictation of longer pieces of discourse (i.e. 7 to 10 words at a time) is recommended as being preferable to the dictation of shorter word groups (i.e. three to five words at a time) as in the traditional dictations of the past. Used in this way, dictation involves a dynamic process of analysis by synthesis, drawing on a learner’s ‘grammar of expectancy’ (Oller, 1972) and resulting in the constructive processing of the message heard.

 

If there is no close relationship between the sounds of a language and the symbols representing them, it may be possible to understand what is being spoken without being able to write it down. However, in English, where there is a fairly close relationship between the sounds and the spelling system, it is sometimes possible to recognise the individual sound elements without fully understanding the meaning of what is spoken. Indeed, some applied linguists and teachers argued that dictation encourages the students to focus his or her attention too much on the individual sounds rather than on the meaning of the text as a whole. Such concentration on single sound segments in itself is sufficient to impair the auditory memory span, thus making it difficult for the students to retain everything they hear.

 

When dictation is given, it is advisable to read through the whole dictation passage at approaching normal conversational speed first of all. Next, the teacher should begin to dictate (either once or twice) in meaningful units of sufficient length to challenge the student’s short-term memory span. (Some teachers mistakenly feel that they can make the dictation easier by reading out the text word by word: this procedure can be extremely harmful and only serves to increase the difficulty of the dictation by obscuring the meaning of each phrase.) finally, after dictation, the whole passage is read once more at slightly slower than normal speed.

 

4.    The communicative approach
The communicative approach to language testing is sometimes linked to the integrative approach. However, although both approaches emphasise the importance of the meaning of utterances rather than their form and structure, there are nevertheless fundamental differences between the two approaches. Communicative tests are concerned primarily (if not totally) with how language is used in communication. Consequently, most aim to incorporate tasks that approximate as closely as possible to those facing the students in real life. Success is judged in terms of the effectiveness of the communication, which takes place rather than formal linguistic accuracy. Language 'use' (Widdowson, 1978) is often emphasised to the exclusion of language ‘usage’. ‘Use’ is concerned with how people actually use language for a multitude of different purposes while ‘usage’ concerns the formal patterns of language (described in prescriptive grammars and lexicons). In practice, however, some tests of a communicative nature include the testing of usage and also assess the ability to handle the formal patterns of the target language. Indeed, few supporters of the communicative approach would argue that communicative competence can ever be achieved without a considerable mastery of the grammar of a language.

 

The attempt to measure different language skills in communicative tests is based on a view of language referred to as the divisibility hypothesis. Communicative testing results in an attempt to obtain different profiles of a learner’s performance in the language. The learner may, for example, have a poor ability in using the spoken language in informal conversations but may score quite highly on tests of reading comprehension. In this sense, communicative testing draws heavily on the recent work on aptitude testing (where it has long been claimed that the most successful tests are those which measure separately such relevant skills as the ability to translate news report, the ability to understand the radio broadcasts, or the ability to interpret speech utterances). The score obtained in a communicative test will thus result in several measures of proficiency rather than simply one overall measure.

 

Unlike the separate testing of skills in the structuralist approach, moreover, it is felt in communicative testing that sometimes the assessment of language skills in isolation may have only a very limited relevance to real life. For example, reading would rarely be undertaken solely for its own sake in academic study but rather for subsequent transfer to the information obtained to writing or speaking.

 

Perhaps the most important criterion for communicative tests is that they should be base on precise and detailed specifications of the needs of the learners for whom they are constructed: hence their particular suitability for the testing of English for specific purposes. However, it would be a mistake to assume that communicative testing is best limited to ESP or even to adult learners with particular obvious short-term goals. Although they may contain totally different task, communicative tests for young learners following general English courses are based on exactly the same principles as those for adult learners intending to enter on highly specialised courses of a professional or academic nature.

 

As pointed out in the beginning, a good test will frequently combine features of the communicative approach, the integrative approach and even the structuralist approach – depending on the particular purpose of the test and also on the various test constraints. If, for instance, the primary purpose of the test is for general placement purposes and there is very little time available for its administration, it may be necessary to administer simply a 50-item cloze test.

 

Language testing constantly involves making compromises between what is deal and what is practicible in a certain situation. Nevertheless this should not be used as an excuse for writing and administering poor tests: whatever the constraints of the situation, it is important to maintain ideals and goals, constantly trying to devise a test which is as valid and reliable as possible – and which has a useful backwash effect on the teaching and learning leading to the test.

 

Artikel lengkap bisa dibaca di buku Writing English Language Test (Heaton, 1990), download disini

Daftar hadir Topic 2: Approaches to Language Testing mata kuliah Language Evaluation disini