EVALUATION OF ENGLISH LANGUAGE TEACHING
Evaluation is included in three essential components of language teaching.
What is evaluation?
What are the two other components?
How do we do evaluation in language teaching?
let's figure out the answers of those questions from this video...
Daftar hadir Topic 1: Evaluation of English Language Teaching disini
TYPES OF LANGUAGE TESTS
A test, in simple terms, is a method of measuring a person’s ability, knowledge, or performance in a given domain1. The needs of assessing the outcome of learning have led to the development and elaboration of different test formats. language tests differ with respect to how they are designed, and what they are for, in other words, in respect to test method and test purpose.
In terms of method, we can broadly distinguish traditional paper-and-pencil language tests from performance tests3. Paper-and-pencil language tests are typically used for the assessment either of separate components of language knowledge (grammar, vocabulary etc.), or of a receptive understanding (listening and reading comprehension). In performance-based tests, the language skills are assessed in an act of communication. Performance tests1 are most commonly tests of speaking and writing.
In terms of purpose, several types of language tests have devised to measure the learning outcomes accordingly. Generally, five types of language tests are given to language leamers in order to make decisions: placement tests, diagnostic tests, achievement tests, proficiency tests and aptitude tests.2
Placement tests
The term ‘placement test’, as Richards et al. (1989) note, does not refer to what a test contains or how it is constructed, but to the purpose for which it used. A placement test is designed to place learners at an appropriate level in a programme or course. In most cases, language learners with similar language abilities should be grouped together. The instructor can then work with individuals who are at approximately the same level of development in language skills and can help them strengthen those skills.
Diagnostic tests
a diagnostic test is primarily designed to diagnose some particular linguistic aspects. The test is prepared in order to discover the strengths and weaknesses of language learners. This information is useful to language instructors in helping the trainees make improvement in their language skills. Diagnostic tests in pronunciation, for example, might have the purpose of determining which particular phonological features of the English language are more likely to pose problems and difficulties for a group of learners.
Basically, diagnostic language tests have a three- fold objective:
1. To provide learners with a way to start learning with their own personal learning programme or what would be called in the literature of testing learning paths.
2. To provide learners with a way to test their knowledge of a language.
3. To provide learners with better information about their strengths and weaknesses.
Achievement tests
An achievement test is also referred to as attainment or summative test. In more practical and pedagogical terms, Brown (1994) defines achievement tests as “tests that are limited to particular material covered in a curriculum within a particular time frame‟. This test is devised to measure how much of a language someone has learned with reference to a particular course of study or programme of instruction and to provide some indication of whether the instructional goals are being met. Achievement test is based directly on the instructional content of the course. As mentioned earlier, the objectives of the instructional program and the objectives of the testing should be in harmony with each other, thus leading to beneficial backwash. The test has a two-fold objective:
1. To help the teachers judge the success of their teaching.
2. To identify the weaknesses of their learners.
Proficiency tests
A proficiency test is devised to measure how much of a language someone has learned. It is not linked or tied to any particular course of instruction or instructional objectives, but measures the learner’s general level of language mastery. as Valette posits, “the aim of a proficiency test is to determine whether this language ability corresponds to specific language requirements‟ (Valette, 1977).
Proficiency tests have traditionally consisted of standardized multiple choice items on grammar, vocabulary, reading comprehension, and aural comprehension. Sometimes, a sample of writing is added, and more recent tests also include oral production performance. The example of standardized proficiency test is TOEFL.
Aptitude tests
Aptitude, or language aptitude, refers to the genuine ability one is endowed with to learn a language. It is thought to be a combination of several abilities:
Phonological ability, i.e. the ability to detect phonetic differences (e.g. of stress, intonation, vowel quality) in a new language.
Syntactic ability, i.e., the ability to recognize the different grammatical functions of words in sentences.
Psychological ability, i.e. rote-learning abilities and the ability to make inferences and inductive learning.
A language aptitude test tends to measure a learner aptitude for language learning. It provides some indication of an individual's ability to learn a language.Thus, it is used to identify those learners who are most likely to succeed. Language aptitude tests usually consist of several different test items which measures such abilities as:
Sound-coding ability, i.e. the ability to identify and remember new sounds in a new language.
Grammar-coding ability, i.e. the ability to identify the grammatical functions of different parts of sentences.
Inductive-learning ability, i.e. the ability to work out meanings without explanation in the new language.
Memorization, i.e. the ability to remember and to recall words, patterns, rules in the new language.
But caution should be used when administering an aptitude test. Some factors, such as language learning strategies, attitude, motivation and personality factors, may not be measured in aptitude tests.
Sources:
http://staffnew.uny.ac.id/upload/131763786/pengabdian/ppm-3a.pdf
https://faclettre.univ-tlemcen.dz/assets/uploads/DOCUMENTS/cours%20en%20ligne/4-TYPE-L-BENM.pdf
Presensi Language Evaluation Topic 3 bisa diakses disini
APPROACHES TO LANGUAGE TESTING
(Sources: Heaton, J. B. 1990. Writing English Language Test. London: Longman. Page 15-24)
Language tests can be roughly classified according to four main approaches to testing: 1) the essay-translation approach; 2) the structuralist approach; 3) the integrative approach; and 4) the communicative approach. Although these approaches are listed here in chronological order, they should not be regarded as being strictly confined to certain periods in the development of language testing. Nor are the four approaches always mutually exclusive. A useful test will generally incorporate features of several of these approaches. Indeed, a test may have certain inherent weaknesses simply because it is limited to one approach, however attractive that approach may appear.
1.
The essay translation approach
This approach is commonly referred to as the
pre-scientific stage of language testing. No special skill or expertise in
testing is required: the subjective judgement of the teacher is considered to
be of paramount importance. Tests usually consist of essay writing,
translation, and grammatical analysis (often in the form of comments about the
language being learnt). The tests also have a heavy literary and cultural bias.
Public examinations (e.g. secondary school leaving examinations) resulting from
the essay-translation approach sometimes have an aural oral component at the
upper intermediate and advanced levels -- though
this has sometimes been regarded in the past as something additional and in no
way an integral part of the syllabus or examination.
2.
The structuralist
approach
This approach is characterised by the view that language learning is chiefly
concerned with the systematic acquisition of a set of habits. It draws on the
work of structural linguistics, in particular the importance of contrastive
analysts and the need to measure the learner' s master of the separate elements
of the target language: phonology, vocabulary and grammar. Such mastery is
tested using words and sentences completely divorced from any context on the
grounds that a larger sample of language forms can be covered in the test in a
comparatively short time. The skills of listening, speaking, reading and
writing are also separated from one another as much as possible because it is
considered essential to test one thing at a time.
Such features of the structuralist approach are, of course, still valid for certain types of test and for certain purposes. For example, to desire to concentrate on the testees’ ability to write by attempting to seperate a composition test from reading (i.e. by making it wholly independent of the ability to read long and complicated instructions or verbal stimuli) is commendable in certain respects. Indeed, there are several features of this approach which merit consideration when constructing any good test.
The psychometric approach to measurement with its emphasis on reliability and objectivity forms an integral part of structuralist testing. Psychometrists have been able to show clearly that such traditional examinations as essay writing are highly subjective and unreliable. As a result, the need for statistical measures of reliability and validity is considered to be of the utmost importance in testing: hence the popularity of the multiple-choice item – a type of item which lends itself admirably to statistical analysis.
At this point, however, the danger of confusing methods of testing with approaches to testing should be stressed. The issue is not basically a question of multiple-choice testing versus communicative testing. There is still a limited use for multiple-choice items in many communicative tests, especially for reading and listening comprehension purposes. Exactly the same argument can be applied to the use of several other item types.
3.
The Integrative approach
This approach involves the testing of language in context and is thus concerned
primarily with meaning and the total communicative effect of discourse.
Consequently, integrative tests do not seek to separate language skills into
neat divisions in order to improve test reliability: instead, they are often
designed to assess the learner's ability to use two or more skills
simultaneously. Thus, integrative tests are concerned with a global view of
proficiency --- an underlying language competence or ‘grammar of expectancy’ (Oller, 1972), which it
is argued every learner possesses regardless of the purpose for which the
language is being learnt. Integrative testing involves ‘functional language’ (Cohen, 1980) but not
the use of functional language. Integrative tests are best characterised by the
use of doze testing and of dictation. Oral interviews, translation and essay
writing are also included in many integrative tests --- a point frequently
overlooked by those who take too narrow a view of integrative testing.
The principle of cloze testing is based on the Gestalt theory of ‘closure’ (closing gaps in patterns subconsciously). Thus, cloze tests measure the reader’s ability to decode ‘interrupted’ or ‘mutilated’ messages by making the most acceptable substitutions from all the contextual clues available. Every nth word is deleted in a text (usually every fifth, sixth, or seventh word), and students have to complete each gap in the text, using the most appropriate word. The text used for the cloze test should be long enough to allow a reasonable number of deletions --- ideally 40 or 50 blanks. The more blanks contained in the text, the more reliable the cloze test will generally prove.
Dictation, another major type of integrative test, was previously regarded solely as a mean of measuring students’ skill of listening comprehension. Thus, the complex elements involved in tests of dictation were largely overlooked until fairly recently. The integrative skills involved in tests of dictation include auditory discrimination, the auditory memory span, spelling, the recognition of sound segments, a familiarity with the grammatical and lexical patterning of the language, and overall textual comprehension. Unfortunately, however, there is no reliable way of assessing the relative importance of the different abilities required, and each error in the dictation is usually penalised in exactly the same way.
Dictation tests can prove good predictors of global language ability even though same recent research has found that dictation tends to measure lower-order language skills such as straightforward comprehension rather than the higher-order skills such as inference(Cohen, 1980). The dictation of longer pieces of discourse (i.e. 7 to 10 words at a time) is recommended as being preferable to the dictation of shorter word groups (i.e. three to five words at a time) as in the traditional dictations of the past. Used in this way, dictation involves a dynamic process of analysis by synthesis, drawing on a learner’s ‘grammar of expectancy’ (Oller, 1972) and resulting in the constructive processing of the message heard.
If there is no close relationship between the sounds of a language and the symbols representing them, it may be possible to understand what is being spoken without being able to write it down. However, in English, where there is a fairly close relationship between the sounds and the spelling system, it is sometimes possible to recognise the individual sound elements without fully understanding the meaning of what is spoken. Indeed, some applied linguists and teachers argued that dictation encourages the students to focus his or her attention too much on the individual sounds rather than on the meaning of the text as a whole. Such concentration on single sound segments in itself is sufficient to impair the auditory memory span, thus making it difficult for the students to retain everything they hear.
When dictation is given, it is advisable to read through the whole dictation passage at approaching normal conversational speed first of all. Next, the teacher should begin to dictate (either once or twice) in meaningful units of sufficient length to challenge the student’s short-term memory span. (Some teachers mistakenly feel that they can make the dictation easier by reading out the text word by word: this procedure can be extremely harmful and only serves to increase the difficulty of the dictation by obscuring the meaning of each phrase.) finally, after dictation, the whole passage is read once more at slightly slower than normal speed.
4.
The communicative
approach
The communicative approach to language testing is sometimes linked to the integrative
approach. However, although both approaches emphasise the importance of the
meaning of utterances rather than their form and structure, there are
nevertheless fundamental differences between the two approaches. Communicative
tests are concerned primarily (if not totally) with how language is used in
communication. Consequently, most aim to incorporate tasks that approximate as
closely as possible to those facing the students in real life. Success is
judged in terms of the effectiveness of the communication, which takes place
rather than formal linguistic accuracy. Language 'use' (Widdowson, 1978) is often
emphasised to the exclusion of language ‘usage’. ‘Use’ is concerned with how people actually use language for a multitude
of different purposes while ‘usage’ concerns the formal patterns of language (described in prescriptive
grammars and lexicons). In practice, however, some tests of a communicative nature include the testing
of usage and also assess the ability to handle the formal patterns of the
target language. Indeed, few supporters of the communicative approach would
argue that communicative competence can ever be achieved without a considerable
mastery of the grammar of a language.
The attempt to measure different language skills in communicative tests is based on a view of language referred to as the divisibility hypothesis. Communicative testing results in an attempt to obtain different profiles of a learner’s performance in the language. The learner may, for example, have a poor ability in using the spoken language in informal conversations but may score quite highly on tests of reading comprehension. In this sense, communicative testing draws heavily on the recent work on aptitude testing (where it has long been claimed that the most successful tests are those which measure separately such relevant skills as the ability to translate news report, the ability to understand the radio broadcasts, or the ability to interpret speech utterances). The score obtained in a communicative test will thus result in several measures of proficiency rather than simply one overall measure.
Unlike the separate testing of skills in the structuralist approach, moreover, it is felt in communicative testing that sometimes the assessment of language skills in isolation may have only a very limited relevance to real life. For example, reading would rarely be undertaken solely for its own sake in academic study but rather for subsequent transfer to the information obtained to writing or speaking.
Perhaps the most important criterion for communicative tests is that they should be base on precise and detailed specifications of the needs of the learners for whom they are constructed: hence their particular suitability for the testing of English for specific purposes. However, it would be a mistake to assume that communicative testing is best limited to ESP or even to adult learners with particular obvious short-term goals. Although they may contain totally different task, communicative tests for young learners following general English courses are based on exactly the same principles as those for adult learners intending to enter on highly specialised courses of a professional or academic nature.
As pointed out in the beginning, a good test will frequently combine features of the communicative approach, the integrative approach and even the structuralist approach – depending on the particular purpose of the test and also on the various test constraints. If, for instance, the primary purpose of the test is for general placement purposes and there is very little time available for its administration, it may be necessary to administer simply a 50-item cloze test.
Language testing constantly involves making compromises between what is deal and what is practicible in a certain situation. Nevertheless this should not be used as an excuse for writing and administering poor tests: whatever the constraints of the situation, it is important to maintain ideals and goals, constantly trying to devise a test which is as valid and reliable as possible – and which has a useful backwash effect on the teaching and learning leading to the test.
Artikel lengkap bisa dibaca di buku Writing English Language Test (Heaton, 1990), download disini
Daftar hadir Topic 2: Approaches to Language Testing mata kuliah Language Evaluation disini
PAIRED SAMPLES T TEST Paired Samples t Test dikenal juga dengan sebutan Dependent Sample t Test . Seperti halnya Independent Samples...
