A survey of test papers used at secondary schools in Thai Nguyen, Vietnam

Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên A SURVEY OF TEST PAPERS USED AT SECONDARY SCHOOLS IN THAI NGUYEN, VIETNAM Duong Duc Minh Thai Nguyen University of Technology, TNU ABSTRACT The purpose of this article is to overview teaching strategies based on the communicative approaches to second language teaching with the focus on testing methods that are related to communicative language teaching in order to discuss advantages and disadvantages of these methods. The study also examined in details the contents of test papers which were randomly collected from secondary school teachers in Thai Nguyen province to support data for the analysis. Although communicative testing is considered to be focused not only on what the learner knows about the second language and about how to use it but also on the learner’s ability to actually demonstrate this knowledge in real-life situations, the fact of language testing shows what is not like theory. As the result, the findings of the analysis will be used to make practical suggestions for the testing procedure to ensure successful learning objectives. Key words: Communicative approaches; Test papers designs; Language Testing.  INTRODUCTION Vietnam's Ministry of Education and Training (MOET) issued a mandate 7984/BGDĐT- GDTrH on 1st September 2008 with the aim to improve the general system of education and training. In the context of modern trends in education in the country, apart from experimenting with new teaching methods, more and more secondary schools in Thai Nguyen have found and applied new types of tests with a view of fostering better achievements. However, nowadays, in secondary schools in Thai Nguyen, testing is used as the only means to evaluate students’ language proficiency, and many problems exist in the way the test papers are made. In this article the techniques that are used in designing test papers by teachers in secondary schools in Thai Nguyen will be surveyed. The focus of the article will be on problematic issues of testing methods. LITERATURE REVIEW Communicative approaches to second language teaching and testing. According to Munby [10], Bachman [1] and Richards and Rodgers [12], there exist three main approaches to second language teaching: (1) grammatical approach, (2) communicative approach, and (3) situational  Duong Duc Minh, Tel: 0280-375-0061, Email: duongducminh@tnut.edu.vn approach. However, since there seems to be a sufficient overlap in objectives between situational approaches and communicative approaches, in this study these two approaches will be subsumed under the communicative approach. To discuss the terms “competence” and “performance”, Berns *2+ defines “competence” and “performance” as follows: “competence is the speaker-hearer’s knowledge of his language and performance is the actual use of language in concrete situation. In fact, it (performance) obviously could not directly reflect competence”. Hymes [9] and Celce-Murcia [5], in their studies, propose a broader notion of competence, namely communicative competence. Communicative competence should be distinguished from communicative performance, which is the realization of these competences and their interactions in the actual production and comprehension of utterances. It is important to take into account differences between the basic concepts of communicative approaches in second language teaching and testing. Canale and Swain *4+ state that “if a communicative approach to second language teaching is adopted then principles of syllabus design must integrate aspects of both grammatical competence and sociolinguistic competence. Furthermore, teaching methodology and Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên assessment instrument must be designed so as to address not only communicative performance but also communicative competence”. To conclude, all these theories are important for designing successful models of foreign language teaching. The theories of communicative competence have contributed to our understanding of language acquisition. Understanding the concepts of communicative competence and applying them to language teaching and testing is hard work, but benefits of language learners are well worth the time invested. Interrelated Problems of Teaching and Testing Nowadays, many language teachers feel skeptical about tests and of testers. It cannot be denied that a great deal of language testing is of poor quality. Hughes [8] states that “too often language tests have a harmful effect on teaching and learning and too often they fail to measure accurately whatever they are supposed to measure”. Some problems related to teaching and testing are discussed below. Confusion between teaching and testing. According to Hughes [8], most teachers use tests that are published in reference books. In class, copies of such tests are distributed and students are asked to do various assignments in these tests. It may easily cause problems as such teaching/testing does not help students to receive knowledge and develop skills in learning; it just helps students to know “how to do a test”. Teaching, but not testing what has been taught. The teaching process requires various types of tests depending, amongst other things, on the objectives of the courses and purposes of tests. “Different purposes will usually require different kinds of test” (Hughes [8]). However, in reality, teachers do not always take care of the main principles of testing and use tests that are more a quiz than a proper test. As a result, such “tests” do not test what teachers want to test. The lack of unity between teaching and testing. For example, testing is not suitable for the given teaching process. The content of testing and the aims of teaching do not accord. Testing as a servant of teaching. “It is the controversial argument that the good test is an obedient servant since it follows and apes the teaching” (Davies *6+). It is true that there may be occasions when teaching is good and appropriate whereas testing is not. Equally, there may be occasions when teaching is poor and inappropriate whereas testing is able to exert a beneficial influence. Teaching as a servant of testing. According to Hughes [8], there is a tendency that heightens the effects of feedback in testing to the teaching process. Teachers just test the language ability of their students so that students tend to have a mental habit of study their teachers teach, furthermore, students do not attach their importance to train their skills in order to be their skills. The result is that the consideration of teaching as a servant of testing and students’ true ability is not always reflected in the test score that they obtain (Hughes [8]). Harmful backwash. Backwash is known as the effect of testing on teaching and learning processes. So, the test contents and testing techniques are at variance with the objectives of the course. The reason, as Hughes *8+ states, “is caused of the inaccurate tests which are used of inappropriate techniques and lack of reliability”. It may be concluded that the proper relationship between teaching and testing is that of partnership. Human factor is of special importance in testing. Our tests should focus on the learning itself, not on the outcome of learning. PRINCIPLES OF DESIGNING A LANGUAGE TEST Up to now, it is clear that we need a detailed list of all the targets of the teaching process when we design a test. In fact, it is only in theory, not in practice. One of the biggest obstacles to overcome in constructing an adequate test is the acknowledgement of workable criteria. Hence, needed for constructing a test. Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên Principles of practicality In George’s *7+ opinion, “no matter how good our test design is if our test is not practical, then it is useless”. Teachers should be able to make a clear and useful interpretation of testing data in order to understand their students better. A test that is too complex or too sophisticated may be of no use for teachers. Principles of reliability Nghiem *11+ defines the term ‘reliability’ as “the measurement of the dissimilar between the real score and the score that candidate should have” (my translation). George *7+ also states that a reliable test is a test that is consistent and dependable. It is intended to answer the following questions “Does the test test anything at all?” and “Are the results consistent or are they haphazard over time and from one student to others” (George [7]). Principles of validity By far the most complex criterion of a good test is validity, the degree to which the test actually measures what it is intended to measure. In language testing, validity is supported most convincingly by subsequent personal observations of teachers and peers. As with reliability, there are different types of validity, such as, “face, content, construct domain and criterion related validity” (James and Stephen 2007: 66). Principles of beneficial backwash As stated by Hughes*8+, ‘Backwash’ is the benefit that tests offer to learning. When students take a test, they should be able, within a reasonably short period of time, to utilize information about their competence that test feedback offers. Their wrong responses may become windows of insight into their further work. Appraisal of strong sides as well as constructive criticism of weaknesses should be expressed. Principles of elicitation techniques Knowing about elicitation techniques may help to determine the purpose of the test that we design in order to have a good language testing. Elicitation techniques are all mentioned by Hughes [8], Ur [13] and Brown [3]. Generally, they include: Questions and answers, True / False, Multiple-choice, Gap-filling and completion, Matching, Dictation, Cloze, Translation, Rewriting, Translation, Essay and Monologue. Language testers have to try out, research and experiment with a wide variety of different types of tests in order to find out which tests are valid, reliable, easy to score, and appropriate for testing language skills. In other words, all types of tests are important as all of them have their strong and weak points. Analysis of English tests in secondary schools in Thai Nguyen The focus of this study is on tests performed by pupils at non-specialized secondary schools. Most of them have been learning English for 4 years at junior secondary schools. So their knowledge of English, as well as their testing skills, is compared to the pre-intermediate level. Almost all teachers who have provided their test papers for the present study are aged from 27 to above 40. They have been teaching English for at least 5 years. The size of their classes varies from 40 to 50 students, and each teacher has 3 or 4 such classes so that one teacher is responsible for the learning results of more than 100 students. At the same time, the main testing material that is available for these teachers is their own experiences gained from books on teaching methodology and reference books that are widely sold in bookshops, namely “Objective testing for secondary school pupils” (2 volumes) printed by Education Publishing House, “English exercises 10, 11, 12” by VNU Press, “Practice Tests (Book 1, 2)” by Modern Academic Center. Contents of test papers In the National Education System of Vietnam, there are three kinds of written tests practiced in secondary schools; these are ‘15 minute-long test’, ‘45 minute-long tests’ and ‘final exams’. Only the last two kinds of tests in 2008 were collected for the purpose of this study. All in all, 285 different test papers Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên designed by secondary school’s teachers for different pupils have been collected and the statistical survey of these tests is presented in Table 1 below: Table 1. Types of test papers collected in secondary schools in Thai Nguyen 45 minute Final exams 10 th form 70 36 11 th form 65 34 12 th form 57 23 Total 192 93 Figure 1 shows the language skills tested in 285 test papers that have been gathered from the language teachers in Thai Nguyen. As can be seen from the graph, there is an unbalanced proportion of testing skills. 235 out of 285 test papers tested grammar, (197/285) vocabulary, (158/285) writing, and (163/285) reading comprehension, whereas only a small number of tests focused on phonetics (28/285) and even fewer on translation (7/285). Neither listening nor speaking skills were tested. This problem leads pupil not to have an overall variety of skills in language learning. There exists problem about testing techniques. There is no difference in the distribution of testing techniques across 45 minute tests and final tests. All the techniques in test papers collected are presented in Figure 2 below. As shown in Figure 2, the majority of testing techniques in use are multiple-choice exercises and fill-in-the-blank items; cloze and other techniques are applied in a small number of test papers. Thus, in this study I will mainly discuss the use of multiple-choice items as this is the most popular testing technique. In all test papers collected, there is a tendency to use multiple-choice techniques by the majority of the teachers (261 out of 285). There is no denial that multiple-choice exercises have many advantages as they check whether pupils have learned facts and routine procedures since these tests require one correct answer. However, some items may have two reasonable answer options. Furthermore, it is possible to get multiple- choice items correct without knowing much or doing any real thinking. As answers are in front of the student, some people call these tests “multiple-guess”. In sum, multiple choice items are used the most as they are an inexpensive and efficient way to check factual “declarative” knowledge (narrate what they have learnt) and routine procedures. However, they are not useful for assessing critical thinking within a tested subject, the ability to write, or the ability to apply knowledge to solve problems. An analysis of a typical test paper: a case study The typical sample test chosen for this case- study is presented in Appendix I; it was collected from the 11th form in a secondary school. My general impression is that this test is poorly structured and lacks validity, reliability and communicative perspective. The test is poorly structured as it does not have oral instruction from teachers, what skills it intends to measure and what points Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên the pupils would get for each correct responses. The time allocated for each task is not stated. One has no idea whether pupils had enough time to perform all the tasks. Only the total time allotted to perform all the tasks is mentioned. Furthermore, it does not have separate sections, such as vocabulary, grammar, writing, or reading. Only Part II is specified as “Reading skill”. Besides, most instructions are not given clearly or directly. For instance, in part I: 2, 3 Viết lại câu dùng đại từ quan hệ (Rewite these sentences using Relative pronouns), but in fact, there are not any relative pronouns in the instruction. The content of the test seems to cover all that is expected from a test of this type. Grammar and syntax are tested in questions 11, 15, 17, 18, 20, 21, 26 and 30, vocabulary in questions 1, 4, 8, 16, 19, 22, 23, 24, 25 and 27, phonetics in questions 9, 12, 28 and 29, reading comprehension in part II. The content of the testing sentences covers the content input but these sentences do not test language usage, the ability to use the language appropriately as well as the ability to operate in real-life situations. Furthermore, testing techniques applied in this test are not variable enough. 25 out of 40 questions are multiple choice questions, 5 questions are True / False questions and 10 are writing questions. Another problem with this test is that test items should be representative of what it is intended to measure and evaluate. However, teachers often borrow tests from other books but do not pay attention to the content of these tests. For instance, in Part I questions 2, 3, 5, 6, 10 contain the task Rewrite this sentence using relative pronoun. If pupils give correct answers, teachers may conclude that pupils have learnt all the relative pronouns. In fact, all the sentences in the test only require two relative pronouns who and what. Questions 11, 13, 18, 20, 21 which suggest the choice between To-infinitive and Present/Past participle have the same problem. On the other hand, all these questions claim to test pupils’ writing skills, but in reality these tasks do not require such skills. Moreover, all the questions from 1 to 30 are single-sentence tasks, as the result, these sentences extensively cannot test pupils’ ability to communicate in English for example: 4. Dinosaurs became million years ago A. extinct B. to extinct C. extincted D. extinction 19. Geothermal heat is only in a few places in the world. A. limited B. clean C. available D. safe The Reading part claims to test reading comprehension skill, but the passage chosen for this purpose is too academic and difficult for pupils in 11th form; it is not likely to be relevant for the pupils of this level. The following tasks Write True/False before each sentence and Choose A, B, C or D to fill in the blank are in the same category as multiple- choice questions. Also, questions 1-10 from Part II focus on recognition but not production, which means that these questions focus on the learning itself, not on the outcome of learning. All in all, an overview of the test leaves the impression of a collage. Different test items were photocopied from different books and stuck on the paper; and on the top of the page the name of the pupil, class, and course are given. This may be a practical way of making a test, but it is impractical when test results are considered as they lack content validity. With all the problems mentioned above, it may be concluded that this test does not have a proper content and constructive validity; it is not reliable, practical and authentic. Therefore, it cannot measure and evaluate what it intends to measure and evaluate. CONCLUSION The findings of this study have sought to shed some light on a number of problems pertaining to the process of testing. The case- study of the typical test paper has demonstrated that tests typically used in Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên many communicative classes are basically designed to evaluate grammatical competence. Such tests are often incorporated in course books and reference materials and are designed to assess the knowledge of grammatical structures, vocabulary and syntax. No doubt, the most commonly used test of this type is multiple- choice. Even though this kind of testing presents an ideal platform for objective grading, and is simple to create, it is in no way a communicative way of testing language skills. Good language testing is an essential tool in creating effective language teaching programs. The more valid and reliable the test is, the better it will be in assessing pupils’ abilities and in placing them accurately in a respective program. The test should test the learner in a variety of language skills. Although it is difficult to meet all these criteria, the teachers should be sensitive to such criteria. The Common European Framework of Reference (CEFR) may be recommended as one of the best reference sources to all teachers and education leaders within our national education system. REFERENCES Primary sources Collection of 285 test papers of teachers at secondary schools in Thai Nguyen, Vietnam. From January to August 2008. Secondary sources [1].Bachman, L.F. 1990. Fundamental Considerations in Language Testing. Oxford University Press. Oxford. [2]. Berns, M. S. 1990. Context of competence. Springer. 30 [3]. Brown, H. D. 1998. Teaching by Principles: An Interactive Approach to Language Pedagogy. Englewood Cliffs, New Jersey Prentice Hall Regents. [4]. Canale, M. and Swain, M. 1980. Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics. Vol.1 No.1. 1-47. [5]. Celce-Murcia, M. 1995 Rethinking the Role of Communicative Competence in Language Teaching. In eds. E. A. Soler and M. R. S. Jorda. Intercultural Language Use and Language Learning. Springer. 2008. 41-77 [6]. Davies, A. 1990. Principles of Language Testing. Basil Blackwell. Oxford. 5 [7]. George, M. J. 2003. Understanding and Implementing the CLT. RELC Journal 34. 5-30. [8]. Hughes, A. 1989. Testing for Language Teacher. Cambridge University Press. Cambridge. [9]. Hymes, D. 1972. On Communicative Competence. In eds. J.B. Pride and J. Homes, Sociolinguistics. Harmondsworth: Penguin. 254-294. [10]. Munby, J. 1981. Communicative Syllabus Design. Cambridge University Press. Cambridge. [11] Nghiem, Xuan Hung. (dị ch). 1995. Trắc nghiệm và đo lường cơ bản trong giáo dục. NXB Đại học Quốc gia. Hà Nội. [12]. Richards, J.C. and Rodgers, T. S. 2001. Approaches and Methods in Language Teaching Cambridge University Press. Cambridge. [13]. Ur, P. 1996. A Course in Language Teaching. Cambridge University Press. C ambridge. Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên TÓM TẮT KHẢO SÁT VỀ THỰC TRẠNG KIỂM TRA, ĐÁNH GIÁ MÔN TIẾNG ANH TẠI CÁC TRƯỜNG THPT TRÊN ĐỊA BÀN TỈNH THÁI NGUYÊN Dương Đức Minh Trường Đại học Kỹ thuật Công nghiệp Thái Nguyên, Đại học Thái Nguyên Bài báo trình bày kết quả nghiên cứu về thực trạng công tác kiểm tra, đánh giá môn tiếng Anh theo đường hướng giao tiếp tại các trường THPT trên đị a bàn tỉ nh Thái Nguyên. Thông tin, dữ liệu cho bài báo được thu thập, tổng hợp từ các bài kiểm tra của học sinh tại các trường THPT của tỉ nh. Kết quả của nghiên cứu cho thấy, mặc dù xu hướng dạy học theo đường hướng giao tiếp, tích cực hóa vai trò của người học đã được phổ biến và áp dụng trong các trường THPT và đặc biệt là trong quá trình dạy và học bộ môn tiếng Anh, tuy nhiên công tác kiểm tra đánh giá môn tiếng Anh còn phần nhiều sử dụng các thủ thuật truyền thống, nặng về kiểm tra ngữ pháp, khả năng đọc hiểu mà ít chú trọng đến kỹ năng giao tiếp của học sinh. Bên cạnh đó, nghiên cứu cũng chỉ ra rằng các cách ra bài tập kiểm tra cũng còn đơn điệu, chưa phong phú, hầu hết mới dừng lại ở các bài tập “nhiều lựa chọn” (multiple-choice). Từ khóa: Phương pháp giao tiếp, thiết kế bài kiểm tra, dạy ngoại ngữ. Appendix 1: Samples of test papers collected in Thai Nguyen secondary schools.  Duong Duc Minh, Tel: 0280-375-0061, Email: duongducminh@tnut.edu.vn Dương Đức Minh Tạp chí KHOA HỌC & CÔNG NGHỆ 59(11): 88 - 94 Số hóa bởi Trung tâm Học liệu – Đại học Thái Nguyên