Journal of Humanities, Arts and Social Science

ISSN Print: 2576-0556 Downloads: 326147 Total View: 2790170
Frequency: monthly ISSN Online: 2576-0548 CODEN: JHASAY
Email: jhass@hillpublisher.com
Article http://dx.doi.org/10.26855/jhass.2024.07.009

A Systematic Literature Review of the Effect of Test Methods on English Learners’ Reading Performance

Shuangmei Xie1,2, Charanjit Kaur Swaran Singh2,*, Wei Lun Wong2

1Panzhihua University, Panzhihua 617000, Sichuan, China.

2Faculty of Languages and Communications, Sultan Idris Education University, Tanjong Malim 35900, Perak, Malaysia.

*Corresponding author: Charanjit Kaur Swaran Singh

Published: July 31,2024

Abstract

This article examines the impact of response formats, the fourth category of the test method facets framework, on English learners’ reading performance using the PRISMA method for a systematic literature review. Data were collected from various sources, including Scopus, Google Scholar, and CNKI. A total of 45 articles were selected based on specific inclusion and exclusion criteria. The research findings indicate that different test methods have unique characteristics in assessing various aspects of reading comprehension proficiency. Furthermore, there is a significant difference in test takers’ performance based on the type of response format used for measuring reading comprehension. Additionally, testing methods may vary in effectiveness based on students’ proficiency levels. It is important to consider the validity and fairness of a test method when choosing one to ensure a more accurate evaluation of language competence. In conclusion, this review enhances understanding of the impact of test methods on English reading performance and encourages further research on the effects of test formats on reading comprehension in English.

References

Akhavan Masoumi, G., & Sadeghi, K. (2020). Impact of test format on vocabulary test performance of EFL learners: the role of gender. Language Testing in Asia, 10(1), 2. https://doi.org/10.1186/s40468-020-00099-x.

Alderson, J. C. (2011). Assessing Reading. Beijing: Foreign Language Teaching and Research Press.

Arshad, A., Shakir, A., & Ahmad, M. (2020). A Review on the Principles of a Reading Comprehension Test Construction to assess the Test Takers at different levels. Psychology and Education, 57(8), 1290-1302.

Bachman, L. F., & Palmer, A. S. (1999). Language testing in practice. Shanghai: Shanghai Foreign Language Teaching Press.

Bachman, L.F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

Bae, M., & Lee, B. (2018). Effects of Text Length and Question Type on Test-Takers’ Performance on Fill-in-the-Blank Items in Korean CSAT. English Teaching, 73(4), 149-174. https://doi.org/ 10.15858/engtea.73.4.201812.149.

Bernhardt, E. B., & Kamil, M. L. (2022). Conducting Second-Language Reading Research: A Methodological Guide. Routledge. 

Brantmeier, C. (2005). Effects of reader’s knowledge, text type, and test type on L1 and L2 reading comprehension in Spanish. The Modern Language Journal, 89(1), 37-53. https://doi.org/10.1111/j.0026-7902.2005.00264.x.

Brunfaut, T., & McCray, G. (2015). Looking into test takers’ cognitive processes while completing reading tasks: A mixed-method eye-tracking and stimulated recall study. London: British Council.

Burgers, C., Brugman, B. C., & Boeynaems, A. (2019). Systematic literature reviews: Four applications for interdisciplinary research. Journal of Pragmatics, 145, 102-109. https://doi.org/10.1016/j.pragma.2019.04.004.

Chen, J. (2006). Effects of Text Types and Response Formats on Proficiency EFL Reading Comprehension. Doctoral dissertation, Language Shanghai International Studies University, Shanghai, China.

Chen, J. L., & Shi, Y. L. (2007). Effects of Text Type and Test type on L2 Reading Comprehension Test Performance. Journal of PLA University of Foreign Languages, 30(4), 43-48.

Cheng, L., Klinger, D. A., & Zheng, Y. (2007). The challenges of the Ontario Secondary School Literacy Test for second language students. Language Testing, 24(2), 185-208. https://doi.org/10.1177/0265532207076363.

Cutting, L. E., & Scarborough, H. S. (2006). Prediction of reading comprehension: Relative contributions of word recognition, language proficiency, and other cognitive skills can depend on how comprehension is measured. Scientific studies of reading, 10(3), 277-299. https://doi.org/10.1207/s1532799xssr1003_5.

Dolosic, H. (2018). An examination of self-assessment and interconnected facets of second language reading. Reading in a Foreign Language (RFL), Volume 30, No. 2. 189–208. http://hdl.handle.net/10125/66921.

Elias Sheybani1, Mitra Zeraatpishe. (2018). On the Dimensionality of Reading Comprehension Tests Composed of Text Comprehension Items and Cloze Test Items. International Journal of Language Testing, Vol. 8, No. 1, March.

Ghahari, S., & Ahmadinejad, S. (2016). Operationalization of Bachman’s model via a multimodal reading comprehension test: Screening test method facets and testees’ characteristics. Studies in Educational Evaluation, 51, 67-76. https://doi.org/10.1016/j.stueduc.2016.09.005.

Ghazal. Akhavan Masoumi., & Karim Sadeghi. (2020). Impact of test format on vocabulary test performance of EFL learners: the role of gender. Language Testing in Asia, 10:2. https://doi.org/10.1186/s40468-020-00099-x.

Gholam-Reza Abbasian., & Hanieh Zadsar. (2019). Relationship between Test Item Arrangements and Testee’s Performance and Test Usefulness Criteria. International Journal of English Language Teaching, Vol. 6, No. 2.

https://doi.org/10.5430/ijelt.v6n2p52.

Hassani, L., & Maasum, T. N. R. T. M. (2012). A study of students’ reading performance in two test formats of summary writing and open-ended questions. Procedia-Social and Behavioral Sciences, 69, 915-923. 

https://doi.org/10.1016/j.sbspro.2012.12.016.

Heaton J.B. (2004). Writing English Language Tests. Beijing: Foreign Language Teaching and Research Press.

Huang, X. X. (2006). Effects of Different Formats on Reading Comprehension Test Performance. Master dissertation, Shanghai Jiaotong University, Shanghai, China. 

Hughes, A. & Hughes, J. (2020). Testing for Language Teachers. Cambridge: Cambridge University Press.

Kastner, M., & Stangla, B. (2011). Multiple choice and constructed response tests: Do test format and scoring matter? Procedia-Social and Behavioral Sciences, 12, 263-273. https://doi.org/10.1016/j.sbspro.2011.02.035.

Khoshsima, H., & Pourjam, F. (2014). A comparative study on the effects of cloze tests and open-ended questions on reading comprehension of Iranian intermediate EFL learners. International Journal on Studies in English Language and Literature (IJSELL), 2(7), 17-27.

Kobayashi, M. (2002). Method effects on reading comprehension test performance: Text organization and response format. Language testing, 19(2), 193-220. 

https://doi.org/10.1191/0265532202lt227oa.

Kobayashi, M. (2004). Investigation of test method effects: text organization and response format: a response to Chen, 2004. Language Testing, 21(2), 235-244. 

https://doi.org/10.1191/0265532204lt282xx.

Kong, J.F. (2019). Investigating the role of test methods in testing reading comprehension. Singapore: Springer. https://doi.org/10.1007/978-981-13-7021-2.

Leila Hassani, Tengku Nor Rizan, & Tengku Mohamad Maasum. (2012). A Study of Students’ Reading Performance in Two Test Formats of Summary Writing and Open-ended Questions. Social and Behavioral Sciences, 69, 915-923. https://doi.org/10.1016/j.sbspro.2012.12.016.

Li, H. J. (2018). A Comparative Study of Two Parallel Reading Comprehension Tests. In 2018 4th International Conference on Social Science and Higher Education (ICSSHE 2018) (pp. 269-273). Atlantis Press. https://doi.org/10.2991/icsshe-18.2018.66.

Lin X.Y. (2022). An Analysis of Reading Comprehension in Comprehensive English Course (Philosophy and Civilization) Test. US-China Foreign Language, Vol. 20, No. 2, 55-62. https://doi.org/10.17265/1539-8080/2022.02.003.

Liu, F. (2009). The effect of three test methods on reading comprehension: An experiment. Asian Social Science, 5(6), 147-153.

Liu, Q.H. (2003). A Preliminary Study on Two Reading Test Formats-MC & SAQ. Master dissertation, Donghua University, Shanghai, China.

McCray, G., & Brunfaut, T. (2018). Exploring the construct validity of a multiple-choice reading comprehension test: An analysis of the relationship between test-takers’ cognitive and metacognitive strategy-use and test performance. Language Testing, 35(1), 3-26.

Osterlind, S. (2002). Constructing Test Items: Multiple-Choice, Constructed Response, Performance and Other Formats. NY: Kluwer Academic Publishers.

Ranjbaran, F., & Alavi, S. M. (2017). Developing a reading comprehension test for cognitive diagnostic assessment: A RUM analysis. Studies in Educational Evaluation, 55, 167-179. https://doi.org/10.1016/j.stueduc.2017.10.007.

Rezaei, O., Barati, H., & Youhanaee, M. (2012). The Effect of Content Familiarity and Test Format on Iranian EFL Test Takers’ Performance on Test of Reading Comprehension. International Journal of Applied Linguistics and English Literature, 1(4), 1-14. https://doi.org/10.7575/ijalel.v.1n.4p.1.

Rodriguez, M. C. (2003). Construct equivalence of multiple‐choice and constructed‐response items: A random effects synthesis of correlations. Journal of Educational Measurement, 40(2), 163-184.

Roediger III, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5), 1155. 

https://doi.org/10.1037/0278-7393.31.5.1155.

Rupp, A. A., Ferne, T., & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the construct: A cognitive processing perspective. Language testing, 23(4), 441-474. 

https://doi.org/10.1191/0265532206lt337oa.

Samson, D.M.M. (1983). “Rasch and Reading” in J. van Weeren (ed.): Practice and problems in language testing. Arnhem: CITO.

Sheybani, E., & Zeraatpishe, M. (2018). On the dimensionality of reading comprehension tests composed of text comprehension items and cloze test items. International Journal of Language Testing, 8(1), 12-26.

Shi, J. (2005). A Comparative Study of Three Different types of Cloze Test in English Reading Comprehension Tests. Master dissertation, Qinghua University, Beijing, China.

Shima Ghahari & Sima Ahmadinejad. (2016). Operationalization of Bachman’s model via a multimodal reading comprehension test: Screening test method facets and testees’ characteristics. Studies in Educational Evaluation, 51, 67-76.

http://dx.doi.org/10.18823/asiatefl.2018.15.3.6.649.

Shohamy, E. (1984). Does the testing method make a difference? The case of reading comprehension. Language Testing, 1(2), 147-170. 

https://doi.org/10.1177/026553228400100203.

Wang, H. (2006). The validity components of multiple-choice and short questions in reading tests. College English (Academic Edition), (1), 320-324.

Wang, X. N. (2009). Effects of Response Formats and Item types on EFL Reading Comprehension Test Performance: An Empirical Study on First-year English Majors of Beijing City University. Master dissertation, Beijing City University, Beijing, China.

Xiao, Y., & Watson, M. (2017). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93-112. https://doi.org/10.1177/0739456x17723971.

Zhou, Q. (2019). The Feasibility of Measuring Reading Ability by the Format of Short Answer Questions. US-China Foreign Language, 17(1), 1-9. https://doi.org/10.17265/1539-8080/2019.01.001.

How to cite this paper

A Systematic Literature Review of the Effect of Test Methods on English Learners' Reading Performance

How to cite this paper: Shuangmei Xie, Charanjit Kaur Swaran Singh, Wei Lun Wong. (2024) A Systematic Literature Review of the Effect of Test Methods on English Learners' Reading Performance. Journal of Humanities, Arts and Social Science8(7), 1591-1599.

DOI: http://dx.doi.org/10.26855/jhass.2024.07.009