magazinelogo

Translation and Foreign Language Learning

ISSN Online: 3069-0315 ISSN Print: 3070-3077 CODEN:
Frequency: monthly Email: tfll@hillpublish.com
Total View: 429732 Downloads: 110768 Citations: 8 (From Dimensions)
ArticleInterdisciplinary Studies of Translation http://dx.doi.org/10.26855/tfll.2026.02.020

Investigating Prompt Effects on Large Language Model Translation Behaviour in Literary Texts: Evidence from Luotuo Xiangzi

Xiao Zhang

School of Humanities and Law, North China University of Technology, Beijing 100144, China.

*Corresponding author: Xiao Zhang

This research was supported by the R&D Program of Beijing Municipal Education Commission (Grant No. SM202310009003).
Published: February 28, 2026

Abstract

Large language models (LLMs) have been increasingly applied to translation practices, raising the question of whether and to what extent prompt design affects LLMs’ performance. Using selected excerpts from Lao She’s Luotuo Xiangzi as the corpus and employing evaluation metrics including BLEU, METEOR, and TER, this study compares the outputs generated by LLMs under different prompts with two published human translations by Shi Xiaojing and Howard Goldblatt. Two LLMs (ChatGPT and DeepSeek) were tested under three prompt conditions (baseline, style-oriented, and culture- and pragmatics-oriented). The findings reveal that prompt effects are non-linear and model-dependent; DeepSeek generally outperformed ChatGPT, but ChatGPT showed stronger responsiveness to detailed prompts. The study also finds that different translation challenges (culture-specific expressions, colloquialisms, figurative language, pragmatic/stylistic features) respond differently to prompts, suggesting task-specific rather than universal prompt strategies. The paper concludes with implications for human–AI collaborative literary translation by highlighting the role of prompt design in optimizing LLM-generated initial drafts.

Keyword

Beijing-style literary translation; large language models; prompt engineering; initial translation quality; human–AI collaborative translation

References

Bu, Z. L. (Ed.). (2004). History of modern Chinese literature. Wuhan University Press.

Gao, Y., Wang, R. L., & Hou, F. (2023). How to design translation prompts for ChatGPT: An empirical study. arXiv. https://doi.org/10.48550/arXiv.2304.02182

Ge, S., & Wang, N. (2024). Literary translation in the era of artificial intelligence: Challenges and opportunities. Foreign Languages and Their Teaching, (1), 94-101+149-150.

Geng, F., & Hu, J. (2023). New direction for post-editing by artificial intelligence translation: A case study of ChatGPT translation. Foreign Languages in China, 20(3), 41-47.

Han, H. J., & Jiang, Y. (2016). A corpus-based comparison of general language features of human translation and online machine translation. Foreign Language Education, 37(5), 102-106.

Han, Z. M., & Chai, T. D. (2024). Knowledge translation competence of artificial intelligence: A case study of literary anecdotes. Technology Enhanced Foreign Language Education, (5), 3-10+105.

Karpinska, M., & Iyyer, M. (2023). Large language models effectively leverage document-level context for literary translation, but critical errors persist. In Proceedings of the eighth conference on machine translation (pp. 419-451). Association for Computational Linguistics.

Li, F. Q. (2022). A comparative study on the quality of English-Chinese translation between translation learners and a machine translation system in the era of artificial intelligence. Foreign Language World, (4), 72-79.

Li, F. Q., Zhang, Y., & Ding, L. J. (2025). C-E legal translation quality comparison: Large language models vs. neural machine translation systems. Shanghai Journal of Translators, (6), 62-67.

Li, Y. F., Yin, Y. J., Li, J., et al. (2022). Prompt-driven neural machine translation. In Findings of the Association for Computational Linguistics: ACL 2022 (pp. 2759-2790). Association for Computational Linguistics.

Velásquez-Henao, J. D., Franco-Cardona, C. J., & Cadavid-Higuita, L. (2023). Prompt engineering: A methodology for optimizing interactions with AI-language models in the field of engineering. DYNA, 90(230), 9-17.

Wang, J. Q., & Wen, Q. F. (2010). Review and implications of the existing automated scoring system—From the perspective of the computer-assisted scoring model of Chinese EFL learners' translation. Foreign Language World, (1), 75-81+91.

Wang, J. S., Zhuang, C. X., & Wei, Y. P. (2024). Machine translation quality assessment: Methods, applications, and outlook. Foreign Languages and Literature, (3), 135-144.

Wen, X., & Tian, Y. L. (2024). The effectiveness of ChatGPT in translating China-specific discourse text. Shanghai Journal of Translators, 40(2), 27-34+94-95.

Wu, M. H., Xu, J. H., Yuan, Y. L., et al. (2025). (Perhaps) beyond human translation: Harnessing multi-agent collaboration for translating ultra-long literary texts. Transactions of the Association for Computational Linguistics, 13, 901-922.

Wu, Y. J., & Hu, G. (2023). Exploring prompt engineering with GPT language models for document-level machine translation: Insights and findings. In Proceedings of the eighth conference on machine translation (pp. 166-169). Asso-ciation for Computational Linguistics.

Yu, L. (2024). Lexical diversity and syntactic complexity in ChatGPT translation. Foreign Language Teaching and Research, 56(2), 297-307+321.

Zhang, S. K., & Zhao, C. Y. (2025). Examining the suitability of large language models for literary translation: A multi-indicator assessment of the English translations of Biancheng. Foreign Languages in China, 22(4), 85-95.

Zhao, Y., Zhang, H., & Yang, Y. C. (2024). Comparative study on the translation quality of large language models—Taking the translation of "Fan Hua" as an example. Technology Enhanced Foreign Language Education, (4), 60-66+109.

Zhu, Y. F., & Zhang, J. D. (2025). Human and machine translation styles in the English versions of The Three-Body Problem: A comparative analysis of Ken Liu, GPT-4, and VolcTrans. Technology Enhanced Foreign Language Education, (4), 88-96+113.

Copyright

© 2026 by the author(s).
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) license, which permits non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and is not modified or adapted.
https://creativecommons.org/licenses/by-nc-nd/4.0/

How to cite this paper

Investigating Prompt Effects on Large Language Model Translation Behaviour in Literary Texts: Evidence from Luotuo Xiangzi

How to cite this paper: Xiao Zhang. (2026). Investigating Prompt Effects on Large Language Model Translation Behaviour in Literary Texts: Evidence from Luotuo Xiangzi. Translation and Foreign Language Learning2(2), 313-321.

DOI: http://dx.doi.org/10.26855/tfll.2026.02.020