Knowledge Assessment Through Tests—Semantic Approach

Abstract

Knowledge assessment through tests is an objective and effective technology, widely used in modern education. Tests (multiple-choice tests), consisting of dichotomous items—a question with one correct and one or more incorrect answers (distractors) are widely used in modern education to assess students’ knowledge. Test developers face the problem of converting the number of correct answers into a numerical grade representing the knowledge of the assessed. A numerical grade is a point in the numerical interval—a scale of grades. Usually, the number of correct answers is converted into a grade based on the evaluators' inner sense of fair evaluation. In the present work, a model for knowledge evaluation through dichotomous tests is proposed, based on the so-called Semantic branch of Information Theory. A critical opinion is given for the Classical Test Theory and the modern Item Response Theory as tools for knowledge assessment. Some concepts in these theories leave the feeling that it could be desired more concerning assessing knowledge through these theories. A new, entirely different approach to knowledge assessment by tests is proposed in the paper. In the proposed information model for knowledge assessment, the process of knowledge assessment is considered as an information process with information transfer. The information is generated by a source (the assessed), which has a goal—to get as close as possible to the error-free solution of the test. The information in the form of an information signal (the answers to the test that the assessed gives) is directed to the recipient—the assessor. The assessor evaluates the value (importance) of this information signal, which is a measure of the knowledge of the assessed. The value of the information signal is measured by the progress of the assessed toward reaching the goal. Formulas were obtained, linking the value of the information signal with a numerical grade of knowledge of the assessed. In particular, evaluation formulas are derived for tests of the most used types—with items with 3, 4, and 5 answers (the example with the scale of grades used in Bulgaria). However, detailed assessment requires answering a large number of items (items bank, included in the test at the stage of development), which increases the time for the examination. The examination time could be shrunken with an adequate algorithm that reduces an item's number included in the exam according to the answers of the assessed, without deteriorating the quality of the examination and assessment. An adaptive algorithm of knowledge assessment is proposed, based on analytical expressions, which can be integrated into computer tests to shorten the examination process by reducing the number of items asked, depending on the previous answers of the assessed. The adaptive algorithm reduces the number of items that the assessed answers, compared to the number of items in the bank. The grade that the assessed receives for his/her knowledge of the examined topic differs from the "exact" grade (that he/she would receive after solving a test with all items in the bank) with a value not exceeding a given toler-ance. The grade is calculated from: 1. the number of items in the items bank, 2. the number of items the assessed has answered, which are a part of all items in the items bank, and 3. the relative number of correct answers.