Split-Paper Testing: A Novel Approach to Evaluate Programming Performance

• Item type: Trans(1)
• 公開日: 2020-10-29
• タイトル: Split-Paper Testing: A Novel Approach to Evaluate Programming Performance
• タイトル / 言語: en
• タイトル: Split-Paper Testing: A Novel Approach to Evaluate Programming Performance
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: [研究論文] evaluating programming performance, constructed response tests, multiple choice tests, computer-based tests, split-paper tests
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_6501
• 資源タイプ: journal article
• 著者所属: University of Electro-Communications
• 著者所属: University of Electro-Communications
• 著者所属: University of Electro-Communications
• 著者所属(英): en
• 著者所属(英): University of Electro-Communications
• 著者所属(英): en
• 著者所属(英): University of Electro-Communications
• 著者所属(英): en
• 著者所属(英): University of Electro-Communications
• 著者名: Yasuichi, Nakayama; Yasushi, Kuno; Hiroyasu, Kakuda
• 著者名(英): Yasuichi, Nakayama; Yasushi, Kuno; Hiroyasu, Kakuda
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: There is a great need to evaluate and/or test programming performance. For this purpose, two schemes have been used. Constructed response (CR) tests let the examinee write programs on a blank sheet (or with a computer keyboard). This scheme can evaluate the programming performance. However, it is difficult to apply in a large volume because skilled human graders are required (automatic evaluation is attempted but not widely used yet). Multiple choice (MC) tests let the examinee choose the correct answer from a list (often corresponding to the “hidden” portion of a complete program). This scheme can be used in a large volume with computer-based testing or mark-sense cards. However, many teachers and researchers are suspicious in that a good score does not necessarily mean the ability to write programs from scratch. We propose a third method, split-paper (SP) testing. Our scheme splits a correct program into each of its lines, shuffles the lines, adds “wrong answer” lines, and prepends them with choice symbols. The examinee answers by using a list of choice symbols corresponding to the correct program, which can be easily graded automatically by using computers. In particular, we propose the use of edit distance (Levenshtein distance) in the scoring scheme, which seems to have affinity with the SP scheme. The research question is whether SP tests scored by using an edit-distance-based scoring scheme measure programming performance as do CR tests. Therefore, we conducted an experiment by using college programming classes with 60 students to compare SP tests against CR tests. As a result, SP and CR test scores are correlated for multiple settings, and the results were statistically significant. Therefore, we might conclude that SP tests with automatic scoring using edit distance are useful tools for evaluating the programming performance.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.28(2020) (online); ------------------------------
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: There is a great need to evaluate and/or test programming performance. For this purpose, two schemes have been used. Constructed response (CR) tests let the examinee write programs on a blank sheet (or with a computer keyboard). This scheme can evaluate the programming performance. However, it is difficult to apply in a large volume because skilled human graders are required (automatic evaluation is attempted but not widely used yet). Multiple choice (MC) tests let the examinee choose the correct answer from a list (often corresponding to the “hidden” portion of a complete program). This scheme can be used in a large volume with computer-based testing or mark-sense cards. However, many teachers and researchers are suspicious in that a good score does not necessarily mean the ability to write programs from scratch. We propose a third method, split-paper (SP) testing. Our scheme splits a correct program into each of its lines, shuffles the lines, adds “wrong answer” lines, and prepends them with choice symbols. The examinee answers by using a list of choice symbols corresponding to the correct program, which can be easily graded automatically by using computers. In particular, we propose the use of edit distance (Levenshtein distance) in the scoring scheme, which seems to have affinity with the SP scheme. The research question is whether SP tests scored by using an edit-distance-based scoring scheme measure programming performance as do CR tests. Therefore, we conducted an experiment by using college programming classes with 60 students to compare SP tests against CR tests. As a result, SP and CR test scores are correlated for multiple settings, and the results were statistically significant. Therefore, we might conclude that SP tests with automatic scoring using edit distance are useful tools for evaluating the programming performance.; ------------------------------; This is a preprint of an article intended for publication Journal of; Information Processing(JIP). This preprint should not be cited. This; article should be cited as: Journal of Information Processing Vol.28(2020) (online); ------------------------------
• 書誌レコードID / 収録物識別子タイプ: NCID
• 書誌レコードID / 収録物識別子: AA12697953
• 書誌情報: 情報処理学会論文誌教育とコンピュータ（TCE）; 巻 6, 号 3, 発行日 2020-10-29
• ISSN / 収録物識別子タイプ: ISSN
• ISSN / 収録物識別子: 2188-4234
• 出版者 / 言語: ja
• 出版者: 情報処理学会