Application of Artificial Intelligence to Assessment

Edited by:
Hong Jiao, University of Maryland
Robert W. Lissitz, University of Maryland

A volume in the series: The MARCES Book Series. Editor(s): Hong Jiao, University of Maryland. Robert W. Lissitz, University of Maryland.

Published 2020

The general theme of this book is to present the applications of artificial intelligence (AI) in test development. In particular, this book includes research and successful examples of using AI technology in automated item generation, automated test assembly, automated scoring, and computerized adaptive testing. By utilizing artificial intelligence, the efficiency of item development, test form construction, test delivery, and scoring could be dramatically increased.

Chapters on automated item generation offer different perspectives related to generating a large number of items with controlled psychometric properties including the latest development of using machine learning methods. Automated scoring is illustrated for different types of assessments such as speaking and writing from both methodological aspects and practical considerations. Further, automated test assembly is elaborated for the conventional linear tests from both classical test theory and item response theory perspectives. Item pool design and assembly for the linear-on-the-fly tests elaborates more complications in practice when test security is a big concern. Finally, several chapters focus on computerized adaptive testing (CAT) at either item or module levels. CAT is further illustrated as an effective approach to increasing test-takers’ engagement in testing.

In summary, the book includes both theoretical, methodological, and applied research and practices that serve as the foundation for future development. These chapters provide illustrations of efforts to automate the process of test development. While some of these automation processes have become common practices such as automated test assembly, automated scoring, and computerized adaptive testing, some others such as automated item generation calls for more research and exploration. When new AI methods are emerging and evolving, it is expected that researchers can expand and improve the methods for automating different steps in test development to enhance the automation features and practitioners can adopt quality automation procedures to improve assessment practices.

Augmented Intelligence and the Future of Item Development, Mark J. Gierl, Hollis Lai, and Donna Matovinovic. Reconceptualizing Items: From Clones and Automatic Item Generation to Task Model Families, Richard Luecht and Matthew Burke. Artificial Intelligence for Scoring Oral Reading Fluency, Jared Bernstein, Jian Cheng, Jennifer Balogh, and Ryan Downey. Natural Language Processing and the Literacy Challenge, Jill Burstein. Practical Considerations for Using AI models in Automated Scoring of Writing, Peter W. Foltz. Item Pool Design and Assembly: The State of the Art, Jeffrey M. Patton and Ray Y. Yan. Automated Test Assembly: Case Studies in Classical Test Theory and Item Response Theory, Siang Chee Chuah, Donovan Hare, Luz Bay, and Thomas Proctor. Multistage Testing in Practice, Duanli Yan. An Intelligent CAT That Can Deal With Disengaged Test Taking, Steven L. Wise. Differences in the Amount of Adaptation Exhibited by Various Computerized Adaptive Testing Designs, Mark D. Reckase, Unhee Ju, and Sewon Kim. Automatic Item Generation With Machine Learning Techniques: A Pathway to Intelligent Assessments, Jaehwa Choi. About the Editors.