Description:

Adaptive, Scalable, and Quality‐Oriented Solutions for Developing Educational Assessments through Human‐AI Collaboration

Assessments are foundational to education, shaping how teachers teach, how students learn, and how accountability is maintained across the educational ecosystem. Yet, creating and delivering high‐quality assessments remains one of the most time‐consuming aspects of teaching. Educators, especially those in under‐resourced schools, often lack the time, training, and support to design effective assessments that are aligned with curricula and diverse student needs. While recent advances in large language models (LLMs) show the potential to reduce this burden, they also raise concerns about quality and teacher trust. In specific domains such as data visualization education, researchers and educators face similar challenges: existing assessments for measuring visualization literacy (the ability to read and interpret visualizations) are lengthy, rigid, and difficult to scale or adapt. Across both general and visualization‐specific contexts, improving efficiency, maintaining quality, and building trust in assessment development and delivery requires systems that lower technical barriers, promote content reuse and adaptation, and integrate meaningfully into educators’ existing workflows. This dissertation addresses these challenges through three complementary projects that develop adaptive, scalable, quality‐oriented solutions for assessment development. The first project introduces A‐VLAT and A‐CALVı, computerized adaptive tests for visualization literacy that maintain precision and reliability while halving test length. The second project presents VILA, a pipeline that leverages LLMs to generate visualization literacy questions at scale, combining automated generation with expert‐driven evaluation to ensure quality and validity. The third project extends these ideas to general K–12 education, where we codesigned a web‐based system, Ripplet, that supports multilevel reusable interactions with LLMs for creating, editing, and adapting assessments. Together, these projects contribute to the development of assessment systems that improve efficiency, scalability, and quality.