Personalized and adaptive assessment: Placing the stakes

Compared to personalized and adaptive learning, the distinction between personalized and adaptive assessment is less contested, but perhaps also less widely discussed. As before, my definition will hinge upon the role of human decisionmaking in distinguishing between adaptive (machine-intelligent) and personalized (machine- and human-intelligent) assessment.

Most understand adaptive assessment in the context of computer–based adaptive testing, which adapts the parameters of the test to student performance in real time. Acing a series of questions may earn more challenging questions, while fumbling on several may elicit easier questions or increased assistance.

In a slightly different perspective on adaptive assessment, the BLE Group suggests that formative and summative (benchmark) assessments “measure whether an academic standard has been learned,” while adaptive assessments “measure growth and identify where the student is on the learning-ladder continuum,” and diagnostic assessments “determine missing skills and remediate them.” I see these as overlapping rather than distinct categories. Adaptive testing is already widespread in summative assessment. Further, questions may be adapted to discriminate between students or to measure mastery of key concepts and skills at any point along an expected learning trajectory, whether a prerequisite or endpoint.

How personalization goes beyond adaptivity in assessment is in explicitly incorporating the decisions of the persons involved in the process, the primary stakeholders being the learner, the instructor-grader, and the external audience interpreting the grade. I will focus on just these three roles as the simplest case, although other configurations may include separate roles for instructor and grader, peer grading, and group assessment.

Learners differ in the goals they have for their education, both in what they hope to learn and in what they will be expected to present as documentation of that learning. Some careers require a particular degree or certificate, while others may solicit work portfolios or barrage candidates with tricky interview questions. Some students seek a general liberal-arts education rather than job-specific training, while others may simply want to broaden their knowledge without regard for the mark received for it. The initial choices are left up to the learner, and the information sought is determined by the audience for the assessment (i.e., the employer, certifying organization, or society). Thus, tailoring what gets assessed and how results are presented around those expectations would entail personalization rather than adaptivity.

How to present assessment information to learners and instructors may also vary depending on their preferences and abilities for interpreting and responding to such information. In some cases, these factors may be adapted based on evaluations of their actual behaviors (e.g., a learner who disengages after seeing comparisons against peers). In other cases, users may have access to better information or more sophisticated responses than the adaptive system, and an appropriately personalized system would allow them to choose their action based on that information. Examples include a learner getting distracted upon trying to interpret very-frequent feedback (with the system failing to distinguish loss of focus from intent, productive concentration), or an instructor recognizing when personal contact would help. Again, personalization builds in opportunities for human intervention to take over when the adaptive system is less suited for the task.

While these distinctions may not seem that significant, highlighting them here enables more precision in examining the criticisms of personalized and adaptive learning. Many limitations apply specifically to adaptive learning systems that do not leave enough room for individual choice or personal interaction. Adopting a fairly broad view allows us to focus on the possibilities and constraints regarding where these developments can go, not just the shortcomings of what some particular instantiations have done so far.

If assessments are diagnoses, what are the prescriptions?

I happen to like statistics. I appreciate qualitative observations, too– data of all sorts can be deeply illuminating. But I also believe that the most important part of interpreting them is understanding what they do and don’t measure. And in terms of policy, it’s important to consider what one will do with the data once collected, organized, analyzed, and interpreted. What do the data tell us that we didn’t know before? Now that we have this knowledge, how will we apply it to achieve the desired change?

In an eloquent, impassioned open letter to President Obama, Education Secretary Arne Duncan, Bill Gates and other billionaires pouring investments into business-driven education reforms (revised version at Washington Post), elementary teacher and literacy coach Peggy Robertson argues that all these standardized tests don’t give her more information than what she already knew from observing her students directly. She also argues that the money that would go toward administering all these tests would be better spent on basic resources such as stocking school libraries with books for the students and reducing poverty.

She doesn’t go so far as to question the current most-talked-about proposals for using those test data: performance-based pay, tenure, and firing decisions. But I will. I can think of a much more immediate and important use for the streams of data many are proposing on educational outcomes and processes: Use them to improve teachers’ professional development, not just to evaluate, reward and punish them.

Simply put, teachers deserve formative assessment too.