Is adaptivity a qualitative or quantitative problem?

One common criticism of adaptive learning is that by tailoring instruction so closely to students’ needs, it doesn’t challenge them enough. As embodied by James Paul Gee’s critique:

People who never confront challenge and frustration, who never acquire new styles of learning, and who never face failure squarely may in the end become impoverished humans. They may become forever stuck with who they are now, never growing and transforming, because they never face new experiences that have not been customized to their current needs and desires.

While I agree with the dangers of what he describes, I question the causal attribution.

First, adaptive learning systems that indulge in too much customization may instead be guilty of relying on a too-narrow prescription for the student’s “zone of proximal development (ZPD)”. Individualized learning does not require giving only incremental steps; it can (and should) include more ambitious steps to occasionally challenge students, perhaps just beyond their conventional ZPD (or at the limits of their ZPD when defined by “lots of help”). Students need to struggle—manageably—as part of their learning. Adapting to students’ needs can include optimizing the nature and amount of that struggle based on past experiences and future expectations.

Second, this can also be overcome by building a certain amount of variability into the system, for the sake of both the students and the system. Occasionally presenting students with problems that may or may not lie within their ZPD can help them learn “what to do when you don’t know what to do” (in the words of a dear colleague of mine, Joe Wise). Whether framed as desirable difficulties, germane cognitive load, preparation for future learning, or the development of adaptive expertise rather than just routine expertise, unexpected challenges can offer invaluable learning opportunities. Further, adaptive learning systems need to reach beyond what is already known in order to improve themselves. A truly intelligent system should be discovering new knowledge about its particular learners and even about learning in general. The possible paths a student might take are infinite, and the system’s designers don’t know what’s best—only what tends to be better compared to other paths that have already been examined. That is, adaptive learning must itself be an adaptive learner.

Both of these issues point to a quantitative problem due to adapting too narrowly or too often. The deeper question is whether adaptivity is a fundamental, qualitative problem: Does having any adaptivity at all invite complacency among students accustomed to having their learning experiences at least partly tailored to their needs? Given the well-established importance of scaffolding instruction according to students’ needs, I would argue that adaptive learning is a valuable tool not simply for accelerating but also for enriching instruction.

Personalized instruction: The other half of personalized learning

As I have explained in a previous post on personalized learning, an important dimension along which personalized learning goes beyond merely adaptive learning is to personalize the experience on the instructional side, not just the learner side. Amidst all the excitement about adaptive learning, teachers remain an often-forgotten yet crucial part of the equation. Well-designed personalization takes advantage of the human intelligence embedded in expert instructors, including opportunities for them to exercise their professional judgment in deciding which activities will work best for their students given their particular contexts and constraints.

This EdSurge report mentions Rocketship’s upcoming changes, as “New model attempts to bring teachers closer to students’ online learning experience” by returning some classroom control back to the teacher:

Rocketship’s new model will shift focus from running purely adaptive programs, to using programs that give teachers greater control over content that gets assigned.

What this highlights is the need for the design of personalized learning programs to identify when to allocate decisions to teachers (possibly with recommendations among which to choose) and when to adapt the students’ learning experience immediately, without need for waiting for additional human input. While this depends in part on the professional knowledge of the instructors implementing the system, some decisions may be straightforward or simple enough to automate. Decisions best left to expert human intervention are likely to be more complex, to depend on more contingencies, to require interpersonal contact, or to have more uncertainty in their effectiveness. Where that balance lies is subject to continual readjustment, but since there are always unknowns and since social interaction is fundamental to the human experience, there will always remain a need for personalization.

Empathizing with actions above appearance

In two previous posts, I emphasized the importance of encouraging children in general and girls in particular to value actions above appearance, noting that I would especially want to promote perspective-taking, self-regulation, and cognitive flexibility rather than looks. “Fit and Feminist” blogger Caitlin echoes this sentiment:

[W]hat your body looks like is not as important as what it can do.

She objects to the disconnect between achievement and the appearance of achievement, with regard to how we judge physical fitness:

When a person has a lean body, it serves as visual shorthand of sorts, indicating that the person most likely trains hard and who [sic] has excellent nutrition… You can’t see a person’s 1RM or their 5K PR, but you can see their visible abs, you know?

Imagery carries an immediacy that surpasses words and numbers, even when crafted into a compelling narrative. We process images much more quickly and viscerally than we process stories, and we have less experience critiquing pictures than we do analyzing and arguing using language.

Images themselves strip away context, removing the periphery and exaggerating the impact of anything contained within the frame. Still images also remove temporal context, inviting us to fill in but allowing us to forget what might have happened before and after the photo. In today’s highly connected and media-saturated world, we archive and recirculate images capturing unusual moments, someone’s “best of” achievement rather than the ordinary everyday which we mentally discard. Instead we create unrealistic expectations based on this visual vocabulary in which only the rare and easily perceived is worth remembering and emulating:

[W]hen we hold up ultra-leanness as The Fitness Goal for recreational athletes like myself as well as people who are just trying to keep themselves healthy, we are basically saying that everyone should be held to the same standards as elite athletes. This is insane! In what other area of our lives are we expected to emulate the best of the best? Are we all expected to write Pulitzer Prize winning novels? Must we all be capable of singing like the angelic offspring of Mariah Carey and Whitney Houston? Should we all be able to engineer the tools necessary to identify the Higgs-Boson particle? No! So why does this idea persist that says we must all have the bodies of Olympic athletes before we can be considered fit and healthy?

Even trickier is our tendency to try to identify with the subject of a photo, mapping ourselves onto the person we see (or imagine). As Paul Bloom notes, people are influenced by appearance and perceived similarity to themselves when judging competence, culpability, and worth:

People are understandably empathetic toward the victims of crime, particularly when they are young and vulnerable, when they are attractive, and when they share our race or ethnicity.

[W]hen the victim of a crime is attractive, the defendant tends to get a longer prison sentence; if the defendant is attractive, he or she gets a lighter sentence.

[B]aby-faced individuals also tend to get lighter punishments, perhaps because they inspire parental warmth.

[J]udging someone based on the geometry of his features is, from a moral and legal standpoint, no better than judging him based on the color of his skin. Actually, both biases reflect the parochial and irrational nature of empathy.

Bloom also highlights our tendency to selectively empathize with people on one side of a conflict while disregarding the other, and with identifiable individuals rather than anonymous numbers:

Typically, political disputes involve a disagreement over whom we should empathize with.

Too often, our concern for specific individuals today means neglecting crises that will harm countless people in the future.

Appearances are seductive, readily merging with our imagined selves and crowding out invisible others.

What we ought to do instead is maintain a healthy separation between ourselves and the targets of our empathy. Parenting expert Janet Lansbury continually highlights the value of empathizing with young children, as distinct from simply identifying with them. It requires acknowledging others’ feelings while also remaining separate from them.

The more you are willing to agree with your child’s feelings while calmly holding on to the boundary, the easier it will be for her to release her resistance and move on.

They need to be able to complain, resist, stomp their feet, cry, express their darker feelings with the assurance that they have our acceptance and acknowledgment. They need to know that they have a leader who will help them to comply with rules and boundaries in the face of their No’s, and not be intimidated by their displeasure and disagreement.

Is my attitude toward my baby’s fussing or crying one of curiosity rather than impatience and assumption?

Am I soothing my baby by understanding and meeting her needs, or shushing, jiggling and stifling her because I want the crying to stop?

Am I following my impulse to calm my child by saying, for example, “You’re okay”? Or am I staying connected and centered by acknowledging her feelings: “You bumped into the table. Ouch, that hurt you!”

Am I hurrying the feelings along, or waiting patiently for them to be fully released?

Our capacity for empathy needs to go beyond thinking of others as replicas or extensions of ourselves, to recognizing that they are distinct from ourselves. We can acknowledge the reality and legitimacy of others’ feelings without assuming responsibility for changing those feelings or giving in to their demands. Instead of rushing to shelter or console innocent and adorable babies—focusing on their obvious appearance of vulnerability and cuteness—we observe their actions to better understand their particular goals and needs, which may not be the same as ours. Empathy is not ownership.

By maintaining this difference in perspective, both between appearance and action and between ourselves and others, we marry empathy to reason and allow space for incorporating multiple angles and unknowns in our empathy calculus.

MOOCsperiments: How should we assign credit for their success and failure?

That San Jose State University’s Udacity project is on “pause” due to comparatively low completion rates is understandably big news for a big venture.

We ourselves should take pause to ponder what this means, not just regarding MOOCs in particular, but regarding how to enable effective learning more broadly. The key questions we need to consider are whether the low completion rates come from the massive scale, the online-only modality, the open enrollment, some combination thereof, or extraneous factors in how the courses were implemented. That is, are MOOCs fundamentally problematic? How can we apply these lessons to future educational innovation?

Both SJSU and Udacity have pointed to the difficulties of hasty deployment and starting with at-risk students. In an interview with MIT Review, Thrun credits certificates and student services with helping to boost completion rates in recent pilots, while the inflexible course length can impede some students’ completion. None of these are inherent to the MOOC model, however; face-to-face and hybrid settings experience the same challenges.

As Thrun also points out, online courses offer some access advantages for students who face geographic hurdles in attending traditional institutions. Yet in their present form, they only partly take advantage of the temporal freedom they can potentially provide. While deadlines and time limits may help to forestall indefinite procrastination and to maintain a sense of shared experience, they also interfere with realizing the “anytime, anywhere” vision of education that is so often promoted.

But the second half of “easy come, easy go” online access makes persistence harder. Especially in combination with massive-scale participation that exacerbates student anonymity, no one notices if you’re absent or falling behind. While improved student services may help, there remain undeveloped opportunities for changing the model of student interaction to ramp up the role of the person, requiring more meaningful contributions and individual feedback. In drawing from a larger pool of students who can interact across space and time, massive online education has great untapped potential for pioneering novel models for cohorting and socially-situated learning.

Online learning also can harness the benefits of AI in rapidly aggregating and analyzing student data, where such data are digitally available, and adapting instruction accordingly. This comes at the cost of either providing learning experiences in digital format, or converting the data to digital format. This is a fundamental tension which all computer-delivered education must continually revisit, as technologies and analytical methods change, as access to equipment and network infrastructure changes, and as interaction patterns change.

The challenges of open enrollment, particularly at massive scale, replay the recurring debates about homogeneous tracking and ability-grouping. This is another area ripe for development, since students’ different prior knowledge, backgrounds, preferences, abilities, and goals all influence their learning, yet they benefit from some heterogeneity. Here, the great variability in what can happen exaggerates its importance: compare the consequences of throwing together random collections of people without much support, vs. constraining group formation by certain limits on homogeneity and heterogeneity and instituting productive interaction norms.

As we all continue to explore better methods for facilitating learning, we should be alert to the distinction between integral and incidental factors that hinder progress.

Expensive assessment

One metric for evaluating automated scoring is to compare it against human scoring. For some domains and test formats (e.g., multiple-choice items on factual knowledge), automation has an accepted advantage in objectivity and reliability, although whether such questions assess meaningful understanding is often debated. With more open-ended domains and designs, human reading is typically considered superior, allowing room for individual nuance to shine through and get recognized.

Yet this exposé of some professional scorers’ experience reveals how even that cherished human judgment can get distorted and devalued. Here, narrow rubrics, mandated consistency, and expectations of bell curves valued sameness over subtlety and efficiency over reflection. In essence, such simplistic algorithms resulted in reverse-engineering cookie-cutter essays that all had to fit one of their six categories, differing details be damned.

Individual algorithms and procedures for assessing tests need to be improved so that they can make better use of a broader base of information. So does a system which relies so heavily on particular assessments that the impact of their weaknesses can get magnified so greatly. Teachers and schools collect a wealth of assessment data all the time; better mechanisms for aggregating and analyzing these data can extract more informational value from them and decrease the disproportionate weight on testing factories. When designed well, algorithms and automated tools for assessment can enhance human judgment rather than reducing it to an arbitrary bin-sorting exercise.