Flipped instruction and self-regulated learning

From Robert Talbert’s “Flipped learning skepticism: Can students really learn on their own?“:

[Students’] primary experience is with pedagogy that emphasizes dependence. They are brainwashed through years of instructor-centered pedagogy that they are helpless when it comes to learning. But the fact remains that as human beings, they retain the capacity to learn and to regulate their learning. It’s just difficult and takes time and patience.

I’ve long believed that the most important quality which a good teacher must possess is the belief that all students can learn. I’ve also come to believe that self-directed learning is a valuable skill which must be explicitly nurtured and sustained from early childhood education on. As Talbert notes, we all start with it, but it can fade quickly in discouraging environments.


Letting children choose promotes prosocial behavior

As described in “Giving Preschoolers Choice Increases Sharing Behavior“:

[S]haring when given a difficult choice leads children to see themselves in a new, more beneficent light. Perceiving themselves as people who like to share makes them more likely to act in a prosocial manner in the future.

Previous research has shown that this idea — as described by the over-justification effect — explains why rewarding children for sharing can backfire. Children come to perceive themselves as people who don’t like to share since they had to be rewarded for doing so. Because they don’t view themselves as “sharers” they are less likely to share in the future.

Developmental psychologists Nadia Chernyak and Tamar Kushnir found that compared to children who were given a non-costly choice or who were required to share, preschoolers given a costly choice were more likely to share again at a subsequent opportunity.

My thoughts:

  1. I would be interested in an analysis comparing the effect of the conditions on the children who did not share– that is, collecting baseline data on children’s initial propensity to share, and then comparing how the interventions affected them across the range of initial tendencies.
  2. I wonder how well this would apply to long-term planning and diligence (e.g., completing homework, practicing a difficult skill, doing chores).

The results do suggest that choice can be a powerful mechanism for promoting positive habits and attitudes, something which I think parents and schools could harness more productively. That choice can potentially foster empathy and perspective-taking is very encouraging.

Full reference: N. Chernyak, T. Kushnir. Giving Preschoolers Choice Increases Sharing Behavior. Psychological Science, 2013; DOI: 10.1177/0956797613482335

Other research-based commentary on “tiger parenting”

For what I hope will be my last post on the subject, I wanted to share some gems I’ve found from my online meanderings following link after link on Amy Chua’s views on parenting. These all draw from relevant research to critique specific practices rather than an imprecise “parenting style.”

1. In this edited interview transcript with Scientific American, Temple University developmental psychologist Laurence Steinberg reviews the literature on many of the specific practices Chua describes, pointing out both the good and the bad. On his “good” list are high expectations, parental involvement, and positive feedback for genuine accomplishment (but not cultivating false self-esteem). On his “bad” list are excessive punishment, being overly restrictive, and squelching autonomy (characteristics of authoritarian rather than authoritative parenting). He further questions Chua’s views on desirable goals for her children and highlights the value of unstructured play for children’s development. Although he mentions cultural differences in parenting and acknowledges that Americans might misperceive Chinese parenting as being more authoritarian than it really is, he doesn’t analyze cultural influences in much depth here.

2. On Parenting Science, Gwen Dewar (an interdisciplinary social scientist whose background also includes psychology) provides a fuller analysis on both the authoritarian / authoritative parenting style dimension and the cultural differences between Chinese and American parenting. Like Steinberg and others, she too affirms the importance of believing in effort over innate ability, noting that this characterizes Chinese more than American values. Ironically, she includes more detail than Steinberg on his own research, describing the potential for positive peer pressure among Chinese-American youth, whose peers encourage them to achieve rather than rejecting them for geekiness. Most thankfully, she highlights that the positive aspects of traditional Chinese parenting can be separated from undesirable authoritarian practices.

3. Finally, on the NY Times Freakonomics blog, Yale professor of law and economics Ian Ayres (who acknowledges being a friend and colleague of Amy Chua’s) delves into the cognitive benefits of some of these parenting practices, rather than their developmental or cultural consequences. While I’m disappointed that he discusses the benefits of “tiger parenting” without strong caveats against its harms (or an acknowledgment that they can be separated), I particularly appreciated his economic analysis of the attitudes and behaviors that may result.

One virtue Ayres extols is delayed gratification, which he quantifies as “the intertemporal marginal rate of substitution, the willingness to forego current consumption in order to consume more in the future.” It’s another lens on the importance of grit, perseverance, and conscientiousness in enduring challenges while pursuing distant goals. He points out research indicating that these skills may be a stronger predictor of future success than intelligence (as measured by IQ).

(Although Ayres didn’t mention it, this further validates Dweck’s research on the importance of believing that effort matters more than innate ability in determining success.)

He also cites Ericsson’s research on the amount of effort necessary to develop expertise. Despite believing that such discipline is likely to transfer over to other pursuits, he admits that he would probably choose skills with more immediate benefits:

My personal bias is in guiding my children toward endeavors (like learning statistics or US History or corporate finance or Python — all subjects of daddy school) that I think are likely to pay higher direct adult dividends than music or sport skills that atrophy in adulthood.

Inclined though I am to agree with him, I wonder how effective such pursuits are as targets for kids to develop discipline and expertise. Aside from the value of music and sports in themselves, they also carry salient milestones—some culturally derived (such as soccer tournaments and numbered Suzuki-method books), but others perceptually evident—that help children self-assess progress. Computer programming has concrete markers of success in getting a program to produce the desired output, but growth in using analytical tools such as statistics is a bit harder for a youngster to perceive and appreciate.

Oddly, Ayres portrays Chua’s methods as an effective “taking choice off the table” technique for building discipline, despite this crucial difference between his parenting approach and hers: he explicitly involved his children in the initial choice process and explained the pros and cons of their choices, whereas Chua imposed her choices on her children. That initial commitment by the child is key. Without it, the child is simply following rules divorced from meaning. With it, the child learns to connect desire with dedication and goal with process.

Collectively, these articles echo the fundamental values of attributing success to effort, nurturing intrinsic motivation, and setting high expectations that I summarized earlier, while also adding a richer perspective on cultural differences, peer pressure, and delayed gratification in promoting perseverance. While these articles haven’t and won’t receive the audience that brazen storytelling attracts, they give voice to relevant research that too often whispers quietly from the archives.

Diligence vs. competitiveness

I continue to be disappointed by the mainstream media’s coverage of the “tiger parenting” phenomenon. Although there’s now somewhat more discussion of the specific parenting practices that are good or bad for the child, distinct practices are still getting conflated. One of the latest that I’ve seen claims that “the intense emphasis on hard work comes with a deep, obsessive competitiveness.”

It ain’t necessarily so.

Setting aside my frustration with the media’s reliance on first-person anecdotes rather than research on aggregate populations, I’ll first acknowledge that emphasizing the value of hard work and discipline is indeed very productive. As I wrote in my previous post, it takes about ten years of deliberate practice to attain expertise[1], and those who believe in the value of effort are more likely to invest further effort[2][3] . A recently reported twin study similarly notes that children with greater self-control do better in school and have better health, financial, and social outcomes as adults[4].

But deliberate practice isn’t just rote practice, and worthwhile effort isn’t merely hours of exhaustion. Both require thoughtfulness in figuring out what needs more work and how to tackle it. Likewise, developing self-control requires more than simply being placed in a constraining environment. As I’ve previously noted, the complex and ill-structured world of imaginative play can improve children’s impulse control and self-regulation[5]. Much like the riddle about the town with two barbers, “just because it looks like what you want doesn’t mean it will produce what you want.” Here, it’s not enough just to put children and students through their paces in rigid settings that prevent them from going astray. The theme underlying all of these phenomena is that people need to learn how to decide for themselves how to manage their efforts and how to improve.

So how does this relate to competitiveness? Competition is based on social comparison, or norm-referenced assessment. While it may be inspiring to see the accomplishments of peers as a possibility for oneself, it can also be limiting to endeavor only to best them and not more generally to excel. Even more damaging, people have no control over the performance of their rivals, and chasing this uncontrollable target can foster that worrisome learned helplessness which can dampen enthusiasm, lower self-esteem, and inhibit future effort[6][7]. Instead, criterion-referenced assessment measures performance against fixed goals which can be set up as benchmarks and eventually internalized by the learner. Not only does this provide a clearer way of measuring success than normative comparisons, but it also helps the budding athlete, musician, or student succeed better in the long run.

I fully recognize that Chinese parents can be obsessively competitive about their children’s achievements, even through their self-effacing denials that their children are anything special. And it’s already been documented that the Chinese culture places a high premium on hard work. But these aren’t uniquely Chinese values, and they can be decoupled. The result isn’t some kind of compromise or blend between Chinese and American philosophies, but a selection of those beliefs and practices that have been shown to be productive across cultures.

[1] Ericsson, K.A. (1996). The acquisition of expert performance: An introduction to some of the issues. In K. A. Ericsson (Ed.), The Road to Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games (pp. 1-50). Mahwah, NJ: Lawrence Erlbaum Associates.
[2] Mueller, C.M., & Dweck, C.S. (1998). Praise for intelligence can undermine children’s motivation and performance. Journal of Personality and Social Psychology, 75, 33-52.
[3] Dweck, C.M. (2006). Mindset: The New Psychology of Success. New York NY: Random House.
[4] Moffitt, T.E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R.J., Harrington, H.L., Houts, R., Poulton, R., Roberts, B.W., Ross, S., Sears, M.R., Thomson, W.M., & Caspi, A. (in press). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences.
[5] Bodrova, E.B., & Leong, D.J. (2003). The importance of being playful. Educational Leadership, 60, 50-53.
[6] Diener, C.I., & Dweck, C.S. (1978). An analysis of learned helplessness: Continuous changes in performance, strategy, and achievement cognitions following failure. Journal of Personality and Social Psychology, 36, 451-462.
[7] Diener, C.I., & Dweck, C.S. (1980). An analysis of learned helplessness: II. The processing of success. Journal of Personality and Social Psychology, 39, 940-952.

Using student evaluations to measure teaching effectiveness

I came across a fascinating discussion on the use of student evaluations to measure teaching effectiveness upon following this Observational Epidemiology blog post by Mark, a statistical consultant. The original paper by Scott Carrell and James West uses value-added modeling to estimate teachers’ contributions to students’ grades in introductory courses and in subsequent courses, then analyzes the relationship between those contributions and student evaluations. (An ungated version of the paper is also available.) Key conclusions are:

Student evaluations are positively correlated with contemporaneous professor value‐added and negatively correlated with follow‐on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value‐added in follow‐on courses).

We find that less experienced and less qualified professors produce students who perform significantly better in the contemporaneous course being taught, whereas more experienced and highly qualified professors produce students who perform better in the follow‐on related curriculum.

Not having closely followed the research on this, I’ll simply note some key comments from other blogs.

Direct examination:

Several have posted links that suggest an endorsement of this paper’s conclusion, such as George Mason University professor of economics Tyler Cowen, Harvard professor of economics Greg Mankiw, and Northwestern professor of managerial economics Sandeep Baliga. Michael Bishop, a contributor to Permutations (“official blog of the Mathematical Sociology Section of the American Sociological Association“), provides some more detail in his analysis:

In my post on Babcock’s and Marks’ research, I touched on the possible unintended consequences of student evaluations of professors.  This paper gives new reasons for concern (not to mention much additional evidence, e.g. that physical attractiveness strongly boosts student evaluations).

That said, the scary thing is that even with random assignment, rich data, and careful analysis there are multiple, quite different, explanations.

The obvious first possibility is that inexperienced professors, (perhaps under pressure to get good teaching evaluations) focus strictly on teaching students what they need to know for good grades.  More experienced professors teach a broader curriculum, the benefits of which you might take on faith but needn’t because their students do better in the follow-up course!

After citing this alternative explanation from the authors:

Students of low value added professors in the introductory course may increase effort in follow-on courses to help “erase” their lower than expected grade in the introductory course.

Bishop also notes that motivating students to invest more effort in future courses would be a desirable effect of good professors as well. (But how to distinguish between “good” and “bad” methods for producing this motivation isn’t obvious.)


Others critique the article and defend the usefulness of student evaluations with observations that provoke further fascinating discussions.

Andrew Gelman, Columbia professor of statistics and political science, expresses skepticism about the claims:

Carrell and West estimate that the effects of instructors on performance in the follow-on class is as large as the effects on the class they’re teaching. This seems hard to believe, and it seems central enough to their story that I don’t know what to think about everything else in the paper.

At Education Sector, Forrest Hinton expresses strong reservations about the conclusions and the methods:

If you’re like me, you are utterly perplexed by a system that would mostly determine the quality of a Calculus I instructor by students’ performance in a Calculus II or aeronautical engineering course taught by a different instructor, while discounting students’ mastery of Calculus I concepts.

The trouble with complex value-added models, like the one used in this report, is that the number of people who have the technical skills necessary to participate in the debate and critique process is very limited—mostly to academics themselves, who have their own special interests.

Jeff Ely, Northwestern professor of economics, objects to the authors’ interpretation of their results:

I don’t see any way the authors have ruled out the following equally plausible explanation for the statistical findings.  First, students are targeting a GPA.  If I am an outstanding teacher and they do unusually well in my class they don’t need to spend as much effort in their next class as those who had lousy teachers, did poorly this time around, and have some catching up to do next time.  Second, students recognize when they are being taught by an outstanding teacher and they give him good evaluations.

In agreement, Ed Dolan, an economist who was also for ten years “a teacher and administrator in a graduate business program that did not have tenure,” comments on Jeff Ely’s blog:

I reject the hypothesis that students give high evaluations to instructors who dumb down their courses, teach to the test, grade high, and joke a lot in class. On the contrary, they resent such teachers because they are not getting their money’s worth. I observed a positive correlation between overall evaluation scores and a key evaluation-form item that indicated that the course required more work than average. Informal conversations with students known to be serious tended to confirm the formal evaluation scores.


Dean Eckles, PhD candidate at Stanford’s CHIMe lab offers this response to Andrew Gelman’s blog post (linked above):

Students like doing well on tests etc. This happens when the teacher is either easier (either through making evaluations easier or teaching more directly to the test) or more effective.

Conditioning on this outcome, is conditioning on a collider that introduces a negative dependence between teacher quality and other factors affecting student satisfaction (e.g., how easy they are).

From Jeff Ely’s blog, a comment by Brian Moore raises this critical question:

“Second, students recognize when they are being taught by an outstanding teacher and they give him good evaluations.”

Do we know this for sure? Perhaps they know when they have an outstanding teacher, but by definition, those are relatively few.

Closing thoughts:

These discussions raise many key questions, namely:

  • how to measure good teaching;
  • tensions between short-term and long-term assessment and evaluation[1];
  • how well students’ grades measure learning, and how grades impact their perception of learning;
  • the relationship between learning, motivation, and affect (satisfaction);
  • but perhaps most deeply, the question of student metacognition.

The anecdotal comments others have provided about how students respond on evaluations are more fairly couched in the terms “some students.” Given the considerable variability among students, interpreting student evaluations needs to account for those individual differences in teasing out the actual teaching and learning that underlie self-reported perceptions. Buried within those evaluations may be a valuable signal masked by a lot of noise– or more problematically, multiple signals that cancel and drown each other out.

[1] For example, see this review of research demonstrating that training which produces better short-term performance can produce worse long-term learning:
Schmidt, R.A., & Bjork, R.A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.

Dealing with the “scientific impotence” excuse

On “Five minutes with the discoverer of the Scientific Impotence Excuse“:

When people are faced with scientific research that clashes with their personal view, they invoke a range of strategies to discount the findings. They will often judge that the topic at hand is not amenable to scientific enquiry [and embrace] the general idea that some topics are beyond the reach of science.

Anyone who seeks to educate, inform, or influence, take note of these techniques to avoid backfire or unwarranted discounting:

  1. Affirm people’s values first.
  2. Frame findings to be consistent with their values.
  3. Present findings in non-threatening ways.
  4. Speak with humility.
  5. Say “discover” instead of “disagree”.
  6. Decrease in-group/out-group salience.
  7. Provide an alternate target for negative emotions.
  8. Teach critical thinking and metacognition in safe settings.

What I really appreciated was the research-based guidance for how to address this resistance to scientific evidence, in the second section of the interview (as summarized above). Misunderstanding the distinction between evidence and belief contributes to the problem, but it may not be so obvious how to highlight that distinction productively. As Munro pointed out, Cohen, Aronson, and Steele’s (2000) research demonstrates one way to resolve this tension, as does some of his own research (which unfortunately didn’t get cited directly in the interview). I think this is an extremely important set of findings because it’s so tempting for people to come down hard on those who “just don’t understand,” lecturing authoritatively and perhaps conveying frustration or even attacking their perspectives.  Unfortunately, that can backfire. Instead, this research shows that a gentler approach can actually be more effective. I take heart in that.

Difficulties of accommodating discrepant information

On “The Wrong Stuff – Reasonable Doubt: Innocence Project Co-Founder Peter Neufeld on Being Wrong“:

I think generally speaking it’s difficult for people to admit they’re wrong, and the higher the stakes, the more difficult it becomes. So what you really want to do is educate people that it’s OK to be wrong. It doesn’t mean you’re a fool. It’s not going to be the end of your life.

There are high social costs to being wrong, and creating a culture that values thoughtfulness and humility rather than tenacity may alleviate this phenomenon. (Ironically, one might expect this to be worse in a collectivist culture, where there could be more shame, surprise, or negative attention attached to retracting publicly stated beliefs. In contrast, individualistic cultures that celebrate different ideas might be more tolerant of changing one’s mind.)

But I think there are high cognitive and metacognitive costs to being wrong as well. Part of it could be a consequence of generating a hypothesis or belief, akin to the dangers of convincing oneself of the correctness of a guess (e.g., when taking a pretest). The more a person articulates or mentally rehearses an idea, the more s/he becomes committed to it (i.e., strengthens the memory trace, elaborates on potential explanations, draws connections to prior knowledge).

Further, someone whose self-concept is strongly linked to having the right answers might feel more threatened by realizing s/he made an error. And someone who thinks that intelligence is knowing facts rather than exercising good reasoning would probably be more disturbed by having to acknowledge getting the facts wrong.

So what does this suggest? Perhaps we should encourage more tentativeness and skepticism, an appreciation of the probabilistic nature of knowledge, comfort with staking cautious claims. Maybe we should ask people to propose multiple conditional hypotheses instead of single predictions. And most of all, we should put wrongness back in its place– linked to the idea, not the person.

How facts backfire

On “How facts backfire“:

Researchers discover a surprising threat to democracy: our brains

This has profound implications for educating the general populace. I’ve actually just been pondering the ethics of educating people up to (down to?) the trough of the U-shaped curve of learning and development.

Lately I’ve found myself coming back to Strike & Posner’s “intelligible, plausible, and fruitful” criteria for conceptual change. If our target audience doesn’t perceive these new ideas to be fruitful, they’ll have no motivation to change.

I’ve also been thinking of all these ways in which a little (or a lot) of knowledge can make learning harder: backfire, U-shaped development, expert blindspot, information overload. I’ll probably think of more to add to the list later. Given the considerable risks of this happening through so many different mechanisms, how can we equip learners against them? It seems that some of the answers may lie in influencing the learner’s affective, motivational, and metacognitive states: making errors and belief change nonthreatening, incentivizing accurate information and valid reasoning, and developing an understanding of these cognitive errors. But I’m still concerned about learners for whom this doesn’t succeed and who then get left worse off than they began.

Knowing when you don’t know

We learn by challenging our beliefs and exploring what we don’t know…

  • (as demonstrated by “Confirmation Bias“:
    The Truth: Your opinions are the result of years of paying attention to information which confirmed what you believed while ignoring information which challenged your preconceived notions.)

…but the problem is how to deal with not knowing that we don’t know.