What If Teachers Decided (for Themselves!) What Counts?

November 17, 2011

By Cathy Davidson

We at HASTAC are extremely proud to announce the opening of a new Competition, designed specifically for educators, “Teacher Mastery and Feedback Badge Competition.”  The purpose is to support educators in their own professional goals, in their desire to develop their skills and knowledge, and in their own professionalism in judging quality—not being judged, top down.  This Competition begins from the premise that great teachers should be acknowledged and, equally, that great teachers work hard to get that way and should be appreciated as such.   Who better to know how to do this than educators themselves?  This Competition invites educators to think about the most creative, interactive, interesting ways of deciding what counts most for great teaching—and how to count it.

My personal investment in this Competition is in that it is based on supporting an ideal of professionalism to one of the great professions that, in recent years, has been something of a whipping boy to many.   Face it, it’s not easy being an educator these days.  It’s not just the de-funding of schools, not just the requirements for end-of-grade testing that may or may not have much relationship to actual knowledge, it’s not just that much of our thinking about what education is for is antiquated and hasn’t been re-thought, top to bottom, for the information and communication revolution that began in April of 1993 when the Mosaic 1.0 browser went public and the Information Age officially began.   No,  it is hard to be an educator these days because teachers have been subjected to some pretty “regulatory” measures lately.

Sometimes these come from higher up, being initiated by politicians rather than educators themselves.   The language is of “certification” and “accreditation” but too often it is not clear if it is real quality that is being measured or something far more bureaucratic, that has little to do with a teacher’s ability to inspire students or to keep up with the changing knowledge in a field.  Too often, these “merit” systems  don’t really measure merit but, instead, undermine teachers’ own sense of professionalism, as if teachers aren’t the ones most concerned with our own high standards (the best ones of us are!).

I’m convinced a lot of the mentality of policing and regulating teachers is contributing to the national crisis of many of the best teachers leaving the profession.   According to the National Education Association, about half of new teachers do not stay in teaching.  In a U.S. Department of Education survey of 7,000 teachers  who had recently quit or said they were likely to quit soon, the #1 reason given was intrusive administration; another was cumbersome and ineffective  accountability procedures.  And some of the other top five reasons were also about this externalizing of  the “metrics” for excellence over the inspiring, creative, intelligent, and powerful ways that motivate kids to learn—and motivate teachers to stay in the classroom, despite the low pay and hard work.

The reason I wrote the “How We Measure” chapter of Now You See It is because all my research, including interviews with dozens of great teachers, underscores that we now have ways of measuring “quality” that are neither about quality, nor even about good ways of measuring.

  • If a multiple choice, end-of-grade test only covers about 25% of the actual content/material in a course, what about all the rest?
  • If we know that you have to “teach to the test” to ensure your students get the best test scores,  what  happens to the ideal of teaching to improve students’ real skills and knowledge (not just test-taking ability that has little real-world relevance)?
  • If we know the biggest motivator to testing well is believing high scores will help get one to college, then what about all the kids who know they will never be able to afford higher education?

All that wasted effort!   All those ways of measuring qualities peripheral to the ones great teachers know are needed to inspire kids.  It’s a tragedy, and it is sending our best teachers out of the profession fast.

Will this one Badges for Teacher Mastery and Feedback Competition solve all problems?  Of course not.  But we are extremely proud, at HASTAC, to announce the opening of a new Competition designed specifically for educators, that puts educators in a leadership role, helping to think about cutting-edge new ways of assessing what they know to be high quality,  important new skills and areas of knowledge.

This isn’t for everybody and shouldn’t be.  We are trying not  to go for one-size-fits-all which we think of as the kind of standardization that de-motivates true learning.   Rather, we invite any educator who is passionate about these issues to compete, to show their ideas on our all-public website, and to inspire others to think deeply, too, about what counts in the classroom, what should be counted, why, and how.

We assume most applicants will be K-12 educators, but we want any teacher, from preschool to professional school, informal and formal learning, who is deeply interested in thinking about new peer feedback and mastery badging systems to apply.  We know that we all have much to learn from one another.

To those educators interested in these issues, we invite you to apply and we thank you for your dedication and your commitment to what, at HASTAC, is our motto:  learning the future together.

Here’s the link to the Competition application page:  http://dmlcompetition.net/Competition/4/teachers.php

Applications due December 5, 2011.

JUST ANNOUNCED: Teacher Mastery and Feedback Badge Competition

November 17, 2011

In conjunction with the Badges for Lifelong Learning Competition, applicants are invited to propose badging systems not only for learning content, but also for teacher learning and feedback. Competitive submissions proposing badge systems that track and promote feedback regarding the competencies and skills as well as the programs and subjects over which teachers acquire expertise will be a central part of the Stage 1 and Stage 2 processes of the Competition. The winning proposal(s) will be awarded funding to develop the proposed badging system.

Learn more at http://www.dmlcompetition.net/Competition/4/teachers.php

Second webinar on Badge System Models and Design will be Nov. 30

November 17, 2011

The HASTAC/MacArthur Foundation’s Badges for Lifelong Learning Competition, launched in collaboration with the Mozilla Foundation, focuses on badges as a means to inspire learning, confirm accomplishment, or validate the acquisition of knowledge or skills.

During this live webinar for prospective Stage Two badge design applicants, we will delve deeper into the badge conversation and explore badge system design and development considerations. We will review different models of existing badge systems and discuss general guidelines and best practices. We will also walk prospective applicants through content, technological and team characteristics that should be considered when developing a badge system and putting together a proposal for Stage 2.

To learn more about the Badges Competition and the Research Competition, visit http://dmlcompetition.net.

Badges for Lifelong Learning Webinar: Stage Two Prep | Badge Systems Models and Design takes place Wednesday, November 30, 2011 at 3pm EST / 12pm PST
  • Time: 3pm EST / 12pm PST
  • Duration: 60 minutes
  • Location: https://www3.gotomeeting.com/register/741026966
    Advanced registration recommended, but not required. Webinar will open at 2:45 PM EST to allow registrants time to establish access to the webinar.

Questions can be submitted in advance by emailing dml@hri.uci.edu and including “webinar question” in the subject line.


Video from our first Badge Design webinar

November 17, 2011

Yesterday we held our first webinar of phase 2, “Badge System Models and Design.” It features a great presentation by Erin Knight of Mozilla about the many considerations of designing an effective digital badge system. You can watch the video below, or at http://youtu.be/1Zrirng0_ls, and you can download the slides directly as a PDF right here.

Could Badges for Lifelong Learning Be Our Tipping Point?

November 14, 2011

By Cathy Davidson

As more and more fascinating and creative and surprising applications to our DML Badges for Lifelong Learning Competition flood in (we’re at almost 100 and the competition does not close until the end of today), I am wondering if, a hundred years from now, some historian ploughing through the dusty data archives of the Internet, will see this moment as digital learning’s tipping point.    I mean that.

This could be our tipping point for how we measure, the entry point to thinking up an array of new forms of deciding what counts for our era.   This Competition isn’t the end point, but the beginning, but it could tip the balance so that, all over, people are wondering why and how they are measuring quality and contribution the way they are and beginning to think about better ways—better ways that fit their organization’s values and goals.   That’s the key.   So many of us work in schools or in jobs where what counts has been decided for us.   The array of badging systems in these applications is starting to suggest that there are many, many of us who are frustrated with our inherited systems and want to come up with new ways of deciding what we want to count, what we want to value and acknowledge and credit and reward.     A badge system can be the symbol of all that,  visible proof of an organization’s some quality of participation and contribution that, previously, wasn’t even defined.

To understand why this is so important, we have to go back to the origins of the system that we have now, a system designed for the Industrial Age as part of the Taylorized movement of “scientific labor management.”   I speak of the invention of the item-response/multiple choice/bubble test, still the corner stone of our national educational policy, passed in 2002, called No Child Left Behind, and invented in 1914.  It was the tipping point in what I call “scientific learning management,” the application of Taylorized theories of uniform, standardized, timed, regulated productivity to education.

The reason I devoted a chapter of Now You See It to “How We Measure” is because, without a uniform method of assessment, there is no standardization. Standardization is the most important ideal of the Industrial Age–but is quite contrary to the peer-led, interactive, contributory, connected ways of learning and interacting that the World Wide Web affords us.    If we are going to truly transform our Industrial Age institutions for the digital age, we have to re-evaluate how we evaluate.  We have to come up with interactive, process-oriented new methods where peers can decide all the different things that count for them and why, and figure out a way to count them.   We do not have to know what those outcomes will be.   If we did, we would be buying into “scientific leaning management” again where, like all Taylorization, the outcomes are determined in advance of the process (Taylor called them “quotas”).    Outcomes–for labor productivity or learning productivity–are defined in advance.  They are the bar you have to get over, the scale on which you are measured.

In scientific labor/learning management,  there is a set scale that measures only pre-defined kinds of productivity and pre-defined forms of achievement and you are assessed by a standardized form of testing only on those things, and you are them measured against all other workers/learners and rewarded on that scale.  Badges for Lifelong Learning offer us other ways of measuring and other ways of thinking about what qualities and contributions we might want to measure.   

Don’t you see it?   At present, just about everything else about school and work rests on evaluation.  If the goal is set in advance, it changes the process.   Even if you try to modify the process for another end, you are modifying it against a set standard.   A standardized assessment metric is a mentality as much as it is a measurement.

How Did We Get Here?

In the “How We Measure” chapter of Now You See It, I go back to the archives to find out who invented the mulitple choice test, the tipping point in fully turning the movement toward compulsory, public education into a more uniform, standardized system for the Industrial Age and conforming to Industrial Age values.  The invention of that item-response form of standardized assessment, invented in 1914 (and virtually unchanged in the present) is based on Taylor’s “scientific labor management” that is the basis for the assembly line model of industrial manufacturing.   Timed, standardized, uniform, in quality and in method of assessment.   Frederick J. Kelly, the inventor of the standardized test, transformed “scientific labor management” into what I call “scientific learning management.”  The test was the single most important apparatus of an educational mentality that has lasted nearly 100 years.

Here’s the background on Kelly.  He was a doctoral student at Kansas State Teachers’ College in 1914.  Men were fighting in Europe in World War I.  Women were in the factory.   Compulsory public education was now the law of the land in every state and the age by which you could leave school had changed to 16, meaning that two years of secondary education were no longer just college prep but for everyone.   At the same time, the rank of immigrants coming into the secondary schools of the U.S. public school system was swelling at an extraordinary rate, from 200,000 in 1890 to 1.5 milliion in Kelly’s day.   There was a crisis.   Kelly looked at Model T’s being turned out in standardized fashion and came up with the itemized test, first, because it gave some kind of objectivity to what was slipshod processing of all these students through the educational system and, second, because it was cheap, fast, and easy—like turning out the Model T’s.

The reason the bubble test (what Kelly called the Kansas Silent Reading Test) caught on is because, in the decentralized state-based educational system in the U.S., a standardized test allowed some form of assessment across schools, school districts, and across states.   From the U.S., the system spread to the world.  America tests earlier and more often than any other country on the planet, but virtually every country has adopted some form of bubble testing.   And it is an industry, worldwide, with billions of dollars of commercial investment and return.   There is a lot at stake.

But does hierarchical, timed, pre-defined, uniform, standardized testing really measure the kinds of intelligence and activity that our kids need for the challenging world they will face as adults?    Do similar forms of standardized evaluation really work in the workplace today?   The “timed test” is a weird way to measure intelligence, when you think about it.   It’s hard to imagine trying to even explain it to Newton or Leonardo or Galileo . . . that a timed bubble test would be the pinnacle of intelligence would convince great thinkers of the past that the 21st century was for lemmings running fast off the cliffs.   I agree!   It is a system for another century.  It may have worked for that one.  We need better ways of evaluating contribution now.

Sadly, Kelly would have agreed with me.   The father of item-response testing himself wanted to abandon this make-shift way of testing “lower-order thinking” (as it was called in 1914) once the First World War was over.  He became a Deweyesque integrated thinker, who believed all subjects were relevant to one another and answers were processes, not products, to be filled in, bubble after bubble.  He went on to be President of the University of Idaho and tried to reform that university to these more integrated, interdisciplinary, process-oriented “higher order thinking” goals.  His faculty was furious at this presidential plan.  The faculty there had wanted to hire the father of scientific learning and measurement.   By then, even the Scholastic Aptitude Test was using a timed bubble test to decide who would or would not get into university.  Kely was fired from his Presidency within two years.

You can find a short version of this story here, in the Washington Post:  http://www.washingtonpost.com/opinions/standardized-tests-for-everyone-i…

Badges for Lifelong Learning: The Competition Closes Today but the Thinking Has Just Begun

If you want to peek in on how this badge competition is unfolding, you can.  You can read about the array of organizations that are taking the chance to try something new, to think in new terms about what they want to measure, and how and why.  Check out the applications.  They are all public.  Some have logos, some do not (that is not a requirement); if you click on their box, you will be able to see their entire application : http://dmlcompetition.net/Competition/4/badges-projects.php?group=dmlc-4b

From these applications, you can learn and get ideas that might for yourinstitution or organization.  That’s the point.  If you are a teacher, you can go into class today and ask your students what they think is most important thing they will learn in the class, what they think they are learning, not just in content but in form.   That, for me, is the best part of this Badges for Lifelong learning:  we can all learn from this process, from these organizations willing to step back and think about what system might work best for them, now.   Standardized testing is not the only way to evaluate quality. 

What Makes a Tipping Point?

Before there can be institutional or organizational change, there often has to be a crisis.  For Kelly, it was World War I and the immigrants who needed to get through the newly required secondary educational system.   For us, now, it is a worldwide economic crisis but also a crisis inhow we work and in defining what work is that is far more complex and complicated than the systems of education that are supposed to prepare kids for independent adult hood.  I don’t mean “job preparation” in a simplistic way.  I mean, systems designed to inspire and reinforce values and forms of responsibility, self-regulation, self-determination, and maturity that can help us to thrive in a complex world.  

This Badges for Lifelong Learning Competition is by no means  the end point but it may be a beginning.  It may be the starting point, a tipping point, in helping us to think about How We Measure, and helping us to think through better ways.    I am gratified beyond words by all those who have taken the last few months to think deeply about what their organization needs and to work together to propose something that the rest of us can be inspired by.   That process, in and of itself, is an original and bold one that very few organizations ever engage in.   It is a bold step towards learning the future together. 



Cathy N. Davidson is co-founder of HASTAC, and author of The Future of Thinking:  Learning Institutions for a Digital Age (with HASTAC co-founder David Theo Goldberg), and  Now You See It:  How the Brain Science of Attention Will Transform the Way We Live, Work, and Learn(Viking Press).  NOTE:  The views expressed in NOW YOU SEE IT are solely those of the author and not of any institution or organization.  For more information, visit www.nowyouseeit.net or order onAmazon.com by clicking on the book below.   To find out Cathy Davidson’s book tour schedule, visit www.nowyouseeit.net/appearances 

November 13, 2011

Two months ago we launched the Badges for Lifelong Learning Competition in Washington, DC, sparking a lively conversation that generated over 150 blog posts2400 + tweets, a series of webinars and, by tomorrow at 5pm PST, a flurry of Stage One applications. That’s right — there’s still a day left to submit an application for Stage One:

Submissions will require a 1000 word written proposal. They can also include optional supplementary materials that help visualize the proposed learning content, programs, or activities. These are highly encouraged and can include a video, diagram, screenshots, napkin sketch, or other visual expression that helps depict the proposed learning content, programs, or activities.

Proposals are submitted through the DML Competition submission web site: See Application Instructions
Multiple proposals are permitted from a single organization, but only one may be selected as a winner and move to Stage Two.

Submissions for Stage One are due no later than November 14, 2011 at 5pm PST.

Written Proposal

In your written proposal, please describe the following, including links to relevant material or resources where applicable. It’s not required that you answer ALL of these questions — you may not know the answers yet and that’s okay. The purpose of the questions is to help you fully express the content and opportunity, so that judges can understand the value of your proposal, as well as so that badge designers in Stage Two of the competition have the information they need to propose badge systems that are relevant and valuable for your content.

  • The learning content, programs, or activities that will be supported by badges. What are the primary domains of learning reflected by the content, programs, or activities? What are the overall goals for learning? Who is the main learning audience or target community? Does learning occur at a specific place or time (i.e., where and when)? How does learning typically occur? What programs and activities will a learner or group of learners experience?
  • The skills, competencies and achievements badges will validate. These may include traditional skills like writing or sports, programming in a particular language, operating a specific type of machinery, or 21st Century skills such as collaboration, communication, and teamwork. One of the benefits of badges over more traditional assessments is their ability to represent a wider range of accomplishments and evidence about an individual’s capacities, and provide a more complete and nuanced picture of their accomplishments and attributes. What are the main skills represented or developed within this content or learning experience? Are these skills and competencies better understood as discrete levels, or measurements of continuing performance?
  • Identity and roles. Do the proposed learning content, programs, or activities support specific identities or roles for the learner? For example, will they assume the role of a scientist, sound engineer, or writer, or build the identity of a collaborator, leader or creator?
  • Opportunities or Privileges. A badge or set of badges can be designed to provide opportunities or confer privileges to learners. What opportunities or privileges can arise from the content, programs, or activities? For example, advancing through a set of badges may provide access to mentorship or internships, available equipment, review of work by professionals, access to an elite community, or a new experience.
  • Existing assessments. What existing assessments or tools, if any, do you have for tracking or measuring performance? Do they align well with badges? How so?
  • Partners and Organizations. Does the proposed badge or set of badges require partners and/or other organizations? What are their roles in the learning content, programs, or activities, and potential roles in creating a badge or set of badges? Does your team, or a partner organization, include someone with expertise in assessment?
  • Administration of the badges. Who would administer the badges — your organization or a partner? Where would the badges be deployed or displayed? Would they appear on your website or another website? How would this occur and what infrastructure will be required to support it?
  • Branding. Ultimately, a badge or set of badges for your community of interest will represent you beyond your institution. What elements of your brand are relevant to your badges? These may be general brand perception elements or specific visual elements like logos, colors, shapes, etc. Consider that you may begin with a single badge but that it may grow into a family of badges and ultimately a full ecosystem.

Important note for organizations with existing badges

If you already have a well articulated design for a badge or set of badges, and/or a supporting technological infrastructure for the learning content, program, or activities, you must still apply through Stage One and describe the core content and programs elements indicated above. Please also briefly describe your existing badge system in your Stage One submission, as well as your goals for the future iteration of your system. You can also include existing badge artifacts in the visual elements of your proposal. If your learning content, programs or activities are selected, you will formally submit the full badge system description, plans, and materials in Stage Two.

Where Are They Now? Updates from the Digital Media & Learning Competition Winners

November 10, 2011

Image via http://www.flickr.com/photos/mikechen-metalman/4394618656/

 Eric Gordon and Steve Schirra of Participatory Chinatown, 2009 Digital Media and Learning Competition winners, have published the article Playing with Empathy: Digital Role-Playing Games in Public Meetings. The article will be published in the ACM proceedings of Communities and Technologies 2011. From the discussion section:

The community’s knowledge that this public meeting would be based around the Participatory Chinatown game attracted different people to the meeting; it also created different expectations of what was to happen at the meeting. It is unknown to what extent the novelty of a “game meeting” played into Participatory Chinatown’s high attendance or energetic participation; the high attendance at the meeting actually required some participants to have to share a single laptop, and thus a single character. Rather than diminishing the experience, this seemed to promote greater cooperation and deliberation as the two players had to come to a consensus about the decisions to make for their shared character.

Josh Hughes, a Game Changer winner from the 2010 DML Competition was quoted in ESA’s online newsletter. “Video games are such a beautiful and vibrant art form. With games you can come up with a one-two punch that engages people of all ages and teaches them about earth science, or the periodic table, or physics.” — Josh Hughes, on the educational value of video games.

Congratulations to Mitch Resnick of Scratch & Share, one of our 2010 DML Competition Learning Lab winners, who was recently awarded a 2011 World Technology Award in the Education category. Mitch is LEGO Papert Professor of Learning Research and Head of the Lifelong Kindergarten Group at MIT Media Lab, and is the Making, Tinkering, and Remixing chair for next year’s 2012 Digital Media and Learning Conference. Awards were given on the UNITED NATIONS stage during the World Technology Summit conference on October 25-26, 2011. According to the WTN site:

The World Technology Award Winnners and Finalists are those individuals and companies/organizations who are — in the opinion of the WTN Fellows and Founding Members, through Awards voting process — doing the innovative work of “the greatest likely long-term significance” in their fields. They are those creating the 21st century.

Greg Niemeyer of Black Cloud, one of our 2008 Digital Media and Learning Competition winners, gave a presentation on Units of Engagement and social entrepreneurship at a TEDx Talk. Greg is Associate Director of the Data and Democracy Program at CITRIS (Center for Information Technology Research in the Interest of Society), where he co-founded the Social Apps Lab (and just released the beta version of citysandbox.com)

Lissa Soep of Mobile Action Lab, a 2010 Digital Media and Learning Competition winner, published Youth Media Goes Mobile in the National Civic Review’s 100th anniversary issue under the theme: Beyond the Digital Divide: How New Technologies Can Amplify Civic Engagement. Lissa’s article explores how young people’s involvement in mobile design and development can foster new literacies and modes of participatory politics. Lissa is also interviewed about her work in a video on New Media and Collaborative Learning posted on the New Learning Institute site. About Mobile Action Lab and Youth Action Radio, Lissa writes,

“We’re running an intensive App inventor workshop over the next couple weeks with special guest facilitation from the guy who literally wrote the book about App Inventor, Professor David Wolber, USF. And we’ve got two other market-bound apps ready to launch, VoxPop (call-and-response, geo-located mobile radio) and Forage City (a new mobile platform to share excess produce with neighbors, non-profits, and people in need).”

“Thanks to the All Day Play app, which the Mobile Action Lab will publish next week, you’ll be able to hear internationally-recognized music tastemakers and cultural commentators anytime and anyplace on Android phones. Youth Radio’s high school interns are part of the publishing and promotions staff of All Day Play, a one-of-a-kind online radio station produced at our downtown Oakland street-side studios. A second group of teens co-created the app’s prototype using App Inventor, a tool launched by Google and now housed at the famed MIT Media Labs. Both youth teams have collaborated on the app’s design, testing, and user engagement processes. We’ll add the link as soon as it’s live! “

David Gibson of Global Challenge Award, a 2009 Digital Media and Learning Competition winner, has been busy launching simSchool 2.0, a “flight simulator for teachers.” Last month, simSchool version 2.0 was released, “as an ongoing effort to improve teaching effectiveness worldwide, with simSchool taking the next leap in building a full ecosystem for playing, learning and sharing around topics in teaching and education. This new release allows users to have access to expanded content creation tools, profile building features, an open library of resources to play and edit, course management tools for students and instructors, and enhanced reports that detail choices and impacts in played sims.”
Paul and Peter Reynolds of FableVision and Fab@School, a 2010 Digital Media and Learning Competition winner, recently announced the launch of their new non-profit Reynolds Center TLC this week at the MIT Endicott House in Dedham, Massachusetts. The goal of the Reynolds Center, is to bring, “inspiration, innovation, and community” to educators and students. FableVision also has a running list of grants for educators on their site, updated monthly. Share widely!

Upcoming webinar: November 16, 2011 @ 3pm EST | Stage 2 Prep: Badge System Models and Design

November 4, 2011

The HASTAC/MacArthur Foundation’s Badges for Lifelong Learning Competition, launched in collaboration with the Mozilla Foundation, focuses on badges as a means to inspire learning, confirm accomplishment, or validate the acquisition of knowledge or skills.

During this live webinar for prospective Stage Two badge design applicants, we will delve deeper into the badge conversation and explore badge system design and development considerations. We will review different models of existing badge systems and discuss general guidelines and best practices. We will also walk prospective applicants through content, technological and team characteristics that should be considered when developing a badge system and putting together a proposal for Stage 2.

To learn more about the Badges Competition and the Research Competition, visit http://dmlcompetition.net.

Badges for Lifelong Learning Webinar: Stage Two Prep | Badge Systems Models and Design takes place Wednesday, November 16, 2011 at 3pm EST / 12pm PST
  • Time: 3pm EST / 12pm PST
  • Duration: 50 minutes
  • Location: https://www3.gotomeeting.com/register/922871318
    Advanced registration recommended, but not required. Webinar will open at 2:45 PM EST to allow registrants time to establish access to the webinar.

Questions can be submitted in advance by emailing dml@hri.uci.edu and including “webinar question” in the subject line.