Second webinar on badge system models and design considerations

Yesterday we held our second webinar of phase 2, “Badge System Models and Design.” It featured a great presentation by Carla Cassilli of Mozilla about the many considerations of designing an effective digital badge system. You can watch the video below, or at http://youtu.be/zCAy5weZyHc, and you can download the slides directly as a PDF right here.

Questions and Answers: Badges 101

During our first live, interactive Badges 101 webinar, we received over 100 questions through email, Twitter, comments, and the webinar chat box, and our Badges for Lifelong Learning team jumped in to answer many of these questions below. Have new questions for us? Join our second Badges 101 webinar Monday, October 17, 2011 at 2pm EST/ 11am PST.

Flickr photo via Veronica Debord

Will ‘complexity’ reduce the value of badges in the eyes of potential employers because they don’t have the time to unravel that complexity?

It depends on the employer.  And, of course, this is true for any evaluation system. We don’t have anything like one current standard–and we won’t with badges, and shouldn’t.  For some, badges will seem too complicated. For others, interesting and important and no more complicated than the current system of resumes with their odd code words, their obscurity, their vagueness, and jargon.  Face to face interviews and verbal recommendations will likely not be replaced by badges, at least not for traditional employers.

Can badges provide a mechanism for employers to better identify and recruit suitable employees (and vice versa)?

Absolutely — if employers wish to devise and use such a system. The point is that badges are community-driven and members of the community have to find ways to use them, and reasons for using them that fit their organization.

People who earn digital badges signify to employers what their skills and knowledge are regardless of whether or not they possess a degree.

Badges are not degrees.  Some people who earn badges will have degrees, some will not.

Who will we see becoming the leading badge authorizers?

No one knows. Badges for Lifelong Learning is an experiment and a competition.  We cannot know who is best or leading or most important before the experiment even begins.

How can we ensure that badge inflation does not occur?

You can never ensure that any system will be perfect until humans become perfect.  Short of that, in systems where there are not negative badges or demerits, people earn badges by doing things.  The badge carries with it the record of what earned that badge.  You can over-accomplish, I suppose, in order to earn more badges but that seems a bit self-defeating. Still, any system is susceptible to abuse, as we certainly know from the current one where everyone is, as the saying goes, “above average.”

Does there need to be a standardisation of badges for them to be valuable?

No. The point of peer evaluation is how it carries what it signifies with it.  All that needs to be standardized is the operability of the badging systems themselves, so they can be readable across different platforms.

Is there a badge standard? Are some worth more than others based on who issues them?

The community or organization or employer seeking the best employee or community member will have standards and will see badges that meet those standards.  But what those standards are for which kind of organizations — nursery school teacher versus Java Script programmer — varies by the nature of the organization and the position being sought or even the kind of role within the organization.   I might want to see a different kind of badge for a nursery school teacher if I am thinking of hiring one versus if I’m thinking whether I want my child to go to a certain nursery school.   Actually, come to think of it, in that situation I would hope the criteria are quite similar, that the owner of the nursery school and the parent wanting to send a child to that school have similar values, requirements, obligations, and expectations for a great nursery school teacher.

Who determines when someone earns a badge? Or that badges from different issuers are equivalent?

Peers give badges within an organization or across organizations.  You can see a badge, click on the badge, and then find all the content that, together, comprises the reason for the badge having been issued.  No external body determines “equivalence.”  The point is to be able to acknowledge different forms of contribution at different levels.  This is high standards without standardization.

It seems that success of badges would partially depend on educating employers and the public on their value?

Of course.  And there will be early adopters and there will be resisters.  Mt. Holyoke was the first university to use the ABCD grade.  The American Meat Packers Association came soon after but they then decided the system was too inflexible and therefore meaningless for grades of sirloin and chuck.  Other universities, however, felt no such qualms and rapidly went to the ABCD grading system.  Something similar may well happen with badges.  To what extent is the ecosystem envisioned as a place where groups — classrooms, clubs, etc. — earn badges, as well as individuals?

How about badges for organisations (providing badges)?

This is certainly something that could be done. Currently, this is not built into the Open Badge Infrastructure (and defined badge metadata specification) because we are focusing on individual accomplishments and skills. At this point, all members of a group could be issued a badge, but each individual would get it (and own it) separately.

If this is going to work on a larger scale, do you think there needs to be a badge accreditation system to assess badge issuers and curricula?

We are talking about a different and new system here so we want to avoid imposing limitations on the system from the beginning. There is already plenty of information carried with each badge, including the criteria/assessment behind the badge and the issuer, so that is enough information to value the badge for most cases. Mozilla is building the capacity for third party endorsements of badges, so that could function as a way to add in more formal accreditation, or there may be markets/organizations that emerge around the consumption side of badges and create filters or ranking systems. We don’t know yet, but there are a lot of possibilities and we should be open to considering all of them at this stage.

How are assessments actually integrated into the badges?

The badge is something issued after the successful completion of an assessment. So the learning experiences, assessments and interactions can still occur as they are now, its just that there is now a badge at the end to recognize the learning/skill. Once issued, the badge includes metadata that explains the badge, including a link back to the criteria behind the badge, which in many cases will be a description of the assessment. So in that sense, the assessment is embedded with the badge so that people consuming the badge understand what was accomplished to earn it. Also, there is an optional piece of metadata that links back to the evidence, or learner work. This might be the work submitted for the assessment, and even the assessment feedback. This information then is also carried with the badge. (NOTE: this metadata field is optional since some evidence URLs may have personal information that the issuer/user does not wish to share).

Could a third party issue a badge based on the completion of an open course, without the input of the original institution?

It depends on how the open content is licensed. If it is completely public and remixable, there could be third parties that take the content and build experiences and badge systems around it. But if the original insitution maintains some rights to the content, then they would most likely have to approve or own the badge system aligned with the content. It may be that they explicitly open it up to other badge designers and endorse a certain subset of those badges more formally.

How about coordinating badge efforts with iTunesU and OpenCourseWare?

Yes, definitely! Both ITunesU (or content owners on it) and MIT could be badge issuers. They would be very valuable additions to the ecosystem.

How do we authenticate users’ identities? Hopefully users can’t transfer badges?!

For badges in the Open Badge Infrastructure, the user is identified via email address (using Mozilla BrowserID). The user can log into their Badge Backpack and view all of their badges associated with that email address. When they try to use the badge, or put it somewhere, the consumer/displayer can call back through the OBI to the issuer and validate/authenticate the badge by asking the issuer if this badge is connected to this user, and if it is still valid. If the issuer responds positively, the badge is validated and confirmed. If there is not a positive response, the badge is unvalidated.

If a badge is an open badge, can it only be used on open platforms? Or could I display these badges on my company intranet?

The badge infrastructure is designed to be open and accessible, thus allowing portability and representation across the web — that includes all platforms. If your company designs a widget that permits badge display on their intranet, the infrastructure could accommodate that sort of display. Should your company develop an internal assessment rubric and associated badge system, they can issue and display badges that can then in turn be shared outside of the corporate intranet. Sharing reinforces the value of the infrastructure and the badge ecosystem.

Should badges ever expire, or should they be permanent?

One of the defining aspects of the Open Badge Infrastructure is the badge manifest, or the information that defines the content embedded in the badge PNG. In the manifest, the badge issuer can assign expiration dates to badges so as to limit their lifespan. As to whether or not the badges should be permanent, that’s a decision to be made by the badge issuer when they develop assessment and award policies.

How much control will I have over my badge portfolio? I’m on the job market right now, and I tweak my resume depending on the job I’m applying for. Could I have separate urls that I give to different employers, each one showing the badges I might want them to see?

As we develop the Open Badge Infrastructure, the badge recipient is foremost in our minds. Badge recipients, or owners, will have complete control over where their badges are displayed. Indeed, recipients will be able to host their own badge backpack, thus controlling the display of their badges. As the badge ecosystem grows, recipients will have increasing opportunities to display their badges in new venues (websites, blogs, Facebook, professional job sites, etc.)

How will badges display obsolescence?

If by obsolescence you mean expiration, badges are designed to have the capacity to expire. We are addressing how expiration might be indicated in badge display as well as in the badge backpack. The question of badge expiration will be answered by the badge issuer during the badge system assessment and award development.

Is everything badge friendly?

An interesting philosophical question. Badge issuers will be addressing this question as they begin to create assessment and award systems that underpin open badges. Open Badges offers one attempt to address learning, skills and competencies that are currently either unrepresented or underrepresented in traditional, formal personal representation on resumes and CVs. Soft skills such as community-mindedness, peer interaction, and mentoring present excellent assessment opportunities that may result in some of the most important badges to arise from the ecosystem.

How would an employer stay on top of an ever-growing list of badges which certify the relevant skills? 

Open badges will carry information needed to understand and validate them so employers can use that information to see the criteria and evidence behind the badge, and confirm that this person did in fact earn this badge from this organization. Beyond that, there may be markets that emerge to help us filter, rank and make sense of badges to help employers quickly figure out how to value the badge.

Will badges make learning cheaper?  

It depends on what is meant by cheaper. Will badges make learning more inexpensive and accessible? They could make some learning channels more viable and legitimate for jobs and other types of advancement. Will they cheapen or diminish learning?  We don’t think so. Badges for Lifelong Learning is about giving recognition for learning that already occurs. We certainly stress the importance of robust assessments and innovative approaches to learning behind many of the badges, but also see value in other kinds of goal-driven badges as well. We hope solid research and attention is given to these badge systems to have more evidence behind their effectiveness and best practices.

Could badges apply both to individuals (students) and institutions (schools, employers, government, etc.)?

The awarding of badges does not have to be limited to individuals, although the current iteration of open badges does focus on individuals and their accomplishments. As the ecosystem evolves and grows, it may begin to encompass large and small organizations, institutions, and government agencies. Endorsers, third parties who endorse certain badges or badge systems, will begin to have an impact on the development of the entire ecosystem. One of the best parts of the Open Badge Infrastructure is its openness, which means that all members of the open web can help frame it, build it, and benefit from it.

Who is the target audience for badges? K-12? Postsecondary? 

While the academic community has responded vocally to the idea of open badges, the target audience includes any organization, institution, individual, group, etc. who would like to offer and support representations of learning, achievements, skills, and competencies.

How are assessments actually integrated into the badges? 

You’ve asked an important question. Mozilla is developing the Open Badge Infrastructure that acts as the plumbing to the badge ecosystem. We’re making it easier to formally represent learning, skills, achievements, and competencies. The other vital aspect for developing the open badge ecosystem involves individuals, organizations, groups, institutions, and government agencies creating and defining badge systems and assessment rubrics. Essentially the ecosystem arises from the creation of those badge systems and assessment rubrics and Open Badge Infrastructure guarantees the portability, personalization, and openness of the system.

It’s not the badges that are of value to a Boy Scout.  It is becoming an Eagle Scout that outsiders value.  If I didn’t go to college and collect badges for 4 years I might have a lot of skills and knowledge.  But how is that packaged into something of value that is recognized? 

A badge system will have different values embedded in it along the way. There are plenty of Scouts that don’t become Eagle Scouts but still proudly wear their badges. There is something to be said for getting incremental achievements, while also having something larger that motivates the learner to work towards.

*****

Want to learn more about badges? We’re aggregating social media around badges at the Badges for Lifelong Learning Group on HASTAC.

New webinar: Digital Media and Learning Competition: Process and Application

Do you have questions about the Competition’s structure and application?

During this webinar, the second in our series, we will walk prospective applicants through this year’s Competition process–reviewing each of the three stages and their requirements, the newly extended timeline, and the application requirements. We will also respond to specific application/process related questions from applicants. Questions can be submitted in advance by emailing dml@hri.uci.edu and including “webinar question” in the subject line.

“Digital Media and Learning Competition: Process and Application” takes place Tuesday, October 11, 2011 at 3pm EST / 12pm PST

  • Date/Time: Tuesday, October 11, 2011 at 3pm EST / 12pm PST
  • Duration: 50 minutes
  • Location: GoToWebinar https://www3.gotomeeting.com/register/953425726
    Advanced registration recommended, but not required. Webinar will open at 2:45 PM EST to allow registrants time to establish access to the webinar.
  • Hosted By:
    David Theo Goldberg, Director of University of California Humanities Research Institute and HASTAC Co-Founder;
    Carla Casilli, Project/Operations Manager, Open Badges, Mozilla Foundation.

An archived version of this event will be available at dmlcompetition.net. Information about other upcoming Badges for Lifelong Learning Competition webinars will be available at http://www.dmlcompetition.net/Blog/

 

Unpacking Badges for Lifelong Learning

Badges are complex. Nothing functions quite the way they do, and at the same time, badges function like a lot of other things. They’re versatile, which makes them interesting. And probably powerful.

In the past week since Badges for Lifelong Learning launched, people have written critical, constructive, and positive things about badges, but I haven’t come across anything that really unpacks what badges are. I’ve read that badges are like credentials, related in ways to diplomas and degrees. Grades are sort of like badges, but worse. Badges can function like currency. The word badge tends to elicit memories of Boy Scouts for guys. Badges are shorthand for skills achieved, and can convey rank and reputation. Badges can be completely silly and extremely serious. Gaming is having a good run with badges, and that bugs some people. People like to collect badges. Marketers are getting drunk on badges and should probably chill. Is there some core definition or badge-ness to explain what makes badges unique?

[badge]

Being a nerd, I did a word look-up in the Oxford English Dictionary (sorry, Wikipedia), which says badges were a device to signal membership and rank within a group (1400s). But badges also signaled immaterial things like love and virtue (1500s) and knowledge (1600s). By the 1800s, one writer says degrees had “become social badges.” So badges have been around a while, doing some different things for sure, but mostly not causing a lot of trouble. When someone likes or doesn’t like badges in 2011, I’m curious what it is about them that triggers strong emotions.

If badges are like degrees, diplomas, grades, or currency — which many of us have collected and displayed and benefited from — what’s wrong with them? Why are badges worse or better? If badges are visual signs of rank, reputation, membership, and identity, and are just another way to show affiliation, why are they different than, say, titles, clothing, hair, language, accents, bumper stickers, friends, or an alma mater?

On Planet OpenBadges, Erin Knight invites people to talk through similar questions. In her helpful summary of four themes driving the badges conversation, it’s the assumptions about motivation mentioned in theme #3 and the latter part of theme #2 — that badges “will ruin our motivations for the things we love to do just because we love to do them” — that seem to deliver a punch.

Why? Because badges hinge on motivation. Most of the energy in the badges conversation seems to have roots in the different ways people think about motivation, and more specifically about motivation and learning. What motivates learners to learn? What de-motivates them?  If you work with youth or have your own, chances are you have some ideas about motivation and what works and why. If you motivate learners, what if it’s at the expense of something else? What if learners are motivated by the wrong reasons? What if we mess up what learners naturally love doing and blow it for everyone? Where’s the line between motivating a learner and manipulating them?

Motivation as a modern construct dates back to Darwin and Freud, just to underscore how colorful the conversation around desire, goal-setting, and achievement can be. In my own research, I’ve been reading about motivation (around participation in online communities), and it seems to me that diverse disciplines each have their own horse in this race. HASTAC exists for this kind of collaboration-by-difference conversation. Maybe we need a HASTAC Scholars’ forum to help talk through what we know about motivation and participation. Media studies, humanities, sociology, information science, education, social psychology, economics, who am I missing? Bring out your motivational theories. Discuss.

For me, the most interesting intersection of the Badges for Lifelong Learning conversation is where learning theories overlap with research into virtual communities, new collectives, commons-based peer production — whatever you want to call what we do online. A good deal of Internet research is about participation and motivation. If anything connects the badges community, it’s seems to be the belief that more participation is better. Collaboration is better still. Making and doing is best. Isn’t that what binds all these diverse disciplines and backgrounds engaged in this conversation? In the virtual community research I’m familiar with, it seemed to take a long time to recognize that lurking was a form of listening. We’ve finally begun to call it reading. And I’m willing to bet that the Badges for Lifelong Learning competition will get us closer to calling it learning. That makes it 15+ years to go from lurking to learning, which is slo-mo in Internet years, and super speed IRL.

We’ve only begun to get our heads around the shiny new Internet, and that goes for social participation and motivation, and in particular for learning. Human-computer interaction and social computing research and design tells us that big and small tweaks in socio-technical structure cause all kinds of interesting things to happen, changing how people participate and contribute online. Different groups, group size, kinds of individuals, individual skills, technical affordances, type of content, no policies, lots of policies, participation over time — changes in each of these areas causes changes in motivation and social participation. Can’t the same be said for motivation and learning online?

(If you do research in this area, maybe you feel flush with answers, but hello. It’s 2011 and the Digital Promise just got funded. ARPA-Ed is still in limbo. Funding for research and development of 21st century digital media and learning is a drop in the bucket compared to investments in other sectors. For now, we need to share what we already know and borrow as much research as we can from better funded areas.)

The communities of practice research links new collectives like Wikipedia with learning and identity, and authenticity is thought to affect people’s motivation to learn and participate and reach goals. Authenticity seems like a rich area when it comes to motivation and badges. Because of Mozilla’s Open Badges and the Badges competition, we’re playing in a bigger badge and learning sandbox than we’ve ever had, with the potential to acknowledge open learning on a scale that’s never been connected quite like this before. We’re entering territory where the 1 percent rule, Pareto’s principle ( the 80/20 rule) and other power laws are usually applied. I might need statistics friends to check my thinking here, but I’m curious: if the 1 percent rule (which some call the Internet rule) of the people contribute content online, 9 percent edit it, and 90 percent don’t contribute at all, how might an open badges system affect that rule, especially if we redefine participation and contribution in terms of reading and learning?

Not to get too nerdy here, but I hear there hasn’t been much research on collectors and collecting behavior. There’s this obscure ID Compensation theory that isn’t even on Wikipedia! yet! — a theory that suggests there is very little objective feedback in people’s lives to tell them if they’re doing well, which leads many people to seek out experiences or situations that offer frequent feedback. What if badges are just one more way to represent feedback? What if they’re the best, most versatile way to provide feedback, whether that feedback is many-to-one, one-to-one, or many-to-many?

I get that some people are down on badges in terms of game-based learning, and no doubt there’s research to show that extrinsic rewards like badges can demotivate learners and mess with what’s to love about informal learning. But frankly, many examples of extrinsic rewards and motivation to participate or contribute seem highly contextual. Research on incentives and participation in virtual communities tells us that small tweaks in design influences extrinsic motivation in surprising ways. When it comes to motivation, extrinsic rewards, authenticity, scale, group dynamics, new collectives, individual or social behavior and technical design, there’s so much we still don’t know. And that doesn’t include what learners have yet to tell us about reading, participating, contributing, collaborating, making, doing, learning, reaching goals and achieving skills on the Internet. And what if those learners were invited to design their own badge systems in their own communities of practice? We need to be thinking: When is a badge system good? When is it not? The critical, constructive, and positive comments on badges for learning have been so valuable (I’ve been collecting badge posts on HASTAC’s Scoop.it topic, Badges for Lifelong Learning, for those who want to read through the collection, plus there’s #dmlbadges and #openbadges on Twitter), but this is a conversation for the big tent. Badges for learning is an undertaking that’s ripe for sharing knowledge.

No doubt there will be Badges for Lifelong Learning applicants who present game-based systems proposals. Perhaps that’s an obvious fall-back, especially given that games for learning are having a moment. But badges were here before games, and I have no doubt there are bigger badge ideas out there, ones that have nothing to do with the G-ification word. If we’re fortunate, those bigger ideas will be inspired by the Badges for Lifelong Learning competition. Or they’ll emerge naturally once Mozilla’s Open Badge infrastructure launches and people start to imagine possibilities and build on early innovations.

Whatever you think about badges, I’m all for Erin’s approach: join the conversation, join the competition. Explore this with us.