One of the discussions I was particularly excited about at the Barcelona Drumbeat Festival was using badges to indicate certain skills, abilities, capacities, traits, or accomplishments. The idea here is what you might find in Boy Scout merit badges, or Foursquare badges, or Stack Overflow badges: a quick way to see what a person knows, can do, and identifies themselves with.
As part of my courses in the coming semester, I am abandoning standard grades and instead using badge-level assessments. As part of each course, students can earn any number of badges for demonstrated abilities. These are generally badges that require you to show that you can do something. That ability must be assessed–often by peers.
Starting with the “data” end, what kind of information must a badge hold? We talked through a lot of this in Barcelona, and I’ve been thinking a lot about it since. What appears below shouldn’t be seen as the consensus of that group–though I found the discussion valuable, a number of the items below are certainly not commonly agreed upon among those, e.g., at P2PU who are talking about badges. At a basic level, a badge should be transparent (everything that went into getting the badge should be as visible as possible), and it should be imbued with the authority and reputation of those who were the evaluators.
Process
First, I should briefly describe the process. In the first courses, this process will largely be implemented “manually,” but you will see that there are many opportunities to automate some of these processes.
1. A person is nominated (or nominates themselves) by filling out all of the information on a form except for the endorsements.
2. Endorsers go to the form and indicate whether they feel that the candidate qualifies, for those that require endorsements–some may not. Note that “bots” may act as the endorsers, and check automatically whether something has occurred. In that case they behave just like human endorsers. Note also that the system that records this application should in some way verify the identity of the endorsers. We won’t be do that initially, but eventually, something (e.g., OpenID check) should provide an indication that people are who they say they are.
3. Once the endorsements are complete, a person may put this badge wherever they like on the web, with a link back to the page to show that they have earned the badge.
Nomination / Evidence Form
So, what is on that form? (With * items required.)
1. Name of the badge*
A short description of what the badge signifies: e.g., “Javascript Expert.” If it is a bootstrap badge, this should be clearly indicated in the title: e.g., “Javascript Expert [bootstrap]” (see #9 below).
2. Issuer of the badge*
Eventually, this may be something like “School of Webcraft” or “Quinnipiac University.” For this initial round, it is likely to be “ICM” for the ones I am doing.
3. Version of this badge*
Date-time last updated the badge.
A unique ID for the badge is formed with #1/#2/#3, e.g., Quinnipiac University/Ph.D. in Social Computing/2011-12-25-7:00:00.
4. Badge Image*
For the purposes of standardization, I will say 250x250px PNG representing what the badge stands for.
5. Description
A textual description of what the badge represents. The idea is that it is reasonably brief–say, less than 200 words.
6. Recipient*
Who is it that is claiming the badge.
7. Nominator*
Who is it that nominated this person for the badge?
By default, any badge can be self-nominated. If for some reason you want to exclude this possibility, it could be listed as a requirement in section 9: E.g. “Candidate is nominated by someone other than themselves” or “Candidate may only be nominated by a member of the track team.”
8. When nominated*
Nomination timestamp.
9. Requirements & Evidence
This is the meat of the form. It includes 0 or more requirements, with links to evidence that those requirements were met. Each requirement includes a record like the following:
a) Textual description of the rubric for assessment. What needs to be shown, and how is an evaluator to decide whether it meets the standard. Outside examples may be linked, including former examples of successful badge earners.
b) Textual description or link to the evidence of assessment. (If a link, we’ll probably need to find a way to archive that link for posterity. Easier with some things than with others; e.g., video.)
c) Nominator’s comments on the work and why they think it qualifies.
d) Qualifications to endorse. For example, you might require that people have the badge they are endorsing, or that they have a badge that qualifies them as “instructors” in the skill (e.g., to get the “pilot” badge, you need to be endorsed by at least one person with the “pilot inspektor” badge, or to get a a QU-PhD badge you need endorsements from three people with the QU-Faculty badge). You might also require that people have a badge that verifies their identity. So if I have the Verisignature–ReallyMe badge, maybe it qualifies me to endorse more badges. C is a list of required badges–there may be more than one.
e) Number of qualified endorsers required. This could be zero or a thousand.
f) List of
1. Endorser name
2. Date of endorsement
3. Comments on endorsement
Note that there is a necessary and automatic exception here in the case where there do not exist in the world the number of qualified endorsers listed in D. In that case, you must be endorsed by as many qualified endorsers as currently exist. It is then clearly indicated that the badge is a [BOOTSTRAP]. At some future point you might want to re-try the badge to get a non-bootstrapped version, once there are enough potential endorsers.
10. Issued Date-Time* (or PENDING)
11. Expires Date-Time
12. Recipient’s Comments & Notes
13. List of community comments
An Example Badge Template / Form
Now certain elements of the above are part of the template of a badge. So, if I nominated someone for the “Good Discussion Summarizer” I would end up with a template that included:
“Good Discussion Summarizer”
The Human Fund
1999-8-14-09:00:00
[Some Cool Badge Art that I don’t have time to dummy up at the moment]
The good discussion summarizer is issued to someone who has demonstrated that she is consistently capable of summarizing a brainstorming or other discussion in an academic setting, both verbally and textually.
Recipient:
Nominator: Alex Halavais (2010-12-2-18:55:03)
Badge Requirements, Evidence, and Endorsements
1. Statements from three members of courses in which the recipient is enrolled attesting to her abilities to accurately summarize materials. Endorser must hold the “current student” badge. (No evidence beyond the endorsements required.)
Evidence: (NB: this would be left blank.)
Endorser:
Comments:
Endorser:
Comments
Endorser:
Comments:
2. Evaluation of a video of the candidate reviewing a discussion. Endorser must hold the “Good Discussion Summarizer” badge.
Evidence (Link to video or audio of summarization):
Endorser:
Comments:
3. Evaluation of a textual summary of the same discussion. Endorser must hold the “Good Discussion Summarizer” badge.
Evidence (Link or Pasted Text of a summary):
Endorser:
Comments:
Issued: PENDING
Expires: TBD
Candidate comments:
Community comments:
The nominator would fill out some of these, including, perhaps, being one of the endorsers.
Other Issues
The natural question is how would endorsers know to find the form? There are lots of possibilities here, including informal or direct invitations, and a queue of badge candidates needing assessment. But that is a solution that does not have to exist in the badge process itself (necessarily). The idea is to keep this piece as simple and light as possible.
Happy to hear any thoughts you might have. As I said, I’m going to take it for a test run in the Spring semester. I’ll likely just have people do it manually on the wiki, unless I find time over the break to code a simple form system that can handle the pieces. And I’ll point to the course and the badge description (as well as some of the early badges) as I write them up.
As a final note, this doesn’t in any way take away from the efforts of the Mozilla Badge Backpack approach. Indeed, one of the advantages to that system is that it might provide the opportunity for several dissimilar badge systems to work together. In this case, what would be passed along to the Backpack is just an image of the badge, its name, and a link back to the form that demonstrates how it was earned.
What makes up a badge?
One of the discussions I was particularly excited about at the Barcelona Drumbeat Festival was using badges to indicate certain skills, abilities, capacities, traits, or accomplishments. The idea here is what you might find in Boy Scout merit badges, or Foursquare badges, or Stack Overflow badges: a quick way to see what a person knows, can do, and identifies themselves with.
As part of my courses in the coming semester, I am abandoning standard grades and instead using badge-level assessments. As part of each course, students can earn any number of badges for demonstrated abilities. These are generally badges that require you to show that you can do something. That ability must be assessed–often by peers.
Starting with the “data” end, what kind of information must a badge hold? We talked through a lot of this in Barcelona, and I’ve been thinking a lot about it since. What appears below shouldn’t be seen as the consensus of that group–though I found the discussion valuable, a number of the items below are certainly not commonly agreed upon among those, e.g., at P2PU who are talking about badges. At a basic level, a badge should be transparent (everything that went into getting the badge should be as visible as possible), and it should be imbued with the authority and reputation of those who were the evaluators.
Process
First, I should briefly describe the process. In the first courses, this process will largely be implemented “manually,” but you will see that there are many opportunities to automate some of these processes.
1. A person is nominated (or nominates themselves) by filling out all of the information on a form except for the endorsements.
2. Endorsers go to the form and indicate whether they feel that the candidate qualifies, for those that require endorsements–some may not. Note that “bots” may act as the endorsers, and check automatically whether something has occurred. In that case they behave just like human endorsers. Note also that the system that records this application should in some way verify the identity of the endorsers. We won’t be do that initially, but eventually, something (e.g., OpenID check) should provide an indication that people are who they say they are.
3. Once the endorsements are complete, a person may put this badge wherever they like on the web, with a link back to the page to show that they have earned the badge.
Nomination / Evidence Form
So, what is on that form? (With * items required.)
1. Name of the badge*
A short description of what the badge signifies: e.g., “Javascript Expert.” If it is a bootstrap badge, this should be clearly indicated in the title: e.g., “Javascript Expert [bootstrap]” (see #9 below).
2. Issuer of the badge*
Eventually, this may be something like “School of Webcraft” or “Quinnipiac University.” For this initial round, it is likely to be “ICM” for the ones I am doing.
3. Version of this badge*
Date-time last updated the badge.
A unique ID for the badge is formed with #1/#2/#3, e.g., Quinnipiac University/Ph.D. in Social Computing/2011-12-25-7:00:00.
4. Badge Image*
For the purposes of standardization, I will say 250x250px PNG representing what the badge stands for.
5. Description
A textual description of what the badge represents. The idea is that it is reasonably brief–say, less than 200 words.
6. Recipient*
Who is it that is claiming the badge.
7. Nominator*
Who is it that nominated this person for the badge?
By default, any badge can be self-nominated. If for some reason you want to exclude this possibility, it could be listed as a requirement in section 9: E.g. “Candidate is nominated by someone other than themselves” or “Candidate may only be nominated by a member of the track team.”
8. When nominated*
Nomination timestamp.
9. Requirements & Evidence
This is the meat of the form. It includes 0 or more requirements, with links to evidence that those requirements were met. Each requirement includes a record like the following:
a) Textual description of the rubric for assessment. What needs to be shown, and how is an evaluator to decide whether it meets the standard. Outside examples may be linked, including former examples of successful badge earners.
b) Textual description or link to the evidence of assessment. (If a link, we’ll probably need to find a way to archive that link for posterity. Easier with some things than with others; e.g., video.)
c) Nominator’s comments on the work and why they think it qualifies.
d) Qualifications to endorse. For example, you might require that people have the badge they are endorsing, or that they have a badge that qualifies them as “instructors” in the skill (e.g., to get the “pilot” badge, you need to be endorsed by at least one person with the “pilot inspektor” badge, or to get a a QU-PhD badge you need endorsements from three people with the QU-Faculty badge). You might also require that people have a badge that verifies their identity. So if I have the Verisignature–ReallyMe badge, maybe it qualifies me to endorse more badges. C is a list of required badges–there may be more than one.
e) Number of qualified endorsers required. This could be zero or a thousand.
f) List of
1. Endorser name
2. Date of endorsement
3. Comments on endorsement
Note that there is a necessary and automatic exception here in the case where there do not exist in the world the number of qualified endorsers listed in D. In that case, you must be endorsed by as many qualified endorsers as currently exist. It is then clearly indicated that the badge is a [BOOTSTRAP]. At some future point you might want to re-try the badge to get a non-bootstrapped version, once there are enough potential endorsers.
10. Issued Date-Time* (or PENDING)
11. Expires Date-Time
12. Recipient’s Comments & Notes
13. List of community comments
An Example Badge Template / Form
Now certain elements of the above are part of the template of a badge. So, if I nominated someone for the “Good Discussion Summarizer” I would end up with a template that included:
“Good Discussion Summarizer”
The Human Fund
1999-8-14-09:00:00
[Some Cool Badge Art that I don’t have time to dummy up at the moment]
The good discussion summarizer is issued to someone who has demonstrated that she is consistently capable of summarizing a brainstorming or other discussion in an academic setting, both verbally and textually.
Recipient:
Nominator: Alex Halavais (2010-12-2-18:55:03)
Badge Requirements, Evidence, and Endorsements
1. Statements from three members of courses in which the recipient is enrolled attesting to her abilities to accurately summarize materials. Endorser must hold the “current student” badge. (No evidence beyond the endorsements required.)
Evidence: (NB: this would be left blank.)
Endorser:
Comments:
Endorser:
Comments
Endorser:
Comments:
2. Evaluation of a video of the candidate reviewing a discussion. Endorser must hold the “Good Discussion Summarizer” badge.
Evidence (Link to video or audio of summarization):
Endorser:
Comments:
3. Evaluation of a textual summary of the same discussion. Endorser must hold the “Good Discussion Summarizer” badge.
Evidence (Link or Pasted Text of a summary):
Endorser:
Comments:
Issued: PENDING
Expires: TBD
Candidate comments:
Community comments:
The nominator would fill out some of these, including, perhaps, being one of the endorsers.
Other Issues
The natural question is how would endorsers know to find the form? There are lots of possibilities here, including informal or direct invitations, and a queue of badge candidates needing assessment. But that is a solution that does not have to exist in the badge process itself (necessarily). The idea is to keep this piece as simple and light as possible.
Happy to hear any thoughts you might have. As I said, I’m going to take it for a test run in the Spring semester. I’ll likely just have people do it manually on the wiki, unless I find time over the break to code a simple form system that can handle the pieces. And I’ll point to the course and the badge description (as well as some of the early badges) as I write them up.
As a final note, this doesn’t in any way take away from the efforts of the Mozilla Badge Backpack approach. Indeed, one of the advantages to that system is that it might provide the opportunity for several dissimilar badge systems to work together. In this case, what would be passed along to the Backpack is just an image of the badge, its name, and a link back to the form that demonstrates how it was earned.
Share this: