Over 500 education technologists from companies and districts gathered last week in Denver for the IMS Learning Impact meeting, hosted by the IMS Global Consortium. This annual event aims to accelerate the development and adoption of data standards that allow technologies used in schools to “talk” to one another. By enabling these connections, IMS, the nonprofit group that organized this event, hopes to create a plug-and play-educational ecosystem where tools can freely and seamlessly share information about student learning that educators can use to drive better learning outcomes.
One of the most exciting and well-attended sessions covered the new IMS Caliper analytics standard, which can capture and share data on student activities in learning apps and software—thus giving teachers and developers a way to measure and glean insights on how students engage with digital tools.
Why is that important? Today, a teacher can see if a student is not doing an assignment or struggling to complete a worksheet. Imagine how much more helpful that data could be if he or she could see how long, or how many attempts, each student made. That information about how much effort was made could inform educators to follow-up more appropriately.
How Caliper Enables a Deeper Level of Data Sharing
You don’t have to be a technologist, teacher, or even work in a school, to appreciate the conveniences that data sharing enables. For example, a educator who is making a restaurant reservation on OpenTable may be delighted to see if their Facebook friends have eaten at that restaurant. Or, those using Spotify may be interested to see what their friends are listening to. This sharing of data between different social apps is possible because they have a standard way of exchanging this data.
So far, data sharing capabilities are fairly limited in today’s education technology tools, because there is no standard agreement on how to share data between them. In a different era where fewer online services and tools were available in schools, there was less digital activity to capture and therefore not as much need for data sharing. However, with the recent proliferation of Chromebooks and iPads into classrooms, there is more student activity in the classroom that has moved online. And that creates data that can be shared.
There have been efforts to create a data standard for measuring student activity before, mostly notably xAPI (also called “Tin Can”). Today, a growing number of edtech companies are also signing on to use Caliper, which offers improved data architecture and ease of implementation.
Caliper promises to do something new and exciting: It can make available sharing the data of students’ online activities, such as students’
- answers to quizzes;
- responses on worksheets;
- views of videos;
- clicks on interactive resources
It can also capture the exact start and stop times of each of the above activities.
In technical terms, IMS defines the above as “learning activities” or “learning events.” They are quite granular and detailed, with what are called “actors, actions and objects.” For example, a learning activity is represented like this: “Student A (actor) started (action) Quiz B (object.)”
So if a student does an online worksheet or takes a quiz, it will capture and timestamp each click of the mouse, allowing each activity to be calculated. For blended learning resources such as videos, it can capture how much of the video the student watched. That means if a teacher assigns a Quizlet and a Smithsonian video to her students, he or she can then see how long students watched each one. This information can be particularly helpful if a teacher wants to see which video on the same topic, but from different providers, is more engaging for students.
Other Potential Uses of Caliper
Let’s examine some more opportunities for data sharing via Caliper, and what they might mean for teachers:
- Most teachers have an electronic gradebook via their school’s learning management system. But these resemble the old paper gradebooks of the past, displaying only grades. Imagine if it was enhanced so that teachers could see the activities each student did before their grades were assigned. What if there were a way to correlate students’ engagement with a digital curriculum or blended learning program to their grades?
- This data could also let the teachers organize class work for students based on their online activities. For example, teachers might be interested looking at a group of the students that spent extra—or less—time on an assignment.
- Today, most teachers simply Google around to find videos or PDFs, usually referring to search result rankings to find out which resources are the best. Aggregate data sharing via Caliper would allow any provider to communicate to users which resources are most engaging, or whose usage correlates to improvements in learning outcomes. (OpenEd published a new white paper and sample code at IMS showing how any edtech company can use Caliper to measure efficacy of these blended learning resources.)
- States and districts get results of summative assessments periodically—usually a couple times a year—but it is rare for them to get more frequently. More and more, teachers will assign a formative quiz in Google Classroom (or the Apple or Microsoft equivalents), but that data stays where it is. Caliper would allow desired data to be analyzed by district administration to measure how students are doing more frequently.
What Caliper Needs to Work
Like any data standard, Caliper’s potential can only be realized if enough online services—both those developed by companies and those used in schools—adopt it. So far the Caliper working group includes over a dozen organizations, including McGraw-Hill, Blackboard, ACT, D2L, IBM, Pacific Metrics, and Instructure.
Also participating are University of Michigan, UC Berkeley, UCLA, Oregon State, and UBC. Two of the most active K-12 districts at IMS Learning Impact are Houston Independent School District and Fulton County schools in Georgia.
The more usage data that is captured, the better we can understand the efficacy of educational resources. This data is stored in what is called a Learning Record Store. (OpenEd has published Callisto; another is published by Apereo.)
Schools and districts have a role to play as well. Often, when they issue requests for proposals to purchase edtech products, data integration capabilities are not at their top of mind. Hopefully that will change as administrators recognize the potential that data interoperability enables (as well as the headaches if tools can’t share data).
The Future
As schools and districts deploy more and more technology tools from different vendors, they may find the vendors that comply with Caliper are able to provide more insights to student performance than those who do not. They will also have better accountability, understand what behaviors lead to better learning outcomes, and have early warning systems.
The IMS Global consortium and its members will need to continue to improve the spec for ease of use, ensure rigorous oversight and protections for student data privacy, and showcase the best ways for data to be shared.
Caliper represents a step forward to addressing the unresolved, elephant-in-the-room question about any piece of education technology: Does it work? By providing a systematic way of collecting and sharing the extent to which these tools are used, educators, researchers, technologists and parents can all take a closer step to getting an answer.
