The first time that Stanford assistant professor Paulo Blikstein studied the impact of maker activities on students' learning—way back in the early 2000’s—the Maker movement wasn't known as such yet. There were researchers out there interested who were in the concept, especially at MIT, but there wasn't a common language to discuss its pedagogical principles and the cost of technology was still a barrier to experimentation in classrooms.
Back then, Blikstein was busy organizing Maker workshops for low-income students in Heliópolis, one of largest and, at that time, most violent Brazilian slums. With cutting edge technologies such as robotic kits and electronic circuits, as well as simple materials like cardboard and ribbons, he challenged the kids to solve real-world problems. In a 2011 TEDx talk, he shared, "They start to look at technology not as something magical, but as something that can be used to improve the life of others."
Fast forward to 2016 in Palo Alto, CA, 6,500 miles north of Sao Paulo. Blikstein is now the director of FabLearn, a program that engages a community of educators in disseminating best practices and resources around how to integrate the principles of Maker ed into formal K-12 learning. He has also become an advocate for equity and diversity in the Maker movement, studying what works and doesn't work in the project-based learning approach.
In an interview with EdSurge, Blikstein talks about efficacy in the Maker movement. He argues that success depends on what is being measured, that assessments should focus on the process and not products, and that school districts have finally embraced the pedagogy of makerspaces. The problem now, he adds, is to prove that they are scalable and sustainable.
How do we measure success in a makerspace?
The first thing to do is define what success means. If it means higher test scores, then we need one kind of measurement. If success means engaging more kids in STEM disciplines, we need a different metric. And if success instead means that kids are happier at school and want to spend more time there, we need yet a third measurement tool.
However, there are some metrics that are recipes for failure. One of them is to rely too much on test scores. There is math in music, for example, but learning music doesn't necessarily increase math scores.
What's a better form of assessment than test scores?
The kinds of assessments that we have right now are not very adequate for maker activities because they measure product, and not process.
Tests are good for pre- and post- measurements; we give a test at the beginning, another at the end and see how much students have improved. That's what we call a product-based, outcome-based assessment. However, most of the learning that happens in makerspaces takes place in the process.
So, how can we measure process?
We have been building a research methodology called "multimodal learning analytics." This is a data collection technique that uses eye tracking, gesture tracking, bio sensing, log file analysis, click chain analysis, and visual analysis. We collect all this data to find interesting patterns in how kids learn.
We use this technique because it's hard to detect a lot of these patterns, even with careful observation. Imagine if you have a room with 20 students and you want to observe what each individual child is doing step-by-step. Without technology, it would very labor intensive.
You mentioned eye trackers, gesture trackers. Not all teacher will have access to this technology in their classrooms...
We've done a couple of studies, the results of which can help teachers.
For example, we analyzed students in hands-on activities, like building a robot, and concluded that kids who are more active—the ones in whom we detected more hand movements and gestures—tend to learn more, which is what we expected. But we were surprised to find that an even better predictor of learning was not how much students were moving, but how many times they alternated between being active and being passive in an activity.
Our takeaway is applicable to all classrooms: build moments of reflection into your hands-on activities.
Any other recommendation about hands-on activities?
In another study, we had students working in pairs on maker activities. We identified one as the "driver" of the activity—in control of the computer, the keyboard, etc.—and the other as the "passenger."
We paired low and high GPA students in different types of group—pairs of high GPA students, pairs of low GPA students and pairs with one of each. When we had a pair of two high GPA students, of course they performed well. When we had a pair of low GPA students, they tended not to perform well. But when we paired a high GPA student with a low GPA student, and mixed up the roles driver and passenger, we found something unexpected. The groups that had the low GPA student as the driver performed almost as well as the groups of two high GPA students. The groups in which the high GPA student was the driver and the low GPA was the passenger performed almost as poorly as the groups of two low GPA students.
The message to teachers is: when you are creating groups in a makerspace, try having the "weaker" student be in control of the computer, Legos, robotic kit, etc. with the help of a "stronger” student.
What are the most common misconceptions around maker activities?
In the first large scale experiment in makerspaces, we surveyed more than 1,085 students from sixth to twelfth grades in the US, Mexico and Australia, and Finland. Our work was sponsored by the National Science Foundation, and will be published this October in the Journal of Engineering Education. We were looking at career choice, familiarity with technology and other aspects. Our main finding was that even though kids around the world are very familiar with technology, most don't use it to create products in the physical world but to consume them in the digital one. They play games and use Facebook, but only 30% use technology to create things in the digital world—such as videos, music, or a blog. And only about 15% of the student surveyed use technology to create physical products, such as an electronic circuit or a little robot.
It's as if they are going to the kitchen to eat, but they are not cooking.
Right. They just go to the kitchen and heat the food in the microwave, but they don't learn how to cook their own food. Because those kids are not familiar with maker technology, they need a lot of help. Without it, they get easily discouraged and run the risk of confirming the stereotypes they already have about themselves: "I can't do this; I'm not good at engineering."
When you are talk to school districts, do you still have to convince them that project-based learning, constructivism and making are good pedagogical approaches?
The discussion has been less and less about the value of the activities and more about how to scale and sustain them. That's great news because we have less resistance to the idea of Maker Ed. But we still have to build careful models of implementation so that we can survive the hype.
