Educators shifting to competency-based models often tout to the power of allowing students to master a topic by taking a test again and again until they do well, rather than failing them if they bomb the first time. The philosophy is that such repeated testing will make sure they get to a level where the material truly sinks in.
But new, externally audited, efficacy research from Pearson about one of the company’s apps shows a significant correlation between increased retesting and lower overall course performance.
The results of the study pose big questions for reform-minded educators who advocate for students to have multiple chances on exams. The new research suggests that far from mastering concepts, students may simply be retaking quizzes on education technology platforms until their score increases, which may actually hurt their overall learning progression.
The Pearson Study
In what was described as an effort to build transparency around their products, Pearson’s CEO, John Fallon, issued a directive for the company to run externally audited public efficacy tests on the firm’s edtech applications. The MyLab Math report is one of 12 efficacy studies the company released about one month ago.
The research, done in 2015, includes data from students taking remedial math courses on three different campuses of Arizona State University. In the courses, students used a tool called MyLab Math, a teaching and learning platform that uses digital content with other learning aids. It is normally used to help lagging students build math skills. Students using MyLab Math can also use QuizMe, an application that lets them demonstrate mastery of a learning objective. It can be done before or after practice assignments.
Typically, a student who doesn't demonstrate that they've mastered the learning objectives through their QuizMe performance is taken back to a “study plan” where they have other opportunities to practice the learning objective. If they show mastery, by getting a certain score on QuizMe, then they can skip the additional practice. But Pearson researchers found that students taking the QuizMe tests multiple times to get higher scores actually earned lower course grades in the end.
“We scratched our heads a little bit about that finding,” explains Emily Lai, the vice president of Impact Evaluation at Pearson. “But one possible explanation could be that the lower-skilled learners were simply repeating those QuizMe activities over and over again until they were able to achieve a relatively high score even without fully mastering the concepts."
Implications For Educators
Lai says that she is not sure whether this finding is a result of a flawed product design or if the way the app was used in in the classroom was ineffective.
However, she notes that the company’s product designers are looking into the results, and stresses that educators using these products should consider their teaching styles with technology when such outcomes are apparent.
“Instructors can use this information to think about how they're using MyLab Math in their class,” Lai continues. “For example, they may want to encourage their students to do more practice questions before taking QuizMe for learning objectives.”
As personalized and other competency-based learning models spread, education technology vendors are adding more applications similar to QuizMe to their product lines. Lai notes that Pearson already has multiple products with retesting features, and this research is a chance for the company to share its finding with their product team to iterate their designs.
“It is something we're looking into as it relates to things like giving students repeated opportunities to share what they know and can do,” says Lai.
