With course materials averaging around $1,200 per year, many colleges over the past decade have adopted open educational resources (OER) to cut costs for students. Since then, a number of studies across many institution types have found that OER—freely available educational materials that can be downloaded, edited and shared—save money and show that students learn just as much using OER as with commercial textbooks. (One review offers evidence that students using OER as their primary course material sometimes perform better.)
This body of research is impressive, given the general dearth of research on and accounting for what students actually learn in college. Unfortunately, however, there is still a huge unknown about OER: We have not found if these materials foster similar learning outcomes for all students, especially traditionally underserved populations including low-income students, adult learners and students of color.
Consider a 2015 study—the largest of its kind, by Lane Fischer, John Hilton III, T. Jared Robinson and David A. Wiley—which looked at almost 5,000 undergraduates using OER at ten colleges and universities. Authors of the report analyzed all students’ course completion, course grade and enrollment intensity during two consecutive semesters. The researchers found that students who used OER generally performed as well or better than students who used traditional textbooks, and that students assigned OER also took significantly more credits the following semester.
While the results of this study are good news, they don’t tell all. That’s because the researchers used a technique called propensity score matching (PSM), a statistical method that reduces the impact certain factors such as age, gender, or race may have on the effectiveness of an intervention—in this case OER—under study.
In the case of Fischer’s study, PSM was used to ensure that students in the group assigned OER were similar to students in the group assigned a traditional textbook. Students’ age, gender and minority status were the factors the researchers tried to control for in each student group. While PSM allowed the researchers to study differences in outcomes between students who were assigned OER vs. commercial content, this technique masks potentially disparate impacts based on age, race, gender or socioeconomic status.
This isn’t uncommon. Most studies on OER either use this technique, don’t control for students’ characteristics at all, or simply don’t disaggregate the outcomes or perceptions of OER by student demographics. Therefore, while the advantages of OER on student performance may be difficult to dispute, there is less clarity on whether student outcomes are similar across all student populations.
Specifically, researchers are currently unable to tell us whether OER are working as well for women, students of color, low-income students and other student populations. It’s important not to assume that they are. A 2011 review of literature on online education revealed underprepared and low-income students struggle when taking classes online. One reason? These students may not have access to the technology required to take online courses without hiccups. Additionally, low-income students’ performance may take a hit because online learning can demand a degree of preparation and self-initiated learning that underprepared students may still have to learn.
This is a significant limitation in available research on OER. Those who study innovative educational practices have a responsibility to unearth how such strategies improve—or don’t improve—the success of every student. So as colleges become more diverse, disaggregated performance data will be essential to understanding if our efforts are having their desired impact for all students.
Disaggregated performance data isn’t just helpful for researchers, though: it may increase the adoption of OER if there is evidence to show that these resources can help even the most marginalized students succeed. Several states and institutions are already heading in this direction, like in New York state where there is a proposed $8 million to allow public institutions to provide OER to students, or how Salt Lake Community College’s OPEN SLCC initiative (which offers 600 OER-based sections) has helped 35,000 students save $3 million in three years.
Yet across the country, OER still haven’t penetrated most institutions, despite growing interest in the approach. Robust research can help in these endeavors. Disaggregated performance data could even be another element OER advocates make open, while further distinguishing themselves from traditional publishers who currently keep this important information closed.
John Hilton III, a research professor at Brigham Young University in Utah, and co-author of the aforementioned study, agrees that disaggregated data is needed. Hilton confirmed in an interview that “it seems like an important next step.”
But, Hilton also agrees that we need more research on the kinds of texts faculty use in class and their impact on students’ learning. “For decades, and longer, we’ve had an explicit assumption that students read the assigned textbooks and that these textbooks make a difference in students’ learning,” Hilton said in an interview. “Is that assumption true today? That’s something we need to keep our eye on.”
Affordability is an important first step to ensuring equity, and OER address cost by making course materials available to all students regardless of their ability to pay. If OER researchers are able to show learning gains for historically underserved students, this would be even more significant—especially for institutions where low-income, adult, and students of color persist and complete at lower rates.
If OER proponents truly want to promote equitable learning opportunities for all students, researchers should go beyond making sure they have the same outcomes as proprietary texts and evaluate whether these materials are working for the students who need the most support.
Moving forward, it will be crucial to measure whether students from disadvantaged backgrounds excel as much as students who aren’t disadvantaged. A commitment to disaggregate performance data are needed to confirm that OER are an academic plus for all.
