Some say the third time’s a charm. But can—or should—the cliché hold up in higher ed?
Tallahassee Community College is finding out with a student-success technology known as early alerts. The idea is to gather information about a student based on factors like academic performance or attendance, and notify a professor or advisor to intervene with students who show early signs of struggle.
The North Florida college first introduced an early-alert system in the mid-2000s. But after two failed attempts and more than a decade later, the college finds itself yet again redesigning its early-alert platform and procedures.
“Faculty told me they hated it,” Feleccia Moore-Davis, provost and vice president for academic affairs at the college, said during a session at Achieving the Dream’s national conference last week. “They didn’t understand why we were doing it and they weren’t getting any feedback.”
Beginner’s (Un)Luck
The first early-alert effort at the college, which serves around 12,000 students, was built into the college’s existing student record system, called Integrow, and focused specifically on first-time college students and those in developmental-education courses. Faculty were asked to “flag” students having trouble and provide an overall status report at specified periods in the term. Students with two or more flags were then assigned to an advisor for follow-up.
The system wasn’t perfect. Referrals were sparse and typically came in bursts around the time reporting was due. And fall-to-fall retention for first-time college students showed no signs of improvement, dipping from 58 percent in 2007 to 53 percent in 2012.
Things got even more difficult around 2013, when the college began to feel the effects of a broader higher-ed trend of decreased student enrollment.
“In the fall of 2012, the institution experiences a 10 percent enrollment decline,” said Sheri Rowland, vice president of student affairs at Tallahassee Community College. “That was a huge hit financially.”
In response, the campus formed a task force on enrollment strategy, with a subgroup focused on retention. “If high school grads are declining [in college enrollment], then we need to keep everybody we have here,” said Rowland. “It’s less expensive to retain students when they get here than to replace those who are walking out the door.”
The retention subgroup was tasked with examining the existing early-alert system, and ultimately decided to replace the homegrown tool with a third-party product. The task force went with a widely-used early-alert provider, called Starfish, whose parent company Hobson’s claims is used at nearly 400 institutions.
Watch the EdSurge on-demand webinar, “How Analytics Can Support Student Success in Higher Ed,” sponsored by Salesforce.org.
Second Chances
Early on, the institution enacted changes to how the system would operate. The new early-alert effort was implemented for all students, no longer limited to a targeted group. To respond to the lack of use from the first system, faculty were also required to report attendance daily. And all of the incoming early alerts were fielded by two “pathway coaches.”
It wasn’t long before new challenges emerged, however. Having only two individuals oversee all flags at the institution “wasn’t a feasible model to sustain,” Rowland told EdSurge. Moore-Davis also stated at the session that the software could be unreliable: “I was hearing so many great things about [early-alerts], but every semester something crashed.”
Poor communication about the purpose or impact of the technology caused some faculty to resent the initiative as well.
“We didn’t tell [faculty] any results,” Moore-Davis said. “It was siloed—faculty raised the flag and someone else was supposed to contact the student, but faculty didn’t know if that person ever contacted them.” Even more than lacking feedback, the provost said the “top-down” approach of asking faculty to flag their students, and adding a new system to their workload, “was a mistake from the beginning.”
Moore-Davis admits one of the biggest failures in the process was leaving out their bottom-line: students. “[Students] saw the flags, but we failed to engage that population about what the flags meant,” she said. “When we implemented this, we forgot a significant part of our population.”
The provost said that the college “couldn’t measure success” using the technology and saw “very few measurable outcomes.” One measure was clear though: fall-to-fall retention rates for first-time college students again weren't improving, going from 57 percent in 2013 to 56 percent in 2016.
But because they spoke with other colleges who had seen success with the tool, it was unclear who or what was to blame. “There are a lot of other people using Starfish,” said Rowland. “We weren’t sure if the product failed, or if we did.”
So in 2016, the school began to reevaluate the system it had set up around Starfish. A group of researchers at the college spent the 2016-2017 academic year studying their own failures and best practices, and talking with other institutions that had implemented similar technology.
The outcome was a plan to redesign early alerts at Tallahassee Community College for a third time. Starting in Fall 2017, the college began issuing early alerts through TeamDynamix, a ticketing software the college already uses in areas on campus such as the IT help desk.
Going For Three
The college will save a sizeable amount by moving away from Starfish. Moore-Davis said with Starfish the school spent at least $73,000 annually. “We already own the TeamDynamix system, so it just costs us to expand its use for intervention purposes,” she explained, estimating that the new system will cost around $2,500 per year.
In addition to the software switch, the college is again overhauling the way faculty, advisors and students interact with early alerts. For starters, instructors no longer have to record attendance daily.
Also under the new ticketing system, faculty must document the ways they first tried to help students who showed signs of struggle. If the instructor's initial intervention is unsuccessful, he or she must report what effort was made before they are able to raise a flag to get additional help.
Then, when a flag is raised for a student, the faculty member must notify the student. In both previous efforts, Moore-Davis said students were unaware they were flagged until they received a call from their advisor. After creating a student focus group to get feedback on the process, she discovered that students had felt they were being unknowingly tracked and reported.
Now, flags are sent directly to a specific intervention team, such as financial aid, pathways coaches, tutoring or counseling, rather than an advisor who would then refer them out or may be unable to help with academic challenges. “When a student has an academic issue, they go to our learning commons intervention team,” explained Moore-Davis. “The same thing happens if the faculty thinks they need counseling—they get referred to counseling intervention team.”
Both Moore-Davis and Rowland acknowledge that the new system could run into its own unique challenges. But rather than ditching the idea entirely after two failed early-alert system attempts, they still cling to the idea. “I believe in the principles of early-alert, and I know there is a better way to get to where we want to go,” said Moore-Davis.
It remains to be seen if the latest iteration will prove more successful for Tallahassee Community College. And to complicate matters further, the school is using TeamDynamix as a temporary solution. The college is in the midst of transferring to Workday for its student records, and the officials don’t yet know if the system they are creating now will integrate when Workday officially launches in 2019.
In the meantime, the school continues to design around its past mistakes. “No faculty would disagree with the outcome you’re trying to get to. It’s disagreements about the process of getting there,” Moore-Davis said. “Understand the culture before you start making changes to it.”
