From Predictive to Proactive: How One Institution is Applying Data to Raise the Bar on Student Support
By Dr. S. David Vaillancourt, Chief Academic Officer, The College of Health Care Professions and Bruce Schneider, Vice President of Business Engineering, Ambassador Education Solutions
Most colleges and universities collect an abundance of data. Whether they use it is another story. Dissecting the facts to find real meaning in the information and applying those indicators to make a real impact can be a challenge.
Data analytics have the power to not only predict how a student can and will progress, but they also give institutions the means to be more proactive when it comes to student support and outreach. While data can tell us everything from where a student is from, to his or her personal and academic background, outside influences, education and career aspirations, and more, it is student behavioral data that is having the biggest impact on student support efforts and retention outcomes.
Student behavioral data can help us understand how the student fits within the institution, as well as how and how often he or she might need support to stay motivated and how they are progressing through their course. It is these factors and processes that are the primary drivers, or impediments, to student retention and outcomes.
Developing a predictive model of student behavioral data analytics enables institutions to forecast student challenges and outcomes.
Specifically, combining information that is readily accessible in a school’s LMS or SIS with eBook and publisher data, together with an aggregator of course materials, can provide a solid foundation for monitoring and positively impacting online student success and retention.
Putting a sensible academic success plan in place is key to both tracking activity data and making it actionable. Colleges and universities that do this can better predict and modify student behavior, increase student achievement and improve performance outcomes. The College of Health Care Professions (CHCP), is one school that is applying this new method of quantifying student engagement thru analysis of specific measurable behaviors. CHCP and its course materials partner, Ambassador Education Solutions (Ambassador), are early innovators in the use of data to raise the bar on student support.
Creating a learning path sequence
Measurable activities that demonstrate student engagement and progress, or lack thereof, can be the most significant drivers of student success and/or failure. Drilling down and understanding these student behaviors both in terms of individual activities and as part of the bigger picture and profile is what should fuel student outreach inquiries and frequency.
First, it is important to understand how and what are students doing academically. Observable activities, for example, the last day the student was active in the system, the amount of time spent in the system, time spent actively engaged with the course and materials, and more, can reveal how connected the student is to the course and how motivated he or she is to continue. Both qualitative metrics (i.e., which activities and sequence of activities) and quantifiable metrics (i.e., time on task, number of times on task, and frequency of being on task) are meaningful in assessing and predicting the student’s academic path.
The next step in the learning path sequence is to tie the academic data to student achievement. For example, looking at completed assignments can be an indicator of performance level. Grades on assessments can signify how the student’s performance aligns with the instructional objectives of the course. Finally, overall course completion can suggest how close the student is and how much progress is being made toward a degree, certification or licensure.
Turning insights into action
All of the measurable activity data mentioned above can be turned into actionable information that can serve as the foundation for student outreach.
There are essentially two types of institutional interventions: a model based on general best practices (foundational) and a model based on individual student behaviors (advanced).
In a general best practices approach, colleges and universities can apply standard outreach processes across a wide student audience. Here, most outreach is systematic, though the institution should determine exactly to what extent outreach is automated. These scalable processes engage similar types of students at consistent time frames. Additionally, the institution typically embraces continual modification of the learning environment as it impacts a significant number of students.
Under an individualized model, institutions need to assess how much outreach can be tailored, creating processes that take into account student-specific triggers and milestones (or lack thereof). Schools need to look at specific things that can be done to help students get over the hurdles they are facing or challenges that might be preventing them from being successful.
Conducting a student support study
CHCP set out to see to what degree it was capable of intervening both at a systemic and individualized level in order to foster student success in online learning. The study looked at undergraduate, online students from 2016 to 2017. CHCP measured data as it related to assignment submissions, last day active and performance levels over time, and the overall effect these behaviors had on monthly attrition.
The CHCP team concluded that best practices intervention strategies are ideal for two foundational needs. The first is optimizing the learning environment. The school’s data insights helped determine how curriculum impacted student behaviors, as well as the role of instructional technologies in the learning experience. The second is identifying appropriate learners for specific learning environments. Here, analytics looked at early performance in a course as an indicator of continued performance, as well as the role of entrance exams in matching students with courses and predicting progress.
As far as individualized student interventions, the CHCP team concluded that they needed to be behaviorally-based. Early indicators can identify areas of support that will make a difference to individual students. For example, usage (correlating factors) and engagement (casual factors) exemplify differing levels of usefulness in behavioral metrics.
Implementing a system of intervention
In establishing its student outreach goals and objectives, CHCP created a Daily Interim Report that tracks student behaviors and actions. It is designed to give advisors a quick reference to relevant student information. This includes standard information (i.e., name, phone number, email, start date and course title), as well as activity-based information (i.e., last day logged in, days since login, place logged in, and grade).
The Daily Interim Report has become the basis for which CHCP advisors create an outreach action plan.
Advisors evaluate the data on individual student behaviors and are able to implement a prescribed action plan of advising. Specifically, they have insights into the things they should do in the first week, second week, third week, and beyond based on those identified activities (logins, time spent, assignment completion, and more).
CHCP advisors are able to filter the students that they are responsible for supporting. For example, advisors who are working with new incoming students can filter by start date to focus on the newest students to help get acclimated.
When it comes to tracking student progression, advisors check student attendance, last day active, and days since active. Next, they review student grades. Generally, good grades indicate students are meeting the requirements while lower grades indicate that students probably do not complete all work or may need extra assistance.
While CHCP does not track every single assignment, triggers are in place that would cause the advisor to dig deeper and check a student’s individual assignments and grades. For example, if days since login hits a certain threshold, or if a grade drops significantly, then the advisor knows to make contact and/or check individual assignments.
The results of CHCP’s individualized outreach plan have shown meaningful improvements. By equipping advisors with the insights and triggers they needed, CHCP has sustained significant improvements in monthly attrition for 15 of 16 months.
Making a difference with inclusive access
Creating a system of intervention relies on data, and there are a variety of key insights that impact student success rates. One of the most intuitive ways for schools to get their own targeted, student support program off the ground is by measuring student behavioral activity as it relates to course materials engagement, and specifically eBook usage data. This intelligence is readily available and a terrific indicator of student activity and engagement.
One recent movement that has been instrumental in improving student outcomes through data has been inclusive access, which provides students with automatic access to digital materials in time for the first day of class, and in turn, gives institutions early indicators through eBook engagement metrics that tell them exactly how students are interacting with their course materials and the course itself.
What is inclusive access?
Under an inclusive access program, students are automatically given access to course materials before class starts, with the option to opt out. Students are charged a fee for access to required course materials either as part of their tuition/fees or as an itemized charge on their account.
Inclusive access programs powered by Ambassador connect students to their course materials in the most effective and affordable way possible. Because students benefit from volume discounts, typically they end up paying less for their course materials than if they had shopped on their own for a traditional print book or eBook. Learning companies partner with colleges and Ambassador to negotiate these discounts because they know the majority of students will purchase materials since they are automatically provided.
Inclusive access also provides a high level of convenience.
Students do not need to know what materials are required nor do they have to search elsewhere; instead the course materials are automatically delivered.
In the case of digital materials, eBooks and other digital resources are portable, easy to navigate and environmentally friendly.
With inclusive access, fewer students tend to go without course materials, and having access to course materials in time for the start of class can contribute to higher success rates as students are more prepared. Not to mention, the immediate availability of course materials gives advisors that same immediate access to data and engagement metrics. Inclusive access programs are taking shape around the country.
How is data from an inclusive access program used for personalized intervention?
Tracking course materials usage and engagement against student progress and success rates provides insights into how and how much students rely on their course materials.
Usage is one way of predicting outcomes, simply measuring how much students are accessing materials. “Predicting Course Outcomes with Digital Textbook Usage Data,” a study led by Iowa State’s Reynol Junco, examined engagement with online textbooks. Using data from over 200 students across 11 college courses, he found that the number of days students used the textbook could predict course performance, and that this was actually a better predictor than previous course grades.
Engagement with course materials is another way to predict outcomes. For example, relying on Ambassador’s student eBook usage reports, advisors can track a student’s note taking, highlighting and the amount of an eBook that has been read, all of which show a variety of specific behaviors that can also be indicative of future course performance. Advisors can also see if students returned their course materials and therefore check in with those students to determine if they are continuing in the course.
Access to eBook metrics through Ambassador’s reporting functionality, which assembles the usage and engagement data for the school, provides a clear indicator of student participation.
Advisors can see right away who is using and engaging with resources and who is not, thus creating a plan for early intervention. Data available through Ambassador also allows schools and publishers to compare student preferences between print and digital, as well as overall and weekly engagement by campus, device type, online or download reader. They are also able to identify student preferences when it comes to reading online or via download, as well as the most popular types of devices for accessing eBooks.
While systematic intervention plans play an important role in supporting students, personalizing the education experience with tailored support based on behavioral data increases opportunities for students to thrive. Student behavioral data provides insights that advisors can directly apply to student outreach, giving each individual the kind of support he or she needs, exactly when he or she needs it.
Founded by physicians, CHCP is 100-percent focused on healthcare education and training. Its accredited programs have been developing healthcare professionals for more than 25 years.
Ambassador creates and manages customized content integration programs that connect students with course materials in the most meaningful and effective way, driving down costs and improving outcomes.
DR. VAILLANCOURT has built a longstanding, credible career in higher education, both as an academic leader and an instructor. Prior to joining The College of Health Care Professions, he served as the Online Director of Education at Ultimate Medical Academy, Chairman of the National Math Bee, National Dean of the Online Division at ITT Technical Institute, and President of Virginia College Online. He has taught everything from Computer Network Engineering and Instructional Technology, to Critical Thinking and General Psychology. He holds Master’s and Doctorate degrees, both in Education.
Contact Information: Dr. S. David Vaillancourt // Chief Academic Officer // The College of Health Care Professions // 832-333-9028 // email@example.com // http://www.chcp.edu
MR. SCHNEIDER has more than 25 years experience developing, delivering and managing business technology solutions. He has worked with institutions and organizations to integrate solutions that allow them to operate more effectively and efficiently. He is an integral part of the team that leads the development of Ambassador’s integrated bookstore service solutions and helps customer institutions improve the quality of the student experience at all levels of the course material supply chain. He is a graduate of the State University of New York at Stony Brook.
Contact Information: Bruce Schneider // Vice President of Business Engineering // Ambassador Education Solutions // 631-770-1010 // firstname.lastname@example.org // www.ambassadored.com // Twitter: @ambassador_ed