Adaptive Learning 3.0: Increasing the Certification Rates of Aviation Maintenance Students Through Online Technologies

By Joel A. English, Vice President, Aviation Institute of Maintenance

With a network of 13 campuses across the United States and more than four decades of experience training aviation maintenance technicians, Aviation Institute of Maintenance (AIM) is the nation’s leading family of aviation maintenance schools. Today, AIM graduates make up 23% of the new graduates that the FAA certifies with its coveted Airframe and Powerplant certificate – nearly one in four newly-certified aircraft mechanics nationwide come from AIM’s 13 campuses.

But it hasn’t always been this way. In 2014, we realized that we had a problem. Despite high graduation rates, AIM students were like students at most aviation maintenance schools: they could graduate the program, but very few of them would sit for the FAA’s certification exams. Because only about one-third of our students became FAA-certified after graduation, many of our graduates went to work as aircraft fuelers, baggage handlers, aircraft deicers, and other low-paying, non-maintenance positions; others went to work as automotive technicians, general maintenance workers, and other positions not even in aviation. Almost all of our graduates earned jobs after school, but many of them were not in the direct field in which students were educated – working on aircraft in authentic, high-paying maintenance positions. Ultimately, our institutional accreditor, the Accrediting Commission of Career Schools and Colleges (ACCSC), began scrutinizing our lower-than-normal in-field placements of graduates.

We were forced to figure out why our graduates were strong enough students to finish the program but weren’t willing to sit for the FAA certification.

As we began our inquiry, we discovered some of the normal reasons students weren’t getting certified at high rates: The cost of the series of four certification exams was about $1,800, and new graduates couldn’t afford to sit for certification immediately. Many students bought into the evil myth of “taking some time off” to make some money with a loose plan to come back and get certified the next year. Students found work in industries other than aviation maintenance and simply never came back to test. These are normal reasons that create a vacuum effect out of school for the new graduate before they sit for certification. But the number one reason students seemed to not test immediately after graduation was not financial, but rather, it was psychological. Students didn’t feel prepared to take exams, even when they had just graduated from a program that did prepare them. Test anxiety stopped students from jumping straight into the certification process, and we found that even a one-month separation from the academic program significantly reduced the students’ confidence in their retention of the material and reduced the likelihood that they would ever seek certification.

Meanwhile, AIM’s industry partners were demanding more certified graduates to fill workforce shortages. For example, Boeing, AIM’s single largest employer over the past three years, projects the need for 193,000 certified aviation mechanics in North America and 769,000 worldwide in the next 20 years.¹ AIM needed a solution that would get its graduates not only prepared for the certification exams, but also confident enough to sit for and pass the critical FAA certification exams. And, since it was clear that students were quick to leave us after graduation, we found that we needed to add this exam preparation program into the curriculum of the program.

As part of our 2016 program overhaul, we added a Capstone course to the end of our program, forcing exam preparation into the educational process, and we searched for an adaptive online learning solution that would complement other changes we were making to our processes and curriculum. We found a partner in Fulcrum Labs, an Adaptive 3.0 Learning technology with a mission to turn students into learners and turn learners into confident subject matter masters.

What’s Adaptive 3.0?

First, the basics. We’ve all seen – and maybe taken – online courses. And most online courses stink. Can we agree on this? Since the early 1900s when higher education started putting content onto the internet, online coursework has been mostly reading. Read a lecture. Then read a case study. Then read a discussion question and read peers talking about that discussion question. Then read quiz questions and click answers. Then read the teacher’s response to your answers. Online courses are so often just reading. And guess what: Students hate reading.

Perhaps a more fair way to say this is: Most online courses focus on auditory learning – providing text as the primary instructional and learning method. Most students are not auditory learners, and because of the complexity of bringing kinesthetic and visual learning into the online classroom, most online courses just don’t. But in the late 2000s, while some course developers began using more video instruction to break up the monotony of online courses, more advanced courseware developers began working on an online theory called Adaptive Learning.

Adaptive learning embeds questions, puzzles, and student responsiveness right along with the instructional content, and then it responds appropriately to the student in relation to their demonstrated competency with the material.

That is, while a traditional online course might provide 10 pages of text for a student to read and just assume the student will read every word of it, an adaptive course might provide a paragraph or a video, ask 3-4 questions, and either provide more instructional content if the student missed some of the questions, or skip over the rest of the material completely if the student got everything right. Adaptive learning means that the online course itself will assess how much a student knows about a given subject on the fly, and then give the student only the content they need, based on their demonstrated competency.

The course that Fulcrum Labs built for AIM is based on this technology, combining multi-modal instruction with top quality production value. The course assesses a student’s competency within all of the 245 learning objectives and provides the instructional content that the individual student needs to master the FFA certification exams. All learning content is presented through visual instructional video, auditory reading, and kinesthetic hands-on activities, and the student is assessed based on the learning methodology they choose².

Figure 1: Examples of the multimodal instruction, production value and interactive, formative assessments of the AIM program within the Fulcrum Platform

In the past decade, adaptive learning has advanced from a basic branching approach that made adaptations based on decision trees and pre-diagnostics (Adaptive 1.0) to an algorithmic approach that made limited adaptations based on pre-determined algorithms (Adaptive 2.0). But what was missing from each of these eras was artificial intelligence. The origins of adaptive learning can be found in the artificial intelligence movement of the 1970s, but none of the subsequent adaptive learning approaches or technologies actually leveraged AI in any way. And the reason for this is simple – AI technology wasn’t developed or accessible enough to be integrated into adaptive learning technology.

Today, AI is everywhere. And armed with AI and advanced data analytics, the market is moving ever closer to its original intention: that a computer might mimic the personalization of a one-on-one instructor³ , adjusting to the needs of each individual learner at scale. In fact, the latest generation of adaptive learning, Adaptive Learning 3.0, moves beyond previous iterations by leveraging AI and machine learning (ML) to deliver real-time adaptations that replicate the one-on-one instructor experience and dramatically improve learning outcomes.

The Benefits of Adaptive 3.0

Adaptive Learning 3.0 powers a more efficient, effective learning experience by analyzing performance and behavioral metrics of each individual within the learning environment. This allows a single online course to deliver:

• Complex, real-time adaptations based on learner/cohort performance and behavior to target each learner’s optimal challenge level to shore up their knowledge gaps. “Optimal challenge” predicts what level of instruction is clever enough to encourage the student to keep trying and moving forward, but not too much instruction that the student becomes bored or fatigued. A good Adaptive 3.0 course feels more like a game than a class lecture.
• Data-driven, personalized hints, feedback, remediation and knowledge reinforcement to keep learners engaged, motivated and in control of their learning experience. The student needs to feel coached, encouraged, even patted on the back for successful attempts.
• Predictive, built-in, data-based insights into future knowledge application, including identifying those who are at risk of not being able to apply the learning. The course must talk back to the student and scream out to the instructor to let them know which students are likely to get stalled out because of frustration or lack of understanding within the course.
• Comprehensive application-level mastery of skills and knowledge through a competency-based microlearning approach. Micro-learning means providing short bursts of content through video, reading, or quizzing – and then scaffolding onto those skills with subsequent “lecturettes” and exercises.
• Opportunities for learner autonomy and self-direction, including various learning modalities, self-remediation tools (such as an adaptive “Pathway to Mastery”), and accessibility through desktop, mobile, and tablet devices.
• Content performance analytics, which can be used to improve the efficacy and integrity of a course. That’s right: The student isn’t the only one being tested for their competence level. In Adaptive 3.0, the course is actually analyzing itself! When the course notices that a question or problem is being answered incorrectly by a majority of students, it kicks that question out of rotation and submits the question to the system administrator to see if the question is wrong, unclear, or too confusing for the average student to grasp.

This powerful combination of Adaptive 3.0 features is a game changer for education providers, and it was exactly the type of solution that AIM was looking for.

Adaptive 3.0 in action at AIM

Together AIM and Fulcrum Labs designed a certification preparation program to support AIM student performance by fully-prepping them for the FAA mechanic certification exams and helping them build the confidence necessary to sit for the exam. As a result of the initial program, AIM was able to improve its pass rates by 14%. This is key because the FAA requires aviation maintenance schools to maintain a minimum pass rate of its recent graduates.

But more than just improving our pass rate on the exams, we needed to improve our attempt rate.

Remember, we were not suffering from too many students failing the exams; rather, we were suffering from not enough students having the courage and confidence to take the exams. Students exhibited higher confidence levels after they had taken our Fulcrum Labs exam preparation course, and they were more willing to sit for the examination as they finished each main section of our program. This is supported by a 25% increase in the number of graduates who felt confident enough in their knowledge and abilities to sit for the FAA exams after completing the test prep program.

The number of graduates becoming certified soon after graduation doubled from 358 in 2014 to 715 in 2018. And as a result, we improved our market share of licensees: in 2014, only 5.3% of new graduates around the country getting licensed by the FAA came from AIM. By 2018, 22% of recently graduated students getting certified by the FAA nationwide came from AIM. And in-field placement immediately followed. In 2017, only an average of 53% of AIM graduates were placed in authentic aviation maintenance positions. In 2019, 72% of AIMs graduates were placed in aviation maintenance positions, with some of AIM’s campuses being as high as 81%. Note the steady increase of in-field placements that the various campus have experienced over the past three years:

Today, the AIM and Fulcrum Labs continue to improve the efficacy of this course. For example, we have conducted a rigorous review of the program’s content, leveraging Fulcrum’s content performance analytics. (The platform uses state-of-the-art statistical techniques to evaluate the contributions of individual questions to students’ overall confidence and mastery, flagging problematic questions for instructor attention.) Rather than simply fixing problematic content, the companies have collaborated to “deepen the adaptive stack” – adjusting the problematic assessments and adding new ones at varying levels of Bloom’s Taxonomy. Therefore, students who struggle with content now have even more opportunities to practice the content both in that particular section and later on as memory boosters.

Additionally, based on performance increases for graduating students, we adjusted our training strategy to provide students access to Fulcrum’s training platform earlier in the learning process (rather than strictly as an FAA test-prep tool for students after they completed all coursework). Today, rather than waiting until the final Capstone course at the end of the two-year program to give students access to the Fulcrum course, we now give students access as soon as they finish the first major section of our program. This new strategy has allowed our students to build their confidence, master the FAA certification content, and take each individual certification exam as they progress through the coursework, rather than just at the end of the two-year program. The result has been that students are taking the FAA certification as they go, which greatly increases the chance that they will finish all four of the FAA’s certification exams and become fully-certified immediately upon graduation.

AIM’s work is helping to alleviate the national shortage of aviation mechanics – a clear and persistent skills gap the aviation industry is facing. It is also making a real difference in the lives of AIM graduates in terms of job opportunities and long-term financial stability. We have found that graduates can get pretty good jobs in aviation without a license, but the opportunities for certified graduates are wider, more robust, and more lucrative. Our employers have made it clear that they will hire anyone who holds the certification and pay them much more money than non-certified mechanics, so it is our duty to force the issue and lead our students to become certified immediately upon graduation.

Our goal is to certify over 25% of all aircraft mechanics annually by 2021, and we are getting very close to achieving this goal.

This online course, produced on Adaptive 3.0 technology, is a critical tool for our certification rates. Our course has won several industry awards, including 13 Brandon Halls Group Awards, ATD’s 2019 Learning in Practice Award, a 2019 Learning in Practice Partnership Gold Award from CLO Magazine and two EdTech Digest “Cool Tool” awards. Awards are great, but students becoming certified, keeping us all safely in the air, and making more money than their instructors is even greater.

References

1. The Boeing Pilot and Technician Report can be found at https://www.boeing.com/commercial/market/pilot-technician-outlook/
2. A brief introduction to the course can be found at https://www.youtube.com/watch?v=HerKfhM5Tjo
3. One-on-one instruction can improve performance by 2 standard deviations. In other words, it can help the C student become an A student. http://www.ascd.org/ASCD/pdf/journals/ed_lead/el_198405_bloom.pdf

DR. JOEL ENGLISH is the Vice President of Operations of Centura College, Aviation Institute of Maintenance, and Tidewater Tech, where he supervises all operations over the 19 campuses across the United States. Dr. English is a board member for the Aviation Technician Education Council (ATEC) and Central States Private Education Network (CSPEN). He previously served as a Commissioner for the Accrediting Commission of Career Schools and Colleges (ACCSC) for six years, and he was the Chairman of the Board for two of those years.

In previous positions, Dr. English served as the CEO for the Ohio, Illinois, Colorado, and Miami Media Schools, a family of institutions dedicated to technical education in radio, television, and internet media broadcasting. He oversaw distance learning and school operations at several campuses as a Regional Director and Executive Director at Centura College, and he was formerly an Assistant Professor of English and Distance Learning at Old Dominion University. Dr. English is the author of “Plugged In: Succeeding as an Online Learner.” He holds a Ph.D. in Rhetoric and Composition from Ball State University, an M.A. in Technical and Expository Writing, and a B.A. in English from the University of Arkansas at Little Rock.