Rebuilding a High-Traffic LMS Platform Mid-Flight

When an enterprise web platform reaches a certain scale, the standard advice for handling architectural debt (freeze feature development, spin up a fresh site, and migrate) stops being an option. At Tag1 Consulting, we maintain the e-learning infrastructure behind two of the largest labor organizations in the United States: the American Federation of Teachers (AFT) and the American Federation of Labor and Congress of Industrial Organizations, AFL-CIO.

Share My Lesson (SML) is AFT's resource hub. It has more than two million registered educators who collaborate on classroom materials and participate in on-demand professional development webinars. Meanwhile, on dedicated platforms like AFT eLearning and the AFL-CIO's LaborLab, union members and stewards across the country complete leadership tracks, health and safety courses, and workplace training.

These platforms carry years of member training records, completion histories, and certificate logs. They also place very different technical demands on the stack: Share My Lesson needed e-learning added to a large existing content repository, while the union portals needed deep, multi-tenant course structures.

By late 2022, the ground beneath us was giving way. Our underlying system, built on the legacy Opigno LMS distribution, was buckling under years of accumulated technical debt.

This is the technical story of how we pulled 100,000 lines of legacy code out of contrib, extracted a decoupled navigation engine over 18 months, survived a major Group module entity overhaul, ran a dual-engine hybrid repository, and ultimately built the open source Drupal LMS module.

The Breaking Point: How Distributions Become Traps

When Tag1 started this e-learning project in 2017, adopting Opigno's early Drupal 8 alpha seemed like a sensible place to start. It gave us ready-made entities for courses, modules, and activities. Crucially, it anchored class memberships on the new Group module (1.0-RC at the time).

We're grateful to Opigno for pioneering e-learning in Drupal. It proved that Drupal's entity system could model a school curriculum. But as Drupal matured through versions 8, 9, and 10, the monolithic distribution architecture began working against us.

In Drupal 7, full-stack distributions were the gold standard. In modern Drupal, they can easily become an architectural straitjacket. Because Opigno was a distribution, it arrived as an all-or-nothing monolith: its own opinionated theme, its own administrative UI, and dozens of deeply interconnected submodules.

When I joined the project in 2021 to help guide the Drupal 8-to-9 upgrade, the cracks were widening. Unused submodules couldn't be disabled without cascading fatal errors. Critical features relied on dangerous runtime anti-patterns, such as serializing raw PHP objects into user dashboard records, which reliably corrupted whenever a module updated. Worst of all, our team was writing high-quality patches to fix infinite recursion and data-loss edge cases, but our fixes sat unreviewed in upstream queues for months, then years. Every upstream release became an event we dreaded, threatening to overwrite our hotfixes or introduce new regressions.

The dealbreaker came when AFT wanted to bring online learning directly to Share My Lesson (SML).

SML was already a high-traffic Drupal site with its own custom theme, established content types, and over 2 million registered educators. You cannot install a distribution on top of an existing Drupal database. We didn't need an entire operating system; we needed a composable module.

Every hour spent rebasing patches and fighting upstream distribution baggage was an hour that could have been spent on extending educational features. Our clients deserved better, and that required a change of course.

Step 1: Taking Ownership (Un-Vendoring 100,000 Lines of Code)

The first step of rebuilding mid-flight didn't feel like brilliant architecture. It felt terrifying.

On September 14, 2022, my colleague Nat Catchpole did the fork. He went into composer.json, removed the constraints for opigno_learning_path and opigno_module, and copied 665 files and over 100,000 lines of code directly from web/modules/contrib/ into web/modules/custom/.

Committing 100,000 lines of someone else's procedural legacy code to your own custom folder goes against every instinct you have as an open source engineer. But it transformed our day-to-day work and gave us hope for a cleaner environment.

Suddenly, we were no longer passive consumers waiting on an upstream release cycle that wasn't responding. We were done maintaining a fragile, multi-layered stack of Composer patches. Within 48 hours of that commit, I was deep inside the navigation internals (LpSteps.php), hunting down and fixing recursive nesting bugs and FALSE->id() fatal errors that had haunted our student completion callbacks for months.

Step 2: Extracting the Engine into TrainingManager

Once the code lived in web/modules/custom/, the real surgery began.

In the legacy distribution, course navigation, access gatekeeping, and scoring calculations were scattered across the codebase. Some logic was trapped inside Form API submit handlers; some lived in controller redirects; some was buried in procedural helper functions inside .module files. If a client asked to change how passing grades were calculated or requested a "no-fail" revisit mode, I had to hunt through half a dozen disconnected files to make sure I didn't break a hidden dependency.

We needed a presentation-agnostic state machine.

In October 2022, Nat planted the seed: he committed a tiny, 51-line service class called TrainingManager.php with a single query method. Over the next 18 months, across more than 70 targeted commits, I used that file as my anchor.

Whenever I touched a legacy feature, I extracted its business logic from the controllers and forms, refactored it into clean, testable object-oriented PHP, and moved it into TrainingManager. Method by method, commit by commit, I taught the service to manage the entire student lifecycle through five responsibilities:

1. Progress Tracking: Loading and creating unambiguous status records across courses, lessons, and activities.

2. Navigation State: Deciding whether a user can advance, whether lesson rules allow backward navigation, and whether jumping ahead is permitted.

3. Lifecycle Gatekeeping: Initializing and updating lesson attempt records and verifying prerequisite lesson completions.

4. Scoring and Routing: Calculating weighted scores, evaluating mandatory vs. optional modules, and routing open-text submissions to instructor grading queues.

5. Maintenance Utilities: Providing administrative methods to recalculate out-of-sync student attempts or safely reset training progress on courses that needed updates.

In the legacy codebase, navigation and tracking had sprawled across nearly 6,000 lines of tangled procedural code between opigno_learning_path.module and opigno_module.module.

Consolidated into a pure service using dependency injection, the same logic took roughly 1,000 lines of clean PHP in the TrainingManager service.

More importantly, our forms and controllers became thin, dumb delegates. They no longer made business decisions; they asked the TrainingManager what should happen next. Because the engine was now decoupled from Drupal's render arrays and Form API, it was inherently headless-ready.

Step 3: Surviving the Group 3.x Entity Overhaul

Courses and classes across our stack are built on the Group module. Because Opigno was tightly coupled to early Drupal 8 APIs, our codebase had been pinned to Group 1.0-RC for six years (2018 to 2024). We were running production sites on a release candidate while Drupal core marched from 8 to 10.

When we finally moved to Group 3.x in early 2024, it wasn't a standard dependency bump. Group maintainer Kristiaan Van den Eynde had re-architected the module's internal domain model: the foundational group_content entity type and its configuration entity (group_content_type) were deprecated and replaced by group_relationship and group_relationship_type.

This wasn't a cosmetic rename of a few methods; it was a seismic shift that touched our live database:

• Physical Database Migrations: On live databases holding years of student history, primary MySQL tables (group_content and group_content_field_data) had to move to group_relationship tables without orphaning or dropping a single active membership.

• Query and Storage Sweeps: We had to rewrite every query, storage handler call, and entity hook (hook_group_content_*) across our stack that targeted the old entity type.

• Broken Views: Dozens of student gradebooks, course catalogs, and administrative rosters lost their relationship joins overnight when the underlying base tables changed, requiring systematic reconfiguration across multiple affiliate sites.

Nat and I survived this live schema migration in two intensive sweeps across dozens of commits. It was grueling work, but shedding six years of technical debt paid off: native alignment with Drupal core's Access Policy API and VariationCache, which permanently eliminated the database permission bottlenecks that used to slow down the site during concurrent quizzes. Kristiaan also helped every upgrading project, including ours, by providing an update hook so a full migration wasn't needed.

Step 4: The Clean 1.0 Rewrite and the Hybrid Strategy

By spring 2024, our custom refactored code was humming in production. It was time to give this work back to the community.

I created the drupal/lms project on Drupal.org. But as soon as I started writing the public 1.0.x branch, I decided not to carry legacy procedural code into the contrib project. I wanted a clean-room architecture featuring typed plugins, modern PHP attributes (#[ActivityAnswer]), Single Directory Components (SDC), and normalized JSON answer data storage.

This led directly into the classic enterprise trap: How do you build a breaking, clean-room 1.0 release on Drupal.org when learners on your live client platforms are taking exams on the older 0.2.x fork?

You cannot run composer update drupal/lms across a decade of live production data and hope for the best.

In June 2025, we made what I consider our most effective risk-management decision: the hybrid repository strategy.

We permanently froze our working 0.2.x code inside web/modules/custom/lms/. Production users continued logging in, taking mandatory tests, and earning licenses against this battle-tested codebase.

Simultaneously, we required our modern drupal/lms contrib package in web/modules/contrib/lms/.

This gave us the freedom to build the future out in the open. Contributors such as Claudiu Cristea standardized our dependency injection and PHP attributes; Oberon Ray modernized our frontend using Single Directory Components and built rich feedback question plugins; and Nat brought core-level traits and caching rigor.

Both engines lived side by side in the exact same codebase and database. Production never stopped, and the contrib project never took on legacy hacks. All we needed was the bridge to move the data across.

Step 5: The In-Place Data Migration

To build that bridge, our colleague Mauricio Dinarte wrote lms_migrate, a comprehensive Migrate Plus pipeline.

Instead of setting up a fragile, multi-server ETL (extract, transform, load) process between two databases, we chose to migrate in place and we had all we needed in every site database already. Because the old custom module and the new contrib module ran in parallel within the same Drupal installation, we gained four advantages:

1. Zero Media and User Overhead: We didn't have to spend weeks migrating user accounts, taxonomy vocabularies, file entities, or media libraries. They were already there.

2. Calling Runtime APIs Directly: In legacy Opigno, certain student statuses were not stored cleanly on entity tables; they were calculated through procedural helper functions. Because both modules were enabled simultaneously, Mauricio's migration plugins could call our backend PHP APIs directly during the ETL process to extract and normalize historical attempt states.

3. URL Alias Swapping and 301 Redirects: Teachers had tens of thousands of bookmarked links to specific courses. The migration automatically applied an administrative prefix to legacy Opigno course URLs, reassigned the canonical URL alias directly to the new lms_course entity, and spun up 301 redirects wherever path structures differed.

4. Pivoting to Revisions Midstream: When we introduced an enterprise course revisioning strategy in Drupal LMS 1.1, the migration pipeline was already halfway built. Because our architecture was modular, Mauricio updated the migration mapping to populate the new revision base fields before the final cutover, preserving complete version auditability.

The generic foundation of this migration pipeline is now available to the entire community at drupal.org/project/lms_migrate.

Architectural Highlights in Drupal LMS

By stripping away the distribution and rebuilding purely on modern Drupal core APIs, we built engineering patterns that I'm proud of.

1. Recursive Modal Subforms

Editing a multi-layered course structure, where courses contain lessons and lessons contain activities, often pushes developers toward heavy decoupled JavaScript apps. Opigno took that route with a React based Course Manager and multiple endpoint controllers.

We replaced that entire decoupled frontend with native Drupal Form API and AJAX modal subforms. A subform inside a modal submits to an internal endpoint that validates the data, rebuilds the parent form, and stores the intermediate state in Drupal's form cache. Because requests handle state recursively, there is no nesting limit. You can edit an activity inside a lesson inside a course, and the parent form receives the unified submission tree, with no third-party JavaScript framework. Using Drupal Core entity form API also automatically handles all security, performance and scalability issues that'd be valid in case of a custom solution.

2. Session-Based Entity Storage for Anonymous Learners

Public education and enterprise demos often require letting anonymous users try a course without registering first.

Traditionally, tracking student progress requires a database row tied to a user ID. We implemented an adaptive entity storage handler for our LessonStatus and Answer entities. If the user is anonymous, the handler stores their progress in their active PHP session. If they register or log in later, the handler can write their attempts to the database. To prevent per-user session states from breaking page caching, action buttons and navigation links use Drupal Core Lazy Builders to remain completely cacheable via BigPipe.

3. Plugin-Driven Bundle Field Installation

In Drupal LMS, ActivityType is a configuration entity that defines a bundle of the Activity content entity. Each bundle connects to an answer plugin (Multiple Choice, Drag-and-Drop, Free Text with Teacher Evaluation, etc.).

Instead of complex schema update scripts, our plugins declare the fields they need using configuration templates containing bundle placeholders:

# Config template provided by an Answer Plugin
id: activity._bundle_placeholder_.field_answer_options
field_name: field_answer_options
entity_type: activity
bundle: '_bundle_placeholder_'

When an administrator creates an Activity Type in the UI, the answer plugin automatically installs those fields onto that bundle. Activities remain 100% fieldable, meaning editors can attach custom media and descriptive fields, or style the entire presentation with Layout Builder.

Reversing the Decay Curve

Long-lived codebases tend to decay. Patches accumulate, dependencies rot, and projects become fragile places where developers dread touching old code.

By isolating the engine and unbundling the monolith, we reversed that curve:

• From 100,000+ lines of fragile procedural code down to ~40,000 lines of clean, object-oriented modern PHP.

• From a 4,000-line procedural monster down to a 1,000-line testable TrainingManager service.

• From an all-or-nothing distribution down to a composable module that dropped into Share My Lesson and is ready for the Drupal CMS Recipe era.

Rebuilding mid-flight is not easy. It demands patience, defensive Git workflows, and an unwavering commitment to protecting live user data. But by taking ownership of the legacy code, isolating the core business logic first, running hybrid tracks, and migrating in place, you don't have to choose between keeping the lights on and building software you're genuinely proud of.

Drupal LMS is production-ready, stable at 1.2.x, and 100% open source. Check out the project at drupal.org/project/lms, or join us in the #lms channel on Drupal Slack.

• Architecture
• Drupal
• Drupal LMS
• Migrations
• Open Source CMS