Say Good-bye to Your Staging Site: True Enterprise Content Management in Drupal with the Workspaces Module

[00:00:01] Hello everybody, and welcome to "Say Good-bye to Your Staging Site." This is about true enterprise content management in Drupal with the Workspaces module. My name is Hank VanZile, I'm the Senior Director of Customer Experience at Tag1. I've been working with Drupal since 2007, and for most of the last dozen years I've held leadership positions at some of the top agencies and platforms in the Drupal community.

[00:00:27] I'm joined today by two of my colleagues. Fabian Franz is a VP of Software Engineering at Tag1. He has been an active contributor to Drupal since 2009. In addition to being a Drupal 7 maintainer and core committer, Fabian has been developing content preview systems, and this might be the answer to his riddle: the Drupal 7 predecessor to Workspaces since 2014. He's been guiding the development of Workspaces for the last three years in pursuit of feature parity with CPS.

[00:01:00] Andrei Mateescu is a longtime, prolific contributor to Drupal. Not only is he the maintainer of the Workspaces module, which just went into core in 10.3.1, he's also the core maintainer for the Entity Field API, Field UI, and transliteration. In his copious free time he also maintains 50 popular contributed modules, including Entity, Trash, and several Commerce modules.

[00:01:26] If you're not familiar with Tag1, we are a global technology consulting group focused on using open source software to solve the toughest problems of well-known organizations. We work with a lot of different technologies, but we are best known as the number two all-time contributor to Drupal. Since 2001 our team has created many of the innovations that have fueled Drupal's success, such as the taxonomy system and the Migrate API. If you use Drupal, and I'm pretty sure you do, you rely on Tag1 every time you update a Drupal site: we manage the infrastructure and tooling used to build, test, and release Drupal on behalf of the Drupal Association and the community.

[00:02:11] And if you or your customers need to keep running Drupal 7 securely after its end of life, you can keep relying on Tag1 with our D7ES service. We're the only company that's both a Drupal Association certified enterprise-grade migration partner and a certified Drupal 7 extended security support provider. We're also the only ones who have actually provided Drupal end-of-life support before, having supported Drupal 6 for six years after its EOL.

[00:02:47] But we're here to talk about true enterprise content management. So what do we mean by that? Since Tag1 was founded, we've worked with a lot of enterprise organizations, and over the last few decades we've helped many of them grow their websites from something pretty basic into a critical part of their business.

[00:03:07] As those websites have become more important, the most successful enterprise organizations have expanded the role of their CMS from a tool that manages web pages into a key element of their content governance. If that isn't a familiar concept, it's just a term for all the things, from procedures and policies to technology, that a complex organization needs to make sure its content meets its guidelines and requirements. Anyone who works with a large enterprise knows that's no small feat. There can be a lot of concerns, from brand management to legal compliance, and that's a lot of cooks in the kitchen. Strong content governance is what stops all of that from turning into chaos.

[00:04:02] So for a CMS to really meet the requirements of enterprise content management, it needs to provide both the tools for creating content and for managing the end-to-end governance of that content. Drupal has been working toward this in core for a long time, from early features like the user permission system and roles, to workflows and content moderation in Drupal 8 for managing the publishing life cycle. With the adoption of Workspaces into core, we've closed another major governance gap that Drupal has lagged on for a while: content staging and review inside the CMS.

[00:04:56] Let me explain why that's more important than ever. In our modern age of composable content and visual page builders, a lot of enterprise websites have gone from looking like this to something more like this, and these are the sites we've worked so hard to make Drupal great at building. Content comes in from a bunch of different systems, it's highly designed, and it's dynamically laid out in the CMS, with edits propagating all over the site.

[00:05:27] But here's the catch. The large-scale updates that Drupal does so well, like adding new products or migrating in a new department, are really visible changes. They impact navigation, views, and blocks throughout the entire site. Because of that, the governance of those changes, reviewing things like brand consistency or accessibility, really needs to happen in the context of the overall site, with all the pieces of the puzzle present. Until now, Drupal hasn't had a great solution for that kind of content staging. It's required workarounds and compromises, like the content staging site.

[00:06:15] The content staging site is basically a duplicate of your production site, where content managers can stage all their changes without disrupting the live site. That sounds great in concept, and as we saw, a lot of you are using them. But it's time to say goodbye, and here's why.

[00:06:38] That staging site requires a duplicate Drupal code base, a database, and a file system, and all of it has to be maintained, often on a whole other server. Keeping it in sync with your live site probably means a complex CI/CD pipeline or a build workflow that has to be learned, built, and used. I get it, we're all working in the cloud with our DevOps teams, and our managed platforms handle a lot of it. But getting it set up and keeping it up to date is time-consuming and expensive. And for what? Staging infrastructure often doesn't have the same power as production, which introduces risks around performance testing and the old chestnut of "it works fine on mine, not yours."

[00:07:39] But okay, we're all enterprise Drupalists, we're comfortable with DevOps, code, and config management. What about the impact on the people who manage the content? External staging sites mean content managers work in two or more different places. What happens when you need to stage more than one variety of content at a time? More staging equals more problems. You know what content managers don't love? Remembering which subdomain, probably auto-generated from some git branch name, they're supposed to be updating content on.

[00:08:22] And even if that doesn't convince you, the hardest part, the part Fabian's been waiting for me to get to, is that you need some way to get that content from one site to another. Content deployment across sites is rife with opportunities for failure. We're not talking about version-controlled code, we're talking about rows of data from multiple tables moving from one database into a second, completely unrelated one. That's always risky.

[00:08:54] There have been numerous attempts in the Drupal community to solve this over the years. I'm certain I'm not the only one who still has nightmares about the Features module. The drupal.org documentation has a page comparing about 50 content deployment modules, and there are external systems that have tried to tackle this too. I'm not here to say any of those are bad, some are quite good. But let's be clear: content staging servers are a costly, complex, inconvenient, and ultimately risky way to work around Drupal's historical limitations.

[00:09:41] With Workspaces in core, we don't need them anymore. Our staging site is happily retired. On a high level, all you need to be productive now is Workspaces. Right now, today, the easiest thing is to enable the module, and you're done. It's in core, it's stable, it works. But because content is always a bit further around, you also want Workspaces Extra, and we'll go into that in more detail later, because WSE and the workspaces menu give you so much more power.

[00:10:30] So what is Workspaces? In a sense it's your staging site, but inside your Drupal. You can see all the changes and preview all your content as if it were already live, including listings working correctly. You can have unlimited staging sites just by naming a workspace, like "this is my campaign." You get isolation, granular permissions, workflows, notifications, and you build it all with the tools you know and love.

[00:11:06] Who here knows Git? Almost everyone, that's great. The simplest way to think about Workspaces is in git terms. You create and modify content in a workspace, that's like creating a git branch. You approve changes, that's like a pull request. And you publish live, that's like merging a pull request. That's the mental model. And now, over to Andrei.

[00:11:45] Okay, let's talk about who Workspaces is for. It's usually for content teams and organizations, not only large ones, for complex layouts and building out campaigns. But it's mainly for every site that needs a way to publish a set of changes together, not just a single change like one page, but changes to menus, media items, and every kind of content that can be managed in Drupal. Who is it not for? Probably small blog sites, but even then, you never know.

[00:12:39] The main idea of Workspaces is to be safe by default, which means the live site is never impacted by a change you make in a workspace. That means it needs to block all form submissions by default, except when you edit a content entity. Non-content entities are blocked as well when you edit them in a workspace.

[00:13:23] Let's look at what your custom or contrib modules need to do to integrate with Workspaces. If you have a custom entity type that's shown in the front-facing part of your site, it has to be publishable and revisionable. Fabian will explain later how that works under the hood and why it's needed. Otherwise, if you have an entity type that's more like an API and isn't shown anywhere on the live site, you can use the ignored workspace handler, which lets it be saved in a workspace but not tracked by it.

[00:14:22] This is an example of how easy it is to change an entity type. This one is Crop, from the contrib Crop API module. It's an API entity type: it needs to store data, but it's not shown on the site, so it's fine to change in a workspace. We just edit the entity type definition and use the ignored workspace handler.

[00:14:56] How about the rest of the forms? You have a couple of options. If you have a custom form where saving or submitting won't impact the live site at all, you can use the workspace-safe form interface. If your form sometimes impacts the live site and sometimes not, you can use the other interface, and there's a helper trait for it, the dynamic safe form interface. So that's basically what you need to do to prepare your modules: decide how to handle your entity types and your forms. Now Fabian will give a deeper dive.

[00:15:48] Let's start under the hood. To solve my little riddle: we've been developing CPS for over 10 years now. "We" in this case is mostly Nathaniel Catchpole and me. We pioneered the approach of how to have a content preview system within Drupal 7, so if you're still running Drupal 7, try out CPS, it works there too. Workspaces still doesn't have full feature parity with CPS for Drupal 7, but we're almost there. Workspaces has taken all the lessons we learned with CPS over the years to arrive at the same clean architecture, because Workspaces, in reality, is really simple. It wasn't easy to build, but once you've created it, it's simple.

[00:16:49] It uses Drupal's revision tables, one additional workspace entity, and a mapping table. As soon as your entity is revisionable and publishable, it works with Workspaces, nothing more to do. Performance was one of the most important things in the design of CPS and Workspaces, because if you're not in a workspace the performance is almost exactly the same. There's almost zero performance overhead when you enable Workspaces. There are no different entities, no query alters, just a very quick check: are we in a workspace? No? Then it's the normal Drupal site.

[00:17:37] A quick refresher on how revisions work. We have a duplication of the main tables, like the node table, plus certain additional fields needed for revision handling, like revision ID, log, and user. When we write a revision, the data goes to the main table but is also written to the revision tables, so we always have a history of all changes.

[00:18:07] Now, we also introduced pending revisions. We write a new revision but we don't update the main table. That is the isolation Workspaces uses: we only write to the revision tables. The data is there and we can access it, but it's hidden from anyone visiting the site publicly or without access to the workspace. Once it's time to publish, we essentially resave that node revision, and then it looks exactly like before.

[00:18:45] So what does Workspaces add? People always ask, "Now I have 10,000 more database tables?" No. We just have two additional tables: the workspace revision table for the entity, and one mapping table, the workspace association table. The mapping table essentially says, in this workspace, we use revision 42 for node one instead of revision one. That's all it is.

[00:19:13] Entity loading in a workspace is simple. We hook into the entity loading process, and a new hook was added for this. We replace the loaded entity with the one in the workspace association table. Entity writing is also simple: we just write a pending revision. It was important that core got pending revisions right, and it did. Again, no data is written to the main data tables, so we can really use it as a secure staging site. No content is going to leak out.

[00:20:02] The biggest problem is new entities, and that's a bit of a drawback and gets a little tricky, because when we create a new entity in a workspace, it would appear on the live site, but it shouldn't. That's why we introduced a publishing interface. With this interface, the entity is written two times, so it exists and does not exist at the same time. It's unpublished while in live, and it's published while in the workspace.

[00:20:43] To put it another way: unpublished while in live, the new entity, say node 20, exists, but no one can see it because it's not published. As long as your views, access control, and everything are set up not to show unpublished things to the public, you're good, and Views does that by default. Within the workspace, though, we can see the entity: it shows up in listings and everywhere.

[00:21:18] The last thing, and the most complicated one, was the COALESCE approach, and huge thanks to the contributor who really innovated this idea. Essentially, whenever we join and want to find the right revision, we join on a COALESCE. We check first whether there's something in the workspace for this particular entity, and if not, we fall back to the main table. So Workspaces can also be seen as an overlay over the other parts, and that way all listings work. Now it's demo time, what you all really came for.

[00:22:06] I have a basic Umami demo site installed with the Workspaces module, Workspaces Extra, and a few submodules: the Workspaces menu, Layout Builder integration, and Trash, which will be fun to show. Let's start by adding a workspace. I'm using the simplified workspace switcher on this demo. There's a new version in the works for the navigation module in Drupal 10.3 that should be ready in a few weeks.

[00:23:00] So I'll create a workspace and start editing as many things as I can in it. Let's start with a recipe, which is a node. Let's edit the title. What else? Let's edit the media field, use a different image. This ingredients field: who doesn't like mozzarella? So we've already changed the title, the media, the difficulty, a text field, and a list field.

[00:24:10] But there's a lot more we can do. For example, we can change the URL alias. Let's quickly check what happened to the live site. As you can see, none of the changes I made made it out to live. Let's try Layout Builder, why not. Let's add a block, since this page isn't using custom blocks so far. Save the layout. Again, we're in the workspace, and when we switch to live, it's not there.

[00:25:25] So that's for a node. Let's look at some listings. If we go to the recipes view, we can see all our changes here. Listings like Views are handled, and entity queries are handled as well: they replace the default revisions of an entity with the workspace-specific one. Let's do another quick edit, and I'll show you in a moment where I'm going with this.

[00:26:09] One question I get a lot: currently Workspaces has edit protection. Once you start editing an entity in a workspace, it locks that entity for other people, and it can only be edited in that workspace. That's a current limitation. There's a next phase we're working on, conflict resolution, that will fix this. But for the moment, say you're editing 100 entities in a large workspace and you want to make one quick edit to one of them before you're ready to publish the whole thing.

[00:27:05] So we switch to a new workspace, this is provided by Workspaces Extra at the moment but may move to core, and we can move the one entity we want to publish faster to another workspace that's less crowded, so it can be published without all the other content going through approval workflows.

[00:27:51] What else? Let's set the menu item. This is why I mentioned the workspaces menu earlier. Without it, you can only change the labels of menu items, but that module lets you change the hierarchy and add new items. Let's add a menu item for the About page. Save. So we have some menu edits, and again, if we go to live we can't see them.

[00:28:52] Now let's say we want to remove an entity because it's not suitable for this edition. You can delete it in a workspace, and it's not shown in the listings or anywhere anymore. Moving to the live site, it's still there. This is a feature provided by the Trash module, by the way, and we can see it highlighted on the manage workspace page.

[00:29:33] This is where we show every change you made to your content in a workspace. You remember the painful upgrade path from Drupal 8.6, converting stuff to be revisionable? This is why we did it. So we have deleted content, changed content, URL alias changes. One interesting one: you might ask why this didn't change. In Umami this is a block, but that's not a problem, because we can handle blocks as well.

[00:30:31] It's also using the path alias for the URL field, but if we switch that to a regular link, this should work fine as well. So this is a content block change. Now you know why it's so important to be able to have several changes at once, because now we need to change a block, an article, and so on, and that's what content staging is all about: a campaign where everything changes on the site at once.

[00:31:12] So we've done a lot of changes. We get a quick overview of what we're about to publish, and, publish. That's it. All your content edits, the work of hundreds of people on this site, are now live. And something that will be in core as well: you can revert. If we go to the workspaces list and filter by closed workspaces, we can revert everything we did, which, being a live demo... anyway, you get the idea. This is in the Workspaces Extra module, and there's a reason that module is experimental: it's where we bake all the new features we want to bring into core.

[00:32:51] So, as Andrei said, Workspaces Extra is our latest test in contrib, stable enough for coursing. We use it in production, but only with very carefully tested snapshots, so you're free to use it if you prepare. Some features are really stable, like the switcher. Ideally what happens is we get this down to an empty module again, again, and again. Last time we had like 3,000 deletions when Workspaces became stable, and all of that was core, so we could remove all the patches. That was great.

[00:33:31] What Workspaces Extra provides right now: a nice workspace switcher, though one is coming to core from the navigation initiative; reverting the publication of a workspace; moving changes from one workspace to another; full menu handling like you've just seen; and scheduling the publication of a workspace, which we get asked about a lot, since people want to publish their content at midnight. There's a group access handler, so groups can have access to one workspace but not another, and preview for outside stakeholders to securely view a workspace: someone might not even have a Drupal account, but you can give them a one-time token link to see the workspace for a set amount of time.

[00:34:15] Trash, as we said, soft-deletes an entity. That delete field also works without Workspaces, so even if you don't need workspaces because it's your personal blog, you might like to trash content and restore it. It was tricky, but the third time we got it right. Trash is very simple and secure: we change some queries, there's an entity, and there's some performance cost.

[00:34:47] Entity workflow: we couldn't show this here, but it's really powerful, and you'll see a demo on YouTube from Tag1 at some point. You can move either all entities or the whole workspace through a workflow, and we're working on a recipe that lets you drop it in and do very complex workflows. Our team just built something where people can approve individual changes, like legal reviewers or subject-matter experts. And translation workflow: you've never seen such wonderful translation workflows as with Workspaces. Everything goes through the workflow; the English version gets approved, then it's sent to translation, comes back, and you can have a language program manager review, a final review, stakeholders, and so on. This is where we come into the enterprise part of it.

[00:35:51] A future outlook. There are some very experimental modules in WSE as well. WSE Config lets you give your content editors access to Views: they can change views, and nothing goes live until they click the button. A component library would let you manage your whole theme in a workspace, combined with WSE Config. And if you really still need a deployment after all that, a deploy process, you can use a transaction-based remote workspace publishing process. But that's only needed now for a 1% case, if you need such high security that you require physical isolation. Even then, Workspaces has you covered.

[00:36:41] Temporary workspaces are coming. We really want to turn content moderation into something powered by workspaces under the hood, so that's on my wish list. And custom schema is something I've been experimenting with. You can even use the workspace structure without Drupal, because it's so simple: you can use a custom database, use all the revisions, and put a Drupal UI on top to view the changes of that custom database. That's possible because the workspace architecture is overall so simple.

[00:37:17] Workspaces is stable, so you can finally start using it now. For the last three years, the most common question I got was, "I'd love to use it, but it's not stable." So we had this chicken-and-egg problem. We're done with that now. It's stable, you can use it. Adoption is now the most important thing: test it, find bugs, help bring the remaining features of WSE to core, and contribute in the queue. Questions?

[00:37:56] (Question from the audience.) I was wondering if the live workspace is impacted by cache invalidation when operations on entities are applied in a workspace. Some operations are impacted, some don't. For example, saving a node in a workspace will currently clear the cache in live, but there's an issue open for that, and it's something we're looking into.

[00:38:29] (Question from the audience.) You said earlier we don't yet have feature parity with the Drupal 7 equivalent, but you didn't say what was missing. It's really the archiving functionality. There's sometimes a legal requirement to see the site exactly as it looked in, say, December 2014. CPS has that feature, and Workspaces will get it, especially in WSE. That means we need to take a snapshot not only of the revisions in the workspace, but of all revisions in the system at that point, and then we can really show you the site as it looked. This works for sites with a stable data model; if you change a lot of fields, it won't go so well.

[00:39:23] (Question from the audience.) Does all this work with search indexing using, say, Solr? It should. In CPS we solved it by doing the update not on the entity save but by implementing the hook when the workspace was published. What you'd need to do is check whether a node is a default revision when it's saved, and that way ensure Solr is only updated when the entity is published, not while it's saved in the workspace.

[00:40:09] (Question from the audience.) What about the search experience within a workspace? In general that depends. Search API with a database back end, probably not right now, but great question, we're looking into it. If you have any ideas of how it could work, we're always listening.

[00:40:43] (Question from the audience.) You mentioned you're working on the navigation UI. Is that the same thing we're doing, or another thing? Same thing.

[00:40:58] (Question from the audience.) You had one comment on a slide you didn't elaborate on, and I'm curious what you meant. That was that even on your blog you still want to use content moderation, because you want a draft and don't want your edit to go live immediately. Say you copied something into a new node and saved it, and now it's live already and people are asking, "What is this?" You still want the draft-and-published functionality that content moderation provides. But once we have temporary workspaces, we can remove all the hacks content moderation currently uses in core and instead create a transparent workspace under the hood. That way, even when you think you're using content moderation, you're using workspaces.

[00:42:00] (Question from the audience.) You said to think of Workspaces like git, and you mentioned you lock the field when it's being edited to avoid others editing it at the same time. How do you manage collisions if they happen? In CPS we decided on "all power to the editors": you get a message that this is edited in another workspace, and whoever publishes last wins. Core decided to be a bit stricter, but you could use the message-based model within WSE or your own custom code. Conflict resolution is always a hard problem, and it's our next step. But usually the people editing these things know what they're doing, because they're in the campaign, and in practice, for the content teams I've worked with, it was never a huge problem.

[00:43:33] (Question from the audience.) How long does the lock last? There's no lock as such: it lasts until the workspace is published or the item is removed from the workspace. Say you edit the front page of Umami, and now I want to make a small change to it as well. I can't, but as soon as you've published your changes, I can. If I need to do something earlier, I ask you to revert your changes or pass them to me, and then I can make the edit. So an item can't be edited in more than one workspace, but that's for individual items, not the whole content.

[00:44:13] (Question from the audience.) But if my workspace takes one month to go live, no one will be able to edit that node? That's why I showed the example of moving one piece of content to another workspace. When you want to make a quick change to a page, publish that, and then start making the initial changes in the initial workspace again. With normal staging sites you have the same problem; it's just that everyone usually works in the same workspace, and nothing prevents you from having a team that works together on a workspace with Workspaces either.

[00:44:51] Okay, that would be it. Please fill out the session survey and let us know what you thought, and please start using Workspaces. Thank you.