How we made the biggest performance gains in a decade in Drupal core

[00:00:04] Nathaniel Catchpole. I'm catch on drupal.org. I'm a senior performance engineer at Tag1. They also sponsor me half time to work on Drupal core. They took over from my previous sponsor of about 10 years about four months ago. I've been using Drupal since the 4.5 version, very long time ago, contributing since 2006. And this is how we made the biggest performance gains in a decade in Drupal core. I ran it through this morning. It ran a bit long, so I'm going to be going through at a reasonable pace. There's quite a lot to talk about, which is, I guess, that's good and bad, right?

[00:00:39] This is a QR code that goes back to these slides. So if you want to look at the slides on your computer, there are links in many of the slides. They go to drupal.org issues that are open that need reviews and committing and things. So please click on them, have a look, try and help out. There's plenty more work to do.

[00:00:57] All right. So first of all, when we talk about performance, we need to talk about what getting faster actually means. And a big problem when people measure performance: they often measure the wrong thing. There was a study done by Robert Miller. It's called "Response time in man-computer conversational transactions," and it was basically a psychology test. Any response of 100 milliseconds or less gets perceived as instant by end users. If it's 1 second, it's fast. And 10 seconds is like you switch browser tab because you don't think it's going to load, right? And even though that was some time ago, computers haven't really got faster in terms of our interactions with them. We just add more crap to use the resources that we were given, to do more things that are often quite pointless. So, like, these numbers, it's like 10 seconds. Yeah, okay. Loads of sites take more than 10 seconds. Loads of it, you know.

[00:01:54] So when we talk about performance, we really talk, with Drupal especially... Drupal has a lot of structured content, a lot of structured data. There's a lot of configuration. There's a lot of plugins, registries like theme templates, entities, lots of different pieces to load. So caching is extremely important for Drupal performance. It's not a static page generator. You can use it as that, but most of the pages that we produce are very dynamic. When you have a hot cache, we can serve pages straight from the CDN or Varnish, and those are like 10 millisecond responses or something. It's probably your network latency is more than it takes to generate the page. And the internal page cache is close to that, 20 milliseconds. There's not really much to trim from those. Where we can trim things from is the cold cache responses. And if your site has, like, hundreds, thousands, hundreds of thousands of nodes, that's a lot of pages after a deployment, when you do a full cache clear, that are not in any of your cache layers. They're all cold cache responses. Hundreds of thousands of cold cache responses.

[00:03:02] But when people benchmark Drupal, there's this blog post a couple years back by Kinsta. They did loads of PHP frameworks, and all they did is they loaded up the PHP framework in, like, DDEV and they ran ab on it. And you get one big request at the beginning, which is a cold cache unless you warm it, and you don't even get that, and then it hits the page cache. So all they did is they benchmarked page cache, but then they benchmarked Laravel or WordPress that doesn't have it, and they come out of it... So they compared completely different things that are not related to each other. If you configured those applications slightly differently, like installed one of the contrib WordPress page cache plugins, they would have got completely different results. It's just not telling you anything useful. So don't do that. There are situations where you can do this and it's useful, but hitting the same page 500 times is not actually what happens to a site.

[00:03:50] This is what it looks like, right? You clear the cache. You have, like, a big red dot where all your caches are cold and they have to be built. And then you visit, like, the first article page on your site, and that will have to do things related to articles, like load the entity system, maybe Views, load the entities themselves, some Twig templates that haven't been compiled yet. And then as you visit the next node page, some of the things from the previous one will be warm, but that node hasn't been loaded yet and it's not cached, and the user pages, taxonomy term pages. So you get this gradually increasing cache warmth, but it's not progressive, because you'll keep hitting other pages that are colder. And then editors will start editing content that will empty different caches, get rid of caches. You get this mosaic of different levels of cache all over the time. So when we're trying to improve performance, what people actually perceive is not the page cache hits when they get those responses, it's those colder requests that are not fully warm yet. And that's what really slows Drupal sites down.

[00:04:56] So for that, we can't use ab. We have to use tools that show us something approximating reality, like, not necessarily reality, but closer than that. There's Gander, which is Drupal core's performance testing framework, which I'll talk about later. There's Goose, which Jeremy from Tag1, who's unfortunately talking now, he wrote, which does load testing, but you can log in, you can browse different pages authenticated, you can submit forms. There are other load testing frameworks that do the same thing. And then real user monitoring like New Relic, and profiling. And this actually tells you what is going on in those different contexts that users actually see.

[00:05:36] So how much faster did we get core? This is what you're here for, right? So these numbers are from Drupal core's performance tests. This is a cold cache request for the Umami demo profile. In 11.1, to serve that request, if you combine the database queries, the cache gets, the cache sets, and the cache tag lookups, we were doing 1,487 separate I/O operations for those things. Five releases later, we're down to 662. So that's a reduction of about two-thirds. For, like, the first article page after visiting the front page, so that kind of intermediate cache warmth, down from 543 to 312. And a different node page, where it's like that particular article isn't cached, it's 484 down to 274. So it's like hundreds of things we're doing on each request.

[00:06:26] But those are the numbers of queries. It's not time. So what we have to do is we have to come up with a conversion between operations and time. And you can't really do it, but I did it anyway, and it's basically one of them is one millisecond. And that's close enough to get an idea of how things work. When we send a database query, we have to build the query in PHP. We have to send it to the database server, which could be on a different server over the network. The query has to run. It has to be sent back over the network. And then Drupal has to do something: put it into an object, unserialize a cache item, whatever it is. And whether it's a database or Redis, it's more or less the same thing going on. It could be half a millisecond, it could be 2 milliseconds. It depends on your network and your hosting setup, but it's in that general region. So when we took 900 things off cold cache responses, if they all take a millisecond, it's 900 milliseconds faster. When we take 200 off, it's 200 milliseconds faster. And these are, like, perceivable performance improvements for end users visiting sites. And a lot of requests to Drupal sites, especially when they've got a decent amount of content, are in this kind of range of requests.

[00:07:37] Another thing is not all cache operations are equal. Drupal has a chained fast backend, which puts high hit rate and finite cache items into APCu. So if you look at that 12.x graph, the big red line is cache gets, but, like, three-quarters of those hit APCu. They don't have to go over the network. So the things that we haven't got rid of, in part that's because those are already very fast. There's not much time to save getting things out of APCu. And we've moved some large objects that used to go to Redis that had low hit rates to small cache objects in APCu with high hit rates. Like, the routing cache is one of them. There's another couple of ones. So below those headline numbers, there were smaller things moving around that also mean more hits, less misses, and faster backends.

[00:08:33] And all of what I've been talking about is cold cache. Hot caches, it's like 20 milliseconds. It's hitting the cache system a couple of times. You're talking 20 millisecond response times. There's not that much to do there. We can make those a bit better. It's just not that interesting. If you go from 20 milliseconds to 10 milliseconds, your server might know, but no one else is going to notice. It's twice as fast, but you made a little thing twice as fast. It's really the 1 second responses that you want to get down to 300 milliseconds. And that is where people will be like, "Oh, this is snappy," as opposed to it's a bit laggy or a bit clunky or much worse than that.

[00:09:09] What does this mean for sites? So I asked around on Slack for people to update from Drupal 10 to 11. I didn't get as many takers as I was hoping for, but I do have two for you. This first one, this is a Tag1 client site that Francesco Placella works on. He's here somewhere. And that graph is the top databases report from New Relic. So in April they updated from 11.2 to 11.3. You can see it's hitting memcached, and there's a few paragraphs queries, path aliases, things like that. And it's something like, what's that, 80% improvement, like 80% less queries running in that 12-hour period.

[00:09:55] Another example. So nicxvan, who's not here, but he's the OOP hooks guy, if you've been following that in core, he updated a site from 10.6 to 11.4 a week ago, and he knew in advance that I wanted these numbers. So he very kindly gave me this data to look at. So on that site, the 96th percentile response time was 32% faster, and the median response time was 68.7% faster. And this was the week before the deployment and the week after the deployment, Blackfire reports, like graphs from Blackfire. I don't have access to this. He just found it and dug it out for me. And if you look at this, this is a couple of very similar requests that he took from before and after the deployment, from the same URLs, with what we hope is the same level of cache warmth. You can see the wall time, the I/O wait, the CPU, and the memory all went down. The network didn't go down, but that's because this site doesn't do any network requests from PHP, so there's nothing to cut. And you can also see, like, my millisecond per request in SQL: it's not accurate, but it's close enough. If it's half a millisecond, you still saved 200 milliseconds on 400 queries. So how did we do that?

[00:11:13] So the first thing we added, in 10.1, was performance testing. We wanted performance testing for a long time. Since about 2009 I'd had an issue open, but it took a long time to add it. Performance tests are just functional JavaScript tests, and there's not really that much to it. You prepare your scenario of the page that you want to visit, and then you put the request that you're going to make, or the form submission that you're going to do, inside a closure. Pass that to the collectPerformanceData method. And there's this, this is new in 11.5 and 12: assertMetrics by name. The test now will write the assertions for you. So instead of you having to count the database queries and figure it out by running the test, you run the test and it writes a YAML file, and a YAML file looks like that. So it gives you the query count, the cache operations. It also has front end metrics, like the CSS and the JavaScript, that didn't fit in the screenshot. And it writes the database queries out to a separate text file. And this is all automated. So you run the test, it hits the page, it writes it out to a file. And the next time you run the test (obviously commit the file that it created), the next time you run it, if it's changed, it will fail and it will tell you what changed.

[00:12:29] And then that also sends data to an OpenTelemetry dashboard based on Grafana, with Grafana Tempo, which tracks, like, front end timings on a graph, for Largest Contentful Paint. That's a link. It's all live, and you can run that locally with DDEV as well.

[00:12:49] So performance tests, when I originally worked with Adam, the idea was to stop regressions going into core, because we would commit stuff and it would make things slower, but you wouldn't know for six months or a year until someone ran into an issue on a production site. And eventually you go back to the git blame and be like, "Oh, it was that." And sometimes it's an innocent change that didn't do anything bad. It just called something that wasn't that well optimized a bit more often, or something like that. But I underestimated how useful it would be as a diagnostic tool. So having the queries and the cache IDs that we see on the page, you can see the same query running more than once. You can see similar queries running multiple times.

[00:13:25] I wrote a performance test for a client, like, a year ago. After a cache clear, their site would take a minute 40 to serve one page, and it ran 80,000, like, database and cache operations combined on that one, like, the front page. drush cr, load it, like, my laptop took off off the table, like, heated my house for three days. And then I looked at it, and 3,000 of the combined cache and database things were Webform fields. And Webform entities, they don't have fields. It's fieldable, but the Webform submissions, they want a base field. There's no configured fields on webforms. You can add it yourself; no one does. And the reason it did it, it's not Webform's fault. It's not this site's fault. It's kind of arguably core's fault. I mean, we fixed it in core, but core never thought that someone was going to make 800 different webforms. But this is a reasonable thing for this site to do. They were running surveys every year, 80 surveys. The site's been up for 10 years. They had 800 surveys. That's 800 webform bundles.

[00:14:33] And what core would do is it would go look in the cache: does this bundle have any configured fields? No. I'm going to check the database. Is it in the database? No. Okay, write an empty array to the cache so I don't do it next time. And it'd do it 800 times. So what's that, 2,400? So all we had to do is, for the entity type webform, do a multiple get for all the possible things from the entity type. That's one cache lookup, one database query, and then write it back. So 2,400 went down to three. Still left 76,000. I'm not going to talk about that because we'd be here all day.

[00:15:09] So in the core test, that looks really little. That's, like, maybe 10 things, because, like, how many bundles does Umami have? It has, like, a couple of taxonomy vocabularies, three node types. It doesn't have 800 webforms. But on your sites you could have, like, 30 paragraph types, or a load of content block bundles, or something like that. It might not be 800, but it might be 200. So, you know, even if you've got 10 node types, that's 30 down to three.

[00:15:40] And if you look at the headline numbers, it looks like we've done a lot of things. But this is from Jesus's Drupal metrics dashboard. He's tracked the performance test numbers against core commits on main. So you can see the progression over the, this is, like, the past year and a half of progress in core. And there's some big jumps, which we're getting to, and there's lots and lots of little jumps that are issues like that. And some of it's from proactively trying to optimize core, and some of it from finding things on sites and then getting the fixes into core to improve it.

[00:16:18] So it's not just performance tests. It's also profiling, browser dev tools, and contributing fixes back. The main thing is it doesn't matter which profiling tool you use or exactly what you do, but just do something and get familiar with it, so that when you look at it you can spot anomalies. That's the thing, because if you look at it, a lot of things are slow, but they're just calling other things that are slow. You need to be able to go in and see, okay, this thing is not what I expect. Like, I didn't expect 3,000... anyway. Yeah. And this is not a rewrite. We did not rewrite core to get, like, a third of the database queries. It's lots of incremental steps, very quickly, in the same direction. It's little things building on top of each other and combining, which gets us to placeholdering.

[00:17:08] So this is, everyone hopefully has seen this, this is the Umami demo profile with the navigation toolbar, logged in as user one because I'm lazy. I didn't make a different admin account. And core has had dynamic page cache for, like, 10 years now. And what that does, it has a placeholdering system with the render API. So on this page, with the same user roles, all this information is the same except those two things, which are user specific. So if you're logged in as admin, let's say not user one but any other admin account, you'd all see the same thing, but your name would be different. Placeholdering lets us store a dynamic page cache item with little placeholders in it, get the item out of the dynamic page cache, and then swap those out, either from separate cache entries or just build it in PHP on the fly each time.

[00:17:58] But when we added the navigation toolbar to core, that's the same on every page, and the dynamic page cache is per page. So what you don't want is, say you're going to 50 pages on your site logged in, you don't want to build the navigation toolbar 50 times, because it's the same every time. You can cache it in its own item, but if you don't placeholder it, the dynamic page cache is just storing a copy of the HTML, which is going to fill up your database. It's using database space for no reason. So we placeholdered it. And so far... whoa, went too far ahead. This is Drupal 8 stuff. We've had all of this for 10 years. But there was a problem with BigPipe. And I don't like saying that, because I like the BigPipe module, but there was a problem with it.

[00:18:39] So when you placeholder the navigation bar, it's not there. And the way the BigPipe... who knows how the BigPipe module works in Drupal core? Okay. So, very quickly: it sends the HTML with no placeholders replaced. There's some JavaScript. When the placeholders get processed in the same PHP process, they're printed at the bottom of the page. And then JavaScript looks for those replacements, matches them up with the placeholder in the markup that was sent, and swaps them in. And what that means is it looks like this, and then navigation gets rendered out somewhere down the bottom that you never see, and it does this, and it puts it in where it's supposed to be. But you get content shift. So you can do this. Some websites basically look like this for, like, two minutes now, right? It's just gray bars. Yeah, we're doing something. So that's, it's okay. It's better than jumping around, but it's not very nice.

[00:19:31] So we added this cached placeholder strategy, and there's some other things mixed in with it. If the navigation bar is render cached, which it will be after it's been built once (it only needs to be built once per role or group, set of roles), then before we hit BigPipe, we look in the render cache, we put it out, and it goes with the initial HTML. And then if that's done, BigPipe doesn't have to do anything, because there's nothing for it to replace. And the cache get for that render cache item is, like, a millisecond or two, right? So you don't need to render that via JavaScript. You just send it with the rest of the page. And that means you load it, and it's all cached, and it's just coming out. It's just HTML. There's no JavaScript anymore. The JavaScript doesn't even load if there's no placeholders to replace. It's just HTML sent to the browser. It's almost instant. But it also solves the cache duplication issue. You're not building it multiple times and you're not storing it multiple times. There's one cache entry per, like, role group, or whatever you want to call it.

[00:20:26] What this let us do is it lets us placeholder everything on the page. So now all of the blocks in Drupal core are placeholdered. If you put createPlaceholder true in your block plugin, your contrib blocks will be placeholdered as well. And that opened up new things with Fibers. So Fibers is a PHP 8.1 feature, and it's a very small API surface for a very cool thing. So who's familiar with Fibers? Okay, so I'll do it as quickly as I can. So what you do, you put a callable into a Fiber. So you create it with a callable, you start it. Any code that runs inside that callable, so Drupal stuff, can check if it's in a Fiber, and if it thinks that it hasn't got anything to do right now (it's mainly for async), it can suspend. And then the thing that called it can do something else in the meantime, and then resume it and go back to it. And when it's done, you get the result, and it can be, like, HTML or a render array or whatever, and then you can do what you want to do with it.

[00:21:34] So we added Fibers to placeholder replacement in Drupal core. But we're not actually using it for async in core yet. We're doing something really hacky that also works well, which is entity and path alias loading. So if you go back to here's all our placeholders. Those four blocks, this is not actually real. It's just, I put stuff on it to explain it. So it's not exactly like that, but it's close enough. There's four blocks. They've each got a node. All the blocks are independent. It could be Views, it could be a custom block, it could be, like, an entity reference or something. They all reference a media item, and the media items all reference a file, because it's Drupal and it's structured data and everything has to be nested down to little tiny pieces. So there's four blocks and there's three entities. Three times four, that's 12 separate entity loads. That's what we had to do in 11.3, or one of the versions, doesn't really matter.

[00:22:27] We don't have to do it anymore, because with Fibers we can do this. So it goes into the first placeholder. We get to the point where we were going to get the entity out of the persistent cache or the database, and we add it to a list, just the ID, to say we need to load this entity at some point. Then we go to the next placeholder and we add that one to the list, but we suspend the Fiber. We go back up once it's got round all the things, like, as much list as there is. You get back to the first placeholder, and it's gone past that point, and it's like, okay, I'm back where I started, but now I've got four entities to load instead of one. So I'm going to multiple load the four entities. It's like one round trip instead of four, and do it all in one go. And as long as you have the same entity types in the same order, which you generally tend to do on a lot of sites, it works for all the media and the file entities. So you go down from 12 entity loads down to three.

[00:23:25] And we've also done it for path aliases. It's almost exactly the same trick, because why not do things when they work? So path aliases are also multiple loaded. And this let us remove a whole bad caching system that I added in 2008 that was, like, per page, so that you'd have all the system paths so you could look them up. But with render caching, that almost never got a cache hit, and it didn't work when it was on cold cache requests. This works for cold cache requests. So we've got all these unrelated things on the page, and we can collect the things that we need. But because Fibers let us do a little bit of time travel, we can find out in advance what we need, and at the last moment we get them all, and then we keep going and render it.

[00:24:09] This isn't done yet. There's more to do. So Views rows are currently not rendered via Fibers, but if you've got a grid with 50 entities, those are all loaded individually. As soon as we put that into Fibers, they'll be loaded in one go. Placeholdering support in Canvas and Layout Builder: they don't do it. If they did do it, they'd get all the same benefits. The issue is open. It's been open for a little while. Hopefully we'll get to it at some point. The Revolt event loop will formalize the Fiber implementation in core. It's currently a bit handcrafted and scattered around. And then FrankenPHP.

[00:24:42] I want to talk a little bit more about FrankenPHP. FrankenPHP is a PHP runtime and web server. So it basically replaces nginx and Apache, and it has a worker mode. So what that means is you have a persistent, like, Drupal app server that has bootstrapped, like, the container, theme registry, plugin caches. They're held in memory, and then that can receive and respond to multiple different requests from the same process. So you don't need to actually bootstrap the PHP process each time. All the work that we've done to get Fiber support into core, and deal with the myriad bugs that made that possible, now means that we could directly work on FrankenPHP worker mode. Someone, about a month ago, started working on it. They've got a patch set that they said themselves is extremely hacky, but that's passing their Drupal 12 alpha 1 custom test suite that they have. There's patches in the core queue underneath this meta issue. And Merlin, I don't know where he is, is also working on it from a different angle, also in the past, like, month, I think. So it's kind of like we can get to this, and that is potentially, like, another most-improvements-in-a-decade kind of change for Drupal performance. But it would need hosting support. So when the hosts support this and Drupal core also does it, it's like hosting costs go down and response times also go down. You know what I mean?

[00:26:13] Entity field loading. All of our fields in Drupal, for configurable fields, are stored in separate tables. We used to load it table by table. In 11.3, 11.4 (it was very hard to do, it went wrong a couple of times, I think we fixed all the bugs), we now do a big join. It's chunked if you have, like, 500 fields, because some entities do have 500 fields, apparently, and they didn't like it when we made this change, and then we had to fix it. But if you have, like, 50 fields, that's now down to one. So that one entity you load that wasn't cached used to do 50 database queries. Now it's two or three, because it's, like, the base table and the single fields and the multiple fields, but it's gone down a lot.

[00:26:55] Entity queries. We found that entity queries were doing extra joins that they didn't need to, which meant, like, temporary tables, filesorts, and things like that. That join is gone. If you use JSON:API, it makes a massive difference on JSON:API-using sites especially, because they're very entity query heavy. But also custom entity queries, it does the same thing. This is from the performance test. You can see that the query changed. Obviously, it doesn't tell you whether it's faster or not. I would like to get EXPLAIN support into the performance testing framework, so you can also see if it's using a temporary table and stuff like that. It's doable. Haven't had any time to do it.

[00:27:35] What's next for that? Andre, and also daffie separately, have been working on JSON entity storage. That has been blocked on cross-database index support with JSON queries for a long time. But Andre found a way around it. I wouldn't call it a workaround. It's very nice and clean. He's laughing. Where we can use generated columns and MySQL views (not Drupal Views, MySQL views, database views) to denormalize the entity data without having to do it in PHP, so that the database manages it, which means you have real columns with real indexes that you query on, and the canonical data is in JSON. That will mean all those horrible hacks we added for the entity loading that I've just talked about, we could drop or move to contrib, because it will be in a single document. But it will also help Views and entity queries, because instead of having, like, 50 tables to join on when you're doing listings with conditions and sorts and group by, it will be, like, one or two. It won't completely fix every possible query like that, but it should improve it a lot.

[00:28:42] And then async. We're not using any async features of Fibers. If we add async database query support, you'll be able to send, like, if you use a Views query that's quite expensive, say it takes 200 milliseconds, just suspend the Fiber, let other placeholders load entities and render or whatever they need to do, and then come back at the end and see if it's returned, and then process the view. Then even with one Views query on a page, it works, because all the other stuff can go while the query is on. If you have two queries, you can run two async queries at the same time with a connection pool. That issue has a couple of child issues. One of them has been RTBC for a very long time, and I haven't been able to review it to get it committed yet. But this is, like, a couple of issues away from being reality.

[00:29:30] We also added discovery improvements. So that first cold cache request, we do a lot of YAML parsing, a lot of attribute parsing for plugins, building, like, plugin registries and Views data and things like that. All of these things, that's what takes the time on that first request, especially when it's a minute and a half. We added a file parsing cache collector. All it does is, instead of parsing YAML every single time you clear the cache, it puts it in a persistent cache backend, which doesn't get cleared on cache clears because it validates with mtime. And that means even after drush cr, you still get hot caches for those particular discovery items that can be compared against the file. That has reduced drush config import: on 11.4 it takes 733 milliseconds (I mean, it did once on Umami; it's different on different sites). On Drupal 12 it's about twice as fast. And there are many other things we can apply that to.

[00:30:29] So it sounds a bit stupid that we put a cache behind the code cache, but we're looking at, like, hundreds of milliseconds for some of these things that you do on local development, or the very first request after a deployment, where it feels like the site's gone down and you're not really sure if it's going to go up. And it all adds up, for, like, our developer experience and user experience, as people who have to rebuild caches all the time, and also for site deployments that, you know, could have 200 requests coming in in a minute, and they're all waiting.

[00:31:00] Front end performance. I don't know how much time I've got. Not a lot. Okay. We added font preloading. You know that link preload: you put that in the top of the HTML, and the browser can discover your font without having to download the CSS and parse the CSS and see if a font is necessary. If you go in Lighthouse and it complains about chained network requests, it's usually web fonts that were the problem. Three lines of YAML in your library definition for your theme will now put that link in the header of your page. You can use this now. And then the browser will start downloading the font alongside the CSS, and it should be immediately available for use. This is, like, hundreds of milliseconds often, because fonts can be big, like hundreds of kilobytes, when the browser cache is cold.

[00:31:52] We also added aggregate targets to library definitions. So themes will have a library definition with, like, 50 CSS files or something, because, you know, it's divided up. It's organized, hopefully well. If you add an aggregate target, those go in their own unique CSS aggregate. What that means is that's more or less independent. The URL is based purely on that library. It always produces the same file with the same files, and whatever other CSS is added on the page, like a bit of Views, like some extra widget that you've got that only adds on one page, doesn't interfere. And the effect of this is it improves the cache hit rates for your CSS and JavaScript. This before and after for libraries also does the same thing. Instead of custom weights for individual files, you set the whole library before and after. That again makes things more predictable. There's less variation.

[00:32:47] This is from Jesus's metrics dashboard again. We've got a performance test in core that visits four pages, and it adds up the total CSS and JavaScript downloaded across those four pages. And that's gone down from 250 kilobytes, like, in January last year, to 64 kilobytes today, because of those three or four changes. And again, you can do this in your own theme. It's three lines of YAML for the preloading. It's two lines of YAML for an aggregate target. Modules can do it. If you have groups of libraries, you can group them together with an arbitrary string.

[00:33:23] This again is also not done. We need to finish off getting rid of per-file weights, and the way that the asset URLs are constructed. The top issue, in this different performance test, it's another one-third reduction in file size downloaded, and it reduces the number of files downloaded now as well, like the number of HTTP requests. That last one is early hints. There's 103 Early Hints in the HTTP spec now. So once we know what your theme is and what the fonts are, because we have the library definitions, we can send the font URL and some of the CSS to the browser before any of the page is rendered. It's an extra header before the page gets sent and the actual real headers, so that your browser can start downloading the fonts and the CSS before any HTML appears on the page. But that needs upstream PHP support. It does not support it yet, or it needs FrankenPHP. But once we can run on FrankenPHP, we get that as an add-on, or PHP can add it.

[00:34:26] How do we compare to other frameworks? There's this HTTP Archive. It uses the CrUX data set from Google, which I think comes from Google Analytics. There's, like, millions of sites in this data set. This is Drupal, WordPress, Next.js, and Laravel at the moment. You can see we're ahead of several open source frameworks. This doesn't necessarily mean that Drupal is better. Some sites are very simple. Some sites have lots of content. Different frameworks have different kinds of hosting combinations. Some are on shared hosting more than others. We're doing okay. If you compare us against the SaaS offerings like Shopify and Wix and these, it's not quite as good a picture. But there's a good reason for that, I think, which is that if you're on, like, WordPress.com, you're always on the most up-to-date version of WordPress, and you have a curated hosting experience with a CDN and all of these things, because WordPress.com can do that. You're not on some random shared host somewhere.

[00:35:24] And if you look at this, this is Drupal core usage data. Those graphs where we're going up, it's not because of Drupal core improvements in the past year, because most sites are still on Drupal 11.3 or less, or even on Drupal 10. There were 100,000 sites on 11.4. There are, like, at least 200,000 sites on Drupal 10. So in the next few months, they're going to update, and hopefully we'll get a bit further ahead than we currently are, but we're not going to find out for a little while.

[00:35:56] This is interesting. So these are two well-known Drupal hosting platforms and the aggregate of all Drupal sites, and this is the time to first byte number. So you can see they were both lagging behind the average of Drupal sites. One of them overtook recently. They're both getting a bit better. But if you're a hosting provider, then I think you should have a look and try your own hosting provider and see what you can do about it. Even if you're ahead, you could be further ahead than you currently are, because 45% of good responses is still not good, is it? If we look at the distribution of this, green is good, yellow is bad, red is terrible. There's quite a lot in this, like, bad but not that bad. So what I'm hoping is, when those sites update and they get, like, 100 milliseconds, 200 milliseconds faster on those things, those yellow bands will move into the green, and we'll just have a few outliers in the long tail that have bad performance. But, like I said, we're not going to find out until that happens.

[00:36:56] So to fix your own sites, please update. Clients don't like running updates. They don't think they get anything out of it. Now they get a much faster website. It could be 70% faster. Don't say that. Don't say I said that, because it could be lies. It could be, like, nothing. But it could be right. You don't find out until you do it. Ideally, measure before and after and take some nice graphs out of whatever it is you want, and send them to me. Even if it's bad, I want to know. You can write performance tests for your own site. There's Drupal Test Traits support, if you use that. So you can just run against your real, not your production database, but a copy of your production database that's been anonymized and all of that. There's a DDEV add-on that gets you all the charts and stuff like that. Do some profiling. Again, you don't have to use xhprof. Don't care what you use, but use something. Get familiar with it. That's XHGui. It gives you nice little summaries of what's going on.

[00:37:50] One little tip: in menu block configuration, you can stop it from using the active trail. The active trail: basically every link in the menu is a different active trail, and depending on the page it will put a little active class on it. A lot of menus, especially in the footer, they don't care. They don't render any differently. But we always rendered separate cache versions based on the active trail, just in case you were theming the CSS, and that's on by default because we can't break anyone's site. If you go into your block configuration (obviously check if you have CSS that targets that particular class that gets added, and it's different to the active class; I cannot explain the difference, but it is different; there are two different active classes in menus and they both get added), you can uncheck that checkbox. And if you've got, like, 20 links in your menu, instead of 20 versions of that menu, you have one. And if you've got, like, a thousand links in your menu (and that 80,000 website did have 2,000 links in their menu, many of them disabled, for breadcrumbs and things), that goes from 2,000 versions of the menu down to one version. And you can help out with core issues tagged performance. That's a link. Okay, any questions?

[00:39:14] Don't you... Microphone? Microphone. Have you got a microphone?

[00:39:28] (From the audience: When you do the Fiber trick and you want to load all the entities at once, does everything that needs to load an entity have to be its own Fiber for that to work?)

[00:39:36] Yes. (From the audience: So if you have 100 entity things on the page, you have to split. When you had a foreach loop before, now you have to have 100 Fibers?) Yes. (From the audience: Uh-huh.) Unfortunately, yes. So when we do Views rows, if you have, like, 50 items in a Views grid, each one of those, they don't have to be a placeholder. We can implement it with pure Fibers. So if it's in a block, it can be placeholdered already. If it's something custom, entity rendering, you can do it. But also, if it's custom, you can just multiple load and do it that way. (From the audience: In those cases you talk about, core does that already now.) Yeah. Yeah. Views is the one we haven't done, and that would be good to do. Who's next?

[00:40:22] (From the audience: Hi. I presume these have an effect on energy usage. Have you measured that?)

[00:40:27] Only when my laptop doesn't spin as much when I'm trying to fix things locally. There were things that I couldn't fit into the talk: we made, like, the installer two or three times as fast. So Drupal core tests used to take nearly 300 CPUs to finish in 5 minutes, and they now take 180, and we have other issues in the queue to hopefully bring that down further. If you went to Francesco Placella's talk earlier today, he talked about a lot of that. So that's directly Drupal Association money that they wouldn't be spending, or AWS credits they wouldn't be using, whatever it is, but resources. Hosting energy, I personally don't have access to that kind of high level of data. But yeah, I do think it would improve things. The reduction in database queries should take load off database servers. And when we get into FrankenPHP and things like that, that's potentially another step change for energy usage, because you need a lot less. Every request with Apache or nginx needs quite a lot of potential memory, and when you serve lots of things from the one process, it can just handle it. But no, I mean, it'd be nice to get numbers like that. We really need more sites to update as well. Even if you had a hosting platform, you won't see this yet, because sites are on 10. (From the audience: Awesome. Thank you.)

[00:42:00] Kristiaan. (From the audience: I never really saw the metrics, but we implemented this thing where you can list cache tags that are common.) Yes. (From the audience: That we all load them in one go.) Yes. (From the audience: And we saw with the Umami test that the lookups went down drastically.) Yeah. (From the audience: But that's just one query. But did that, on bigger sites, when you got these results from various people, did that also show up as something that improved performance enough that you would advise people to start using that, or was that a very small performance gain?)

[00:42:36] So the common... like, this is, I would have talked about this, but it was a long talk. We multiple load cache tags based on the cache items. So the more cache items that you load in one go from the render cache, when they all have cache tags, all the cache tags get loaded together. So you get a double benefit from that. And there's also a list in the container of common cache tags that are on basically every request, that it just loads. So things like, what is it, http_response and things like that. They're on every HTML page. So if you have a contrib module that has very, very commonly used cache tags, yes, you should use that. But be careful, because you don't want to look things up that aren't needed. But I think just the core issue saves us, like, 15 cache tag lookups or something, in one patch. So it is, like, 15 cache tags on every single request. That's a lot of savings, in my opinion. It's all cumulative as well. Anyone else?

[00:43:45] (From the audience: Thank you. So I wanted to ask, I have a feeling like this is a drastic change in Drupal 11, like maybe Drupal 10 was not so much changing. But is my feeling correct, or has it always been faster and faster?)

[00:44:02] So, Drupal 8, we added dynamic page cache and BigPipe and things. They made a big difference then, and we made a lot of little incremental changes. But yeah, I would put it down to performance testing. But also, once we had performance testing, other people, some of them are in this room, started helping a lot more with performance issues. So there's been momentum from, like, a pool of contributors reviewing each other's work. And we also had a lot of things in place, like placeholdering and BigPipe and stuff like that, and we were able to tweak it to get a lot more out of stuff we had already built. It was a lot of work, but it's not, like, huge thousands of lines of code changes. They're, like, 20, 30 lines here or there that then get rid of 10 or 20 database queries, and another one gets rid of another 20, and stuff like that. So it really has accelerated the past three releases. I mean, 11.1 to 11.2 was almost no change. 2 to 3 was, like, like that. 3 to 4 is like that again. And 4 to 5, there's not as much to get rid of anymore, but it's another 20% reduction in database queries for several different scenarios. And there were things to come through. So yeah, the past three releases, a lot of things came together in a short space of time.

[00:45:24] (From the audience: Okay, thank you. We are on Drupal 10.6, so I'm excited now to go back.) Yeah. I mean, depending on your custom code and your contrib modules and how much content you have, you will see different effects from this. You can't give a number, but it is promising. And in some cases, the core tests are actually underplaying how big the changes are, because sites are more complex than Umami. So it can be, you know, 10 queries in core, and, like I said, like 300, 500 on a real site. But yeah, find out.

[00:46:00] (From the audience: If you want to do this performance test in a website project or in a contrib module, there's going to be a lot more noise. For a website project, anytime you update composer.lock, you're going to have maybe some changes that you have to commit. In a package, like in a module, it's going to be even worse, because if you don't have a composer.lock, then anytime you get an update just coming from the cloud, you get an update and then you get a change and it fails, without even changing any code. So if you want to do it, you have to make a composer.lock, I guess.)

[00:46:29] Yeah, make a composer.lock. I mean, the good thing is the tests write themselves now. So with the updates, you run the test, you commit the results. You don't have to sit there changing numbers. (From the audience: Yeah, but you would have to do it even if there's just some new Symfony or just some minor core update that is just applied through the CI, and now you have to, without even having...)

[00:46:48] I think the way that we can improve that is, instead of just functional JavaScript tests, we could make the test applicable for kernel tests, so that you can performance test smaller units of code, like you call your whatever form thing and test that. We haven't done it yet, but we could do it. And that would be much more targeted for contrib code that way. Not necessarily good for sites, unless they're very disciplined with their testing. But for contrib, it would make things a lot better, because you could just test what your own module does and the code it directly calls, instead of a whole page.

[00:47:26] I think we'll call it done. Thanks, everyone.