Caching and Performance Deep Dive

[00:00:04] Welcome to this caching and performance deep dive. I'm very happy to present it to you today, and we'll start with a quick overview of what to expect. First, quickly about me: I'm Fabian Franz. I've been using Drupal for over ten years, and I'm currently the Vice President of Software Engineering at Tag1 Consulting. I'm a co-author of Big Pipe and the Drupal 8 and 9 caching system, and I'm also a Drupal 7 maintainer in several subsystems like the theme system. My motivation today is to teach you all I know about caching.

[00:01:05] A quick disclaimer: this is an absolute beginner session, but we'll look at some concepts from a different angle. It's roughly three parts of about ten minutes each, with five minutes for questions in between. Please mark your questions with a "Q:" so June can collect them for me, and I'll answer them between parts so we don't leave everything to the end. Part one is general caching, invalidation strategies, cache items, cache max age, and tags. Part two is cache variation, cache hit ratio, placeholders, and uncacheable things. Part three is caching layers and common pitfalls.

[00:02:12] So what is caching about? In computing, a cache is a hardware or software component that stores data, but that's too much theory. I love pizza. Imagine we run a restaurant and prepare meals, which are our pages. Normally a pizza takes ten minutes to prepare, but takeaway shops have figured something out: the pizza is already made and wrapped, ready to hand out. That is caching. We have something pre-made and we can hand it out directly. The performance of pizza delivery improved a lot once takeaway shops figured this out.

[00:03:41] Every item in your cache gets a name, a cache item name or cache address. In our example that's pizza marinara or pizza margarita. In Drupal the cache ID, also called a cache key, works the same way. So let's make pizza. We check the pizza cache and see if we can get a pizza margarita from it. If we have one, we return it. If not, we ask the pizza oven service to make a fresh margarita, and afterwards we set it in the cache. That's a basic caching example: you get something, if you have it you return it, and if not you recreate it.

[00:05:03] Someone in the back spotted a bug on that slide. Either the get cache ID or the set cache ID is wrong. Good catch. The important point is that if you use one cache ID variable for both the get and the set, it's much simpler to avoid that mistake. It might sound trivial, but I've seen it in the wild and it happened to me, and it leads to hard-to-debug problems, so really use this pattern.

[00:06:13] How long is a product valid? In the supermarket we have a best-before date. After a while you really don't want to eat the pizza or drink the orange juice anymore. The solution is the expiration date. In Drupal it's simple: we take the cache ID, we add a time to live, and instead of the plain set command we add the time to live to it. That's how the Drupal page cache worked in Drupal 6, and it's still a perfect pattern. We've run a huge production site just on this, caching everything in Akamai for ten minutes unconditionally. If your editors can accept a ten-minute delay, it's a great pattern for high-traffic sites.

[00:07:48] At the weekend we can also delete the pizza, take it home, and eat it ourselves. That's how you delete things from the cache: by cache ID, you delete. Our shop is growing and customers are happy, so we offer a frozen pizza margarita and can store it for thirty days. How does the shop work now? The customer orders the frozen margarita, the waiter gets it from the counter, checks the expiration date, and if it's expired he gets a fresh one from central storage. Checking for expiration is an active process the waiter does for us.

[00:09:03] Now we want to offer marinara as well, a vegan pizza, good for all my friends in the vegan community. It's a completely new pizza, so we make it the same way and change the cache ID. Then we improve the recipe with a new dough, version two, so we need to invalidate all the cached old pizzas without waiting thirty days. One option is to add the version to the name, but that doesn't scale and keeps all the old versions around. It becomes a mess. The solution is to tag it: the margarita, the marinara, an expiration date, and a tag, "dough version two." In Drupal you add a fourth argument to the cache set with the cache tag, and to expire it you call the invalidate tags function.

[00:11:05] What's very important about cache tags, and it took me a long time myself to understand, is that a tag is essentially a version number for your cache. Drupal versions tags automatically. You could use a counter, a timestamp, or a name. Core counts, in a cache tags table, how often each tag has been invalidated. So if node one's cache tag is invalidated, core knows node one should be at version 43, and anything tagged with node one must have that version or it's invalid. At first you wonder how this indirection works, but once you understand it, everything gets much easier.

[00:12:45] The power is that the cache item includes both the name and the version, and the canonical store, the cache tags table, always stores the current version of the tag. So whenever we set something with a given tag, what you don't see is that Drupal stores both the tag and the current version it's on. One thing that's important: everything in the same request uses the same current version. Back to the pizza analogy, the waiter checks the list of dough versions once a day, not every minute.

[00:14:04] So our shop now works the same way, except the waiter checks the expiration date and the tags, marks the pizza valid or invalid, and if it's not valid gets a fresh one. So far we know how to get an item, set an item, delete an item by name, use time-based invalidation with cache max age, and use tag-based invalidation. And we know core is cheating a little, because it caches those cache tags for the rest of the request. We used to joke that we solved cache invalidation, but if you look closer, we just moved it somewhere else.

[00:15:10] (Question from the audience.) Can a cache block a page load on a cache miss? Basically yes. If your cache is very slow to retrieve things, for example because of a network problem, then a cache can block the page load. We can even do that deliberately with stampede protection, which the slides cover.

[00:16:08] (Question from the audience.) How do you create your own cache bin, like the pizza bin, and what's the benefit? We'll learn more later, but the main benefit is that you can put different cache bins on different systems. The database, the warehouse across the street, is not the best place to store everything. It might be better to keep the pizza dough below the counter than to drive ten miles.

[00:16:40] (Question from the audience.) How do you preload a cache in Drupal 7? There's a module called APDQC, an asynchronous prefetch query cache, that can preload things frequently used on a page. And on using cache tags for real-time data from a third party: you need some way to know when the data changes. Either the third party gives you a heads-up and you synchronize the cache tag, or you run a cron job that checks the data and invalidates the tag when it changes.

[00:18:38] So what should you cache? Two years later, our shop is thriving and we offer new variations: a gluten-free dough, vegan mozzarella, a spinach pizza. The customer comes, and the waiter asks for preferences: would you like a vegan or gluten-free pizza? That's our cache context. The waiter checks for the right variation and either serves it or produces it and stores it. Again, we could put all those variations in the name, but then we'd get a vegetarian marinara that never makes sense, and we'd double the space. What we really want are buckets: you go to the pizza margarita shelf, then one tag for gluten-free, one for gluten, and so on down.

[00:20:41] Cache contexts are used for variation in Drupal 8 and 9. They're computed on demand and internally add the context values to the cache ID name. The end result looks similar to what we did before, but it gives core more room to optimize. Internally, every cache item has two things: the cache item itself, our margarita shelf, and the cache context, is this vegan, is this gluten-free. So when we look at the margarita we know all the variations it can have. There are little tricks too: if no one ever orders a gluten-free margarita, that variation never shows up. Then core adds expires and tags, so we have an efficient two-tier caching system, and this is essentially how dynamic page cache works.

[00:22:30] With intelligent variation, the waiter looks at the pizza's variations first, then asks the customer only the relevant question. If you ordered marinara, he'd only ask about gluten-free. Unfortunately cache contexts still only work with render arrays. It took Wim and me quite some time to understand them, and hopefully render cache will provide this in the future. A render array is how all Drupal rendering works. Before, we set up our cache and then called a make function. With a render array, we set up the cache metadata and a pre-render callback, and Drupal internally does the cache get, calls the function to prepare our pizza, renders it, and stores it, so we don't have to do all the get and set ourselves.

[00:24:40] If we add cache contexts, we could add them directly at the top of the array, but doing it unconditionally would also vary the marinara by vegan or non-vegan, which we don't want. So instead we do it in the pre-render: if the pizza is a margarita, add the vegan context; if not, only the gluten-free context. The nice thing about cache contexts is that if you know something varies within the tree, by user or by role, you can add the context right where you need it, deep in the tree, without hunting for the exact entry point of that block.

[00:26:30] Now the fridge is full, we have many variations, and a spinach pizza that's bought far less. A good idea when dealing with performance is to check your cache hit ratio and invalidations. Wim's project Cache Metrics collects these and can send them to New Relic, so you can see which tags are invalidated really often. Maybe it's not a good idea to cache the custom pizza at all. The easiest fix is not to cache it: somewhere in your tree you set cache max age to zero, and it bubbles up. You can also do this on cacheable objects with set cache max age zero.

[00:28:58] A full example: in the pre-render, if the pizza is custom, we make it, set max age zero, and return early; if it's spinach, we set max age zero and continue. One pitfall: if you add max age zero after the function has already rendered, that means we already tried to get the item from the cache. This happened to me more often than I'd like to admit. If an item is still in the cache, it will happily retrieve it, and you'll wonder why your max age zero doesn't work. You debug for fifteen minutes, then clear the cache and suddenly it works, because it still had an old version.

[00:30:26] There are three ways to clear tags: invalidate the render cache tag, do it via Drush, or call the function directly. A render cache tag is added to all render arrays, so there's a good chance you get the cache you want. You can also set max age before retrieving from the cache. If the pizza is custom or spinach, we set max age zero as a request-based cache policy, so we're not even trying to retrieve from cache. That's more efficient but not always possible.

[00:31:39] No pizza shop makes pizza entirely from scratch. The dough rests for twelve to twenty-four hours, and the tomato sauce is ready. In the same way, Drupal pages consist of a main page response, which you usually need to cache yourself except for dynamic page cache, plus blocks, menus, header, and footer as decoration. There are two ways to make a pizza with mushrooms: start with an empty pan, dough, sauce, cheese, then mushrooms, or start with a pre-made margarita and just add mushrooms. That second one is dynamic page cache: a margarita with mushrooms. Drupal caches most of the page and just adds the mushrooms on top.

[00:32:57] This is how I came up with the system. I was looking at Drupal.org and thinking only my tiny username is different for each user, and most of the experience is the same, so why can't we cache that? In Drupal 8 and 9 we can. We cache the response as much as possible, then add placeholders for very dynamic data. Blocks are currently the only things in core that automatically get this lazy builder functionality, which lets them be placed out of band. Note that gluten-free can't be a placeholder, it's a foundation of the pizza, so we need both variation and placeholders. It's a case-by-case decision.

[00:34:15] A placeholder can be independently rendered and must not depend on anything executed before it. In the analogy, you can't add more yeast after the dough has finished. Here's the top-secret trick: to create a placeholder manually, you put it in "attached," you put the same placeholder string in your markup, and you build a render array. That's it. The contract is that it's executed after all the other parts have rendered. From there we go from a plain placeholder to a lazy builder, which is a placeholder with a contract.

[00:35:19] A lazy builder can be a controller with a special syntax or a static function. You pass the pizza name, but you can't put an object in there, only simple values, because the array must be serializable. You can also explicitly say you always want a placeholder with "create placeholder: true" combined with the lazy builder. This is what lets us use Big Pipe, which is in core: you enable it and it's good to go. It lets us cache everything cacheable and render the rest, and it lets us break variation per page and per user, so we get a cached page plus a small per-user snippet.

[00:36:32] (Question from the audience.) When you update a cached view that lists five nodes, are the other nodes' tags regenerated? Basically yes. Listings are still a big problem, because if something new comes in or goes out, cache tags get tricky. But if you just update a title, the full view is regenerated. A simple real-life example of a cache context: an e-commerce site that gives a different experience by country. You get the country as an HTTP header from Akamai or a geo API, create a cache context, and you'll never serve non-country-specific content. That was difficult in Drupal 7 and is simple in Drupal 8.

[00:37:46] (Question from the audience.) What about placeholders? Placeholders are the topping on the pizza. Say you have a page that looks the same for every user except a small block that says "Hi Fabian." You make that a placeholder. In Drupal, if you use a block and vary it by user, Drupal already creates a placeholder for you. So Drupal caches the shared content completely without the placeholder, then executes the "Hi Fabian" placeholder.

[00:39:46] So where should you shop? The shop is even more successful, but customers have to drive two hours or more. It would be better if it were near them. The solution is a content delivery network, and Drupal 8 and 9 make it easy: you choose a CDN, enable the respective module, and profit. The CDN now does all the checks, like a supermarket checking expiration dates and whether the dough version still matches. You can see the headers yourself with a debug option: set the debug cacheability headers to true in services and you'll get the cache tags, contexts, and expiry.

[00:41:18] What about the dough itself? We don't want to fetch it from the warehouse; let's put it in the fridge on the counter. Drupal has ChainFast: it's APCu, shared memory within PHP, a key-value store. If you have things that seldom change, put them in a special bin and connect that bin to ChainFast. It's important that this is mostly read-only traffic. But don't store custom-made pizzas near the counter, and don't put things with lots of variations there. You can get serious problems if your cache gets full and locks up. Be very careful with what you put in ChainFast; it can bring e-commerce shops down. I've seen it during a load test. It's great for file cache, class cache, and config cache, things that seldom change.

[00:43:03] Don't forget Redis and Memcached. MySQL is the warehouse across the street, Memcached is the fridge in the next room, and APCu is the fridge below the counter. The times are roughly two to five milliseconds, half a millisecond, and 0.05 milliseconds. There are two kinds of caches to distinguish: caches used to create the pizza from parts, and caches used to deliver it, like page cache, dynamic page cache, and CDN cache. If lots of customers want a pizza with spring onions at once, we can cache it for a very short while, which is micro-caching. Stampede protection is built into most CDNs: with a shield, if a thousand people want the same page, it's generated once and then replicated.

[00:44:15] Common caching pitfalls are missing dependencies, variations, or putting max age zero everywhere because caching doesn't seem to work. Please don't do that. I hope this session helps you understand caching so you don't have to litter your code with max age zero, and so you can remove it before launch. If you need something dynamic, use a lazy builder and a placeholder. It helps Drupal make much better decisions in optimizing things. Just remember, a margarita with added mushrooms is much faster than making it from scratch. Know what depends on what, know when something needs to be invalidated, and use the render visualization tools to see it. Have fun, and I'll go make pizza now, because I'm really hungry.

[00:45:26] (Question from the audience.) A good cache tags use case is when you need to expire something, like a block on the front page: you add a cache tag and invalidate it whenever you want to expire it. On dynamic caching for authenticated users, the session or user can be stored in the HTTP request header, and if you do it on a CDN you need development effort, so you'd want someone like a performance shop for that.

[00:47:00] (Question from the audience.) On APCu versus dedicated caching servers like Redis or Memcached: you use both, you don't decide between them. APCu is the fridge below the counter; for Memcached you go into the next room, which is less efficient. If you want one dynamic block, the easiest is to make a placeholder and render your own code in it, then use an Ajax request to replace the placeholder.

[00:48:26] We're way out of time. Thank you very much, that was all the questions. Thanks a lot.