Drush 12: Modernized

[00:00:01] All right, so you found your way to Drush 12: Modernized. Thanks for coming, thanks for getting to Pittsburgh and getting to our talk. The two of us put a lot of work into Drush 12, along with lots of contributors, and we're excited to tell you about it. My name is Moshe Weitzman. I'm an independent web developer, primarily working for Tag1 as a consultant. And this is my colleague, Greg Anderson.

[00:00:44] Greg Anderson: I'm an engineer at Pantheon.

[00:00:52] Moshe: So the first group of changes I want to talk about have to do with authoring Drush commands. Hopefully many of you have written your own Drush commands. It's a rather easy thing to do, and they're good for you: they're sort of self-documenting, and they're really easy for other people on the team to use because of that self-documenting feature.

[00:01:21] The first thing to say about Drush 12 is that it only runs with the newest version of Drupal, Drupal 10. So you'll want to adopt this when you start running Drupal 10 sites, or if you already are, then great, this is a good time to adopt Drush 12. Since Drupal 10 is PHP 8.1 and higher, that's what Drush is too, and we really embraced that restriction and updated all of our code to use the newest features of PHP. That helps us find bugs related to types that are changing, and it helps us communicate what is extensible and what is not.

[00:02:12] So the biggest change for authoring is that we encourage the use of PHP attributes instead of annotations. I'll give you an example of the way Drush commands used to be and the way they are in Drush 12. Right now we're looking at the annotations way of authoring commands. You can see this @command, and then the command name, XKCD fetch. It takes one parameter, two options, two usage examples, and an alias. Below that you have the actual method, which takes a bunch of parameters and does some stuff.

[00:03:18] When you upgrade to Drush 12, optionally you can start using attributes. It's encouraged, but we wanted to have a nod to backward compatibility, so all of the existing commands out there are going to run fine on Drush 12.

[00:03:45] So here's the new way, to upgrade at your leisure. If these look alien to you, that's okay, I think they look alien to everyone when you first look at this stuff, but you get used to it. The nice thing about PHP 8 attributes is that they're an actual language construct, so there's much better performance and much better autocomplete support in your IDE. Annotations, on the other hand, are completely separate from PHP. So the command attribute here: the name is one named parameter, and aliases are specified here. The argument, the two options, the two usage examples, most of this is just one for one, transformed from the annotation example into PHP attributes.

[00:04:59] As for the method name and the method body, there are no required changes. You can pretty much keep what you had. So going back to the presentation: strict types, command name constants, typed properties, return types, parameter types. Let's take a look. This class here is called user commands. If we look at the top of the class, we're declaring strict types, which helps avoid type changes for variables. This is true of all the Drush internals.

[00:06:09] All of our command files are declared as final, which means you can't extend them directly, or at least you have to do a bit more work with the reflection API. We're communicating that our command files are not an API unto themselves. Usually with Drush commands you have a service that does the actual work of the command; that's the API. But the command definition itself is not an API, so that's why these classes are final. Yes, people have tried.

[00:06:48] One other nice change we made for our commands, and encourage for yours, is that there are now class constants for each of the command names. So when you're writing scripts and need to call Drush commands inside them, you can use these constants and you won't make a typo. And you can tell what's using what by linking back to the constant.

[00:07:16] Here's a quick look at a new feature Greg's going to talk about later: a new static create method where you can set up the dependencies for your command file. I also want to mention that for a while now Drush has shipped with a generator called Drupal command file, DCF. If you run drush generate DCF, it gives you a skeleton command file. That will help you with your final and your class statement, and some boilerplate, all updated for Drush 12. That really is the best practice for how you start creating your own custom command: run our DCF generator.

[00:08:22] So in addition to commands, Drush has a lot of generators. Generators are the thing that generates boilerplate for you: if you're creating new Drupal services, new blocks, new access control plugins, all these kinds of things, you can get a kickstart with the drush generate command. In addition to all the generators that ship with Drush, there's the possibility of custom generators, and the custom generator API has changed a bit. We upgraded to version three of the Drupal Code Generator, which is a separate open source project that we wrap inside of Drush. So the net net is you have to slightly alter your generators for this new version.

[00:09:17] Here's what a new generator looks like. This is in fact the Drush command file generator I just mentioned. This also uses PHP attributes: here we have the generator attribute, which takes a bunch of named parameters, name, description, alias, and so on. This class-based generator is new in Drush 12, so you'll want to extend that class. And this interviewer object is new as well. The interviewer is a name I think is fantastic; I didn't name it, the maintainer of the Drupal Code Generator did. The point of the interviewer is to ask the user a bunch of questions before the code gets generated. You can see he's building an IR variable, that's the interviewer, and then it asks what machine name the class should get, what class name, and what services should get injected in. Then it's a matter of using the Twig file that ships with the generator and writing the result to this path.

[00:10:41] So that's generators. Hopefully you'll find them even a little easier to use in Drush 12 with this new setup and PHP attributes. Okay, I'm handing off to Greg.

[00:11:06] Greg: All right, I'm going to go over what it looks like a bit more, creating command files for Drush 12. As Moshe mentioned, we have a complete backwards compatibility layer, so you shouldn't have to change your commands at all. The layout for a Drush command is the same as before: there's a well-known location inside your module, drush/Commands, where you put your command definition, and Drush just finds it there. You'll notice the drush.services.yml file is gone. It doesn't have to be gone; if it's still there, the backwards compatibility layer will load it, but you can run without it now.

[00:11:47] In addition to module command discovery, we have a new PSR-4 discovery mechanism. Any library you add to Drupal through Composer has an autoload section in its composer.json file. What Drush does is use that information, walk through all the autoloader sections, and if it finds this well-known path, drush/Commands, it takes the base namespace from the beginning of your autoloader and adds your namespace followed by Drush Commands. Similarly, wherever you tell the autoloader to put your source, the source file is located in Drush Commands.

[00:12:44] The other thing to notice about this mechanism is there's really no way to do a drush.services.yml type thing; there's no good place to put the YAML. So instead, and Moshe already gave you a preview, we switched to a static create factory method. This is the same pattern you're already familiar with in Drupal if you've ever made a form or controller. When Drush is instantiating your class object, it notices the static create method is there and passes it the Drupal DI container, and you can pull whatever services you want out of that to pass into your constructor. Following good practice, make your constructor protected so that nobody instantiates it unless they're going through the factory.

[00:13:46] Porting from drush.services.yml to a static create factory is actually pretty easy. These things look quite a bit different, but if you open up your drush.services.yml, the first thing you'll notice is it's full of service names. I've got a couple underlined here, the config manager and the config storage export. So when you write your static create, you can use that same name, drop the @, call container get, and pass it on into your constructor.

[00:14:25] I'm going to back up one slide, because you'll notice that second underlined example from your drush.services.yml has a little question mark after the @ sign. That means it's an optional thing: the service doesn't have to be there and might be null. In practice, if people set up their services file like this, it's possible that if you call that method it's going to blow up, because of strict type checking. So the obvious solution is you just ask the container, does it have the service? If it does, then you call through. Much more readable, very PHP-like.

[00:15:08] Drush also had a mechanism called inflection, and maybe a lot of you don't know what that is because it was sort of niche. The short version is it's something you inherit when you extend the DrushCommands class. Inflection involves a marker interface that provides a set method, and if your class has this, Drush notices the marker interface and does injection automatically during creation. We're deprecating this; we really find that the static create is clearer. For now, most of the inflection classes still work if you're using them, but we completely removed the really unusual ones that aren't used anywhere outside of core. So that's one thing you might encounter when upgrading commands, because there's no BC layer for the removed stuff.

[00:16:11] Installation and execution. For a really long time we've been encouraging everyone to say there's only one supported installation method, and that's through Composer. But for Drush 9, 10, and 11 it just happened to work if you globally installed Drush through a number of different means; it would figure out what to do, and it would even merge together the two autoload files if it really had to. Not great, because things can very easily catch fire. So we're formally desupporting that. A lot of the Drush bootstrap code has gotten way simpler; all of that code is going away. Drush 12 now requires that it be installed in the same site, in the same vendor, as the Drupal site it's going to bootstrap. It won't bootstrap any other site except for itself.

[00:17:07] The upshot is that the root option is now vestigial. You can pass root to Drush 12 and it'll pretty much just ignore it, unless you pass a different root, in which case you'll get an error. We again want that backwards compatibility in case you're using Drush 11 on one machine to remotely call Drush 12 on another. But you don't have to type it in yourself any longer. If you just run Drush out of the vendor directory, you're going to bootstrap whatever site is there. So that's a lot simpler.

[00:17:49] The result is that the launcher isn't really necessary. We changed the internals of how Drush starts up commands; now we're taking advantage of some of the new variables Composer provides to find the bin directory and little things like that. The launcher doesn't do these things, which could lead to some strange edge cases, so we recently just desupported the current version of the launcher. Maybe someone might resuscitate it in the future, but it's not really necessary.

[00:18:26] Something that's worked for a long time: if you just add a relative path to the end of your bash or other shell PATH variable, vendor/bin, then you can call Drush as drush, and if your current working directory is at the base of a Drupal project, your shell finds it there. It's already a bash feature, so why have a wrapper if the shell has that feature? That's sort of our policy. Similarly, if you only have one Drush on your system, you could just put the full path to vendor/bin/drush in your PATH, and then you wouldn't even have to change the working directory, because Drush always finds the Drupal it's paired with regardless of your current working directory.

[00:19:24] There are a lot of ways to manage navigating through directories. I've thrown up a couple here. I have a little shell alias called fd, which is probably my most used shell alias. Instead of doing a cd to change the directory, you can do an fd to find a directory. Just the same way your terminal searches a PATH for executables, the fd path searches up a list of locations where you might have projects installed. This project also has a little startup fd that suggests places where you can find your projects, and it supports tab completion, so it's a fast way to navigate between Drupal projects or any other project. It's bash only. There's another thing out there in the open source universe called zsh z, a little program that remembers which directories you cd to a lot and then hints that maybe you want to go back somewhere you've been before. By using things like this and other bash utilities, we don't need to overload Drush with things that are already features of other programs, which leads to better maintainability.

[00:20:52] Site aliases are still around. We're really encouraging people to think of site aliases as just being other environments of the site you're working on. So keep your alias checked into your git repository in the site you're working on, preferably inside the self.site.yml file. Regardless of that recommendation, other sorts of aliases are still supported, so you can have aliases that point out to all of your different sites wherever they happen to be hosted.

[00:21:34] If you run an alias, the root option isn't passed anymore; Drush 11 can still figure out where the site is without a root, so this is okay-ish. And a vendor/bin is assumed. People, don't relocate your vendor/bin directory, but if you do, Drush is assuming you're consistent in where you relocate it to. As far as aliases are concerned, we're finding the community is moving towards wrappers like DDEV or Terminus or BLT; people aren't using Drush aliases as much. They just call some wrapper that will SSH over to the final place and get you to Drush. So I don't know that we're necessarily going to remove aliases in the future, but maybe if there was some other program like the launcher that also had aliases, then Drush wouldn't need to have it. All right, I'm going to pass back.

[00:22:53] Moshe: In addition to helpful tweaks to lots of our existing commands, we did add a bunch of new commands in Drush 12, and I'm going to talk a little about new commands added to Drush 11, since folks may have missed that during this never-ending pandemic. So the yaml edit commands are brand new in 12. There's a wonderful open source project called yaml edit by a Drupal developer, Matt Grasmick, and we decided to bring in those Symfony commands via Composer. Now you can run the yaml commands listed here and a couple more. The main benefit is if you're writing a script that has to do YAML stuff, change a key, change a value, remove a couple of keys, lint a YAML file, you don't have to figure out the sed to do that; you can just use these easy commands.

[00:24:20] The entity commands are also present in 11. I want to highlight the entity save command; it's pretty helpful. I use it when I'm developing something, let's say a save operation in Drupal, like the insert and update hooks for nodes, and I just need to save a node over and over again to make sure my code runs correctly on every node save. You can quickly run drush entity save, give it an entity type and entity ID, in this case node 12, and the command line will save that entity and your hooks will fire.

[00:25:17] Entity delete is useful once in a while. You can delete all content with a given entity, or just a given bundle. This is really useful; I've used it with pm uninstall. You can't uninstall certain entity type modules if their content is still around; Drupal's validators will say you're about to orphan your data, so uninstall denied. So entity delete is a way to get rid of data you don't need, and that lets the validator proceed with uninstall.

[00:26:02] The archive commands are pretty new, well, in a new iteration. They existed in older versions of Drush, went away for a while, and now they're back. Archive dump creates a tarball of your whole site, meaning the code, the files, and the database. That's useful if you want to make a quick backup or snapshot. People also use it when switching hosting providers; the providers know what to do with one of these tarballs, they untar it and commit stuff to git, and you're on a new provider. The new iteration understands how we now build Drupal sites using Composer, so it archives the composer.json and composer.lock along with the database, files, and code, and you're able to run composer install on the other side and have your code base again.

[00:27:18] We added three commands in the field definition category. This is useful when you want to introspect your Drupal site and know what fields you're running, and which widgets and formatters. These three commands provide a nice table of information.

[00:27:51] The maintenance commands were recently added to solve an interesting problem people have been seeing and maybe been perplexed about, like I was. During deployments, once in a while I'd do a prod deployment and the cache had no idea what fields I was running, and the deployment would fail, and now prod is in maintenance mode and won't work, and it's a whole race to fix things. What looked like was happening is the cron job was running every two or five minutes during the deployment, so the caches were getting populated while Drupal was in a state where it wasn't possible to know what was installed. So this command, drush maint status followed by drush cron, is an excellent way to avoid this. The maint status command will fail if maintenance mode is on. The first thing you're supposed to do in a deployment is turn maintenance mode on. With this as your cron statement, you'll never get to the second half, drush cron, if you're in maintenance mode. Once your site's out of maintenance mode, it's safe to run drush cron. So hooray for maint status, and hooray for this double ampersand that requires success on the left before the right will run. Once you upgrade to 12, I encourage you to change your cron statement to something like this.

[00:29:51] A few more things we've done recently. The drush.org website is always getting better. We now have release-specific documentation, so if you're running Drush 11 or 12 you can look at the docs specific to your version. We expanded what's available there to include all the commands core to Drush and all the generators, listed along with their arguments and options. In addition to regular command options, the global options are listed, so you don't have to run a special command to see those. And the drush topic command is a way to read the documentation we've all authored right at the command line. Let me show you the website. Release-specific documentation, right here: you can look at the 12 docs or the 11 docs and see exactly the information that pertains to you.

[00:31:32] The listing of commands is over here. Here we can look at a particular command; here's the cron command we were just talking about, and here's the example we just discussed. This command doesn't have arguments or options, so you go right into the global options, the most common ones you need to know about. Any topics relevant for your command are listed along with the aliases. I want to call attention to the legend at the bottom of each command page; it tells you how to interpret what you're seeing, what it means if options have square brackets or not. It's nice to have a URL for each command that you can share and ask for help with.

[00:32:53] Similarly, all of the generators are now getting their own page on our doc site. Drupal 9 added bundle classes; if you aren't using those, they're super cool, and you can generate a bundle class using this generator. If folks haven't used the search feature on this site, it's fantastic; it comes up immediately, and here there are nine matching documents about cron. This website is a static site generated by MkDocs and Material for MkDocs. If you're working on documentation-related websites, I'd highly recommend that system; you just write markdown files and they make an excellent documentation website.

[00:34:22] I referenced the drush topic command; I want to show you what that is. So I ran drush topic inside my Drupal site, and here you get 18 different choices to pick from. These are all the markdown files the community maintains around Drush and how each part of the system works. So if you're a pure command line person, you have this option for reading the docs; if you're a web browser person, read them on the web. It's the same markdown files, just made pretty for the web.

[00:35:38] The next topic is shell completion. We took advantage of advancements in Symfony Console version six, which has integrated completion. If you configure your shell correctly, you don't have to type out argument names or option names fully. You can say drush core and it'll give you the option of running core cron, core edit, or the different things in that category. Folks are encouraged to enhance their shells so you don't have to type that stuff out; option names can be long, so it's a nice time saver. The mechanism is documented on the installation page; go take a look and you'll see how to configure each of the three shells, bash, zsh, or fish. This feature is largely inherited from Symfony Console, so if you have experience with other console CLIs, Drush configures the same way. Completion on argument values is a bit more complicated; we support some of those now, and if folks want more supported, feel free to submit a pull request.

[00:37:24] We noticed the Composer team added a great new feature in Composer 2.2: a composer audit command. Audit looks at all your dependencies and tells you which ones you're behind on from a security perspective, which have outstanding SAs that you need to take care of ASAP. That was formerly the responsibility of drush pm:security-php; we offered the same service looking at the same data set that Composer does. There's no need for the two tools to exist, so we've dropped ours in Drush 12; use composer audit instead. We kept a command we've had for a while, pm:security, which specifically looks at your contrib modules and tells you which have SAs against them that you're too far behind on. If and when the community gets the SAs for contrib modules into the corpus that composer audit looks at, we'll deprecate this command too, but for now you need both to make sure your PHP is up to date.

[00:39:01] Okay, that's the end of the formal slides. I'd love to open the floor for whatever's on your mind; let's have a conversation about Drush, and then we've got a special treat at the end.

[00:39:26] (Question from the audience about removed commands.) That security-php we just talked about is gone. I can't think of other commands that are gone. When we deprecate a command, we actually keep it around just for the sake of telling you what to do instead. So just run what you're used to, and if it doesn't work, we'll tell you what to do.

[00:41:10] (Question from the audience about entity delete.) We're using Drupal's API for deleting the entity, so we do an entity load and then entity delete. I don't think there's multiple handling in Drupal for deletion, so you actually have to delete them one by one; you load them many at a time but delete them one by one. You asked about batch handling: I think there is batch handling, so we'll load up 50, delete them one by one, then load up 50 more. The command line doesn't tend to do as much batching as the web does, so I think it all runs in one process still.

[00:42:03] (Question from the audience about archive restore.) The question was about archive restore, and whether I can run that on a site that isn't functional, like a non-bootstrapping site. You can; you'll have to make it functional after everything is restored. It's going to use the Drush SQL commands to load the database, so if your database server is running, you'll get the database and the code back.

[00:42:59] (Question from the audience about transforming entities.) The question was whether there's any thought to a command that takes a bunch of entities and transforms them in some way. One good way is to write a Drush command or a Drush script that loads all the entities and does the transformation you want. You can also write a hook implementation of insert and update, temporarily or forever, and then run the entity save command, or run it through all of those. So I think that would work too.

[00:44:19] (Question from the audience about exporting config for a feature.) The question was whether there's a way to export all the config related to a feature. That's not part of core Drush. People have tried stuff like that; the Features module sounds like it's like that, and the config community has quite a lot of innovation there. I'd look to config devel and config manager; I can't keep track of all the synonyms for config commands, but I'd look to that stuff. That's a bit outside the purview of core Drush, so we haven't quite gone there.

[00:45:29] All right, I just want to thank everyone for coming, and hope we can have a little celebration as I tag the 12.0.0 release right here at the end of our session. This is all it is. If you thought there was more to it, and then this magical button. Somebody converted a lot of commands. Let's continue and I'll keep refining. This is no longer a pre-release, this is the real release, the latest release. Now it's published. All right everyone, great talking to you; we'll see you in the hallway.