A joyful tour through the CLI now in Drupal Core

[00:00:04] All right, let's get started. This is a joyful tour of the CLI in Drupal core. Want to share a little bit about myself. My name is Moshe Weitzman, and I've been in Drupal for the whole time. My user ID is 23. I do consulting with Tag1 Consulting. A member of the Drupal security team, and a few projects that I maintain are listed there. The most relevant one here is the Drush project, which is the CLI for Drupal.

[00:00:48] But there's a new CLI in Drupal, and I'm really happy about it. I helped a team of people get this merged into core. You run it with two letters, DR. I like to call it Doctor. I would love if that became the name for Drupal's CLI. There's a little stethoscope icon that I decided I would use here. And so, well, let me show it to you. Or before I show it, I'm just going to talk about what the status is of Doctor.

[00:01:28] When Doctor was first merged into Drupal core, there were already some CLI commands that core had for installing Drupal, for applying recipes. Those are all in there and runnable by Doctor. We've put a bunch of commands in since then. Here's the list: extension list, so it will list all of the active themes and modules on your site; cache rebuild, status, cron, user login. Those are all in Drupal core now and runnable via Doctor.

[00:02:07] There are a bunch more that are proposed and in review, MR review: updatedb, a better install command, config import and config export, and more extension commands like installing and uninstalling extensions. So, you know, there's a lot of stuff here that is only in Drush and not yet in Doctor, and, you know, that's going to be the effort over the next year or two years, to get all this stuff into Doctor.

[00:02:43] A question I get all the time is like, why are we taking something that works, which is the Drush project, and moving it into Drupal core? There's some fear that all of the integrations we built are going to stop working, we're going to have to just rework things that have been working for years, and muscle memory and so forth. And so it's a fair question. I thought I'd give a shot at answering it.

[00:03:12] Here's four bullet points. The out-of-the-box experience of Drupal really matters to me and matters to other people. Right now, you start your Drupal and you can't do anything at the command line. You can't follow the install instructions that are posted all over the web. You don't have Drush. You aren't sure if Drush is the one you're supposed to get or not get. And so, like, newbies are just not having the right experience. And in order to fix that, you got to crack some eggs. And so that's what we're up to here.

[00:03:54] I think there's something to be said for people who are evaluating the Drupal project for their next CMS project. If they don't see a CLI in there, they don't think it's a serious project. You know, lots of other CMSs have a CLI built in. Drupal has it; it's outside of the core download. And, you know, it's hard to say how many people we're losing because of that, but let's not lose anyone. All right.

[00:04:25] Our documentation suffers by having our CLI be outside of Drupal core. We have to, you know, branch our documentation, say, "Do you want to do it with Drush? Or maybe you don't have Drush, and here's how you have to do the thing." And so we have branches and branches all over the place, and so there's not much clarity in the documentation. And then the agents consume it, and they don't quite know as well what to do. And boy, doesn't it make a lot of sense to have this thing be core.

[00:05:01] One nice thing about putting it in core is that we take advantage of all of the localization that core already offers for its strings. Okay, so command help, command confirmation messages, all that stuff is now translatable. So people who want to run their CLI in different languages finally can. It's something that Drush, like, gave half an effort to but never really got there. And so now we're going to piggyback on Drupal's localization server and localization everything. And yeah.

[00:05:40] And then there's the sort of bus factor of, like, what happens if myself and the other maintainers go away, get sick, lose interest. When you give something to the project, you kind of have more robust maintenance. So I think those are four pretty compelling reasons for why we're breaking all the eggs and moving things in. I guess I use that word, breaking the eggs. We're not breaking anything. Drush is still intended to be maintained for a long time, and people are going to move their commands, you know, in their scripts and so forth, at their leisure. You know, certainly Drupal 12 will have a functioning Drush.

[00:06:29] All right. So let's take a look at what we've made. All right. Hopefully people can see this. Can you see in the back? Good. Okay. All right. So the way you run Doctor is by prefixing vendor/bin/dr, just like vendor/bin/phpunit, PHPStan, PHPCS. This is one of those; this is how Composer likes you to do your command line tools. If you don't like the word vendor in there, it's pretty simple to use Composer to say, like, bin/dr, or wherever else you want to have it. That's a Composer feature, moving your bin directory.

[00:07:23] Okay. So you just saw me run the cron command. It's super fast. So that's a nice thing about having it in core. There's sort of less pre-flight work that we're doing, so there's even a few fewer milliseconds to running these things. Some of the formatting has changed, I think for the better. So in this case you can see that big green bar. That's the success logging. It has moved into these green confirmation bars.

[00:08:01] Just like all Symfony Console applications, you can say -v and -vv and -vvv for even more verbose logging. When you do that, you can see exactly when we're starting and when we're stopping each, you know, cron hook, and how long each one of them took. So if you get an error, you'll be able to debug exactly which one's erroring. And this logging happens automatically. Essentially, the Doctor pre-flight is adding its own logger to the list of enabled loggers, like the dblog module and so forth. The command line's another one of those. So anything that your commands log will automatically show up in the log output.

[00:08:58] So you can quickly get the status of your site and see what version of Drupal you're running and what themes you're running and your version of PHP and so forth. This list is shorter than Drush's status, and, you know, I'm counting on you guys and everyone to just add things as they become necessary. This is, you know, just a couple lines in an MR to add useful things to status.

[00:09:31] The sort of formatted output here is similar to what Drush did. It's really Symfony Console's, I think it's the list render component that they offer. If you look up SymfonyStyle, you can see all the different output types that you can do for these commands. I'll show you in a second. These are straight-up vanilla Symfony Console commands, so there's, like, nothing new to learn for people who are familiar with that.

[00:10:06] So we saw cron, we saw status. Cache rebuild, certainly used a lot, is already part of Doctor. I don't think we have more output when you... yeah, so you got to trust that all the caches were rebuilt. Okay. So that's what running Doctor looks like.

[00:10:41] Let's take a look at the code. We are looking at the cron command. The file name is CronCommand.php. And there's this PHP attribute that Symfony Console wants you to use now, the modern way to declare commands: a name, a description, optional aliases. In here are also your usage examples, an array of examples. And continuing on your class declaration, you don't have to actually extend any class; they can be bare classes now. Your autowired, injected dependencies for your command are here. Just by type hinting them, the right things get injected. So, you know, if you've used modern Drupal development, you've seen this. It's available now in these classes.

[00:11:59] When you actually write the method that does the work of your command, you can use the __invoke method. It's the built-in method from PHP. And by default, the input and output interfaces get passed. You always want to have your method return an integer. This is the Symfony Console's requirements. Construct an IO object of type SymfonyStyle, and then do the work. In this case, we're calling on the cron service from Drupal. We're calling its run method, and then we declare success and return an integer. That's another... Command::SUCCESS and Command::FAILURE are zero and one, provided by this Symfony Console Command class.

[00:12:53] And that's it. This is a simple command. That's all there is to it. You know, your commands can take arguments and options, and I'll show you how to do that. One thing I want to point out here is that this cron service is doing all the work, and I'd encourage you to follow that model. Do the work outside of your Drush command. Don't have, you know, hundreds of lines of Drush command. And a nice thing about doing that is that when you get to testing your command, you can often skip testing of Doctor. Do your testing of your service and not of your CLI command. You certainly can, and I'll show you how to do testing the CLI command, but you may not have to.

[00:13:55] So let's look at the test file. Again, more PHP attributes. This is part of the system group. Run tests in separate process; we have that all the time. Which class you're actually testing: CronCommandTest. And then we're extending BrowserTestBase, which is a core testing class. That's the one that does enough setup to make these things run smoothly. It's possible we'll find a different class we want to extend at some point, but that's the one these core tests are using.

[00:14:37] And then this stuff should be familiar to you if you've done testing with BrowserTestBase before: which modules you want to enable for your test, which theme. I don't think that matters here. And then you construct a CommandTester object. Instantiate your command, construct the command tester. CommandTester is provided by Symfony Console, so that's the way they recommend you test your commands. So we're essentially setting up a Symfony Console application with one command in it, the cron command. And we're calling the execute method, and we're passing no arguments. And then we do our assertions. Did our output look right? Did our return code look right? That's it. That's what a test looks like for Doctor.

[00:15:40] You can get exotic, and you can, like, make sure that, you know, this is really testing the cron service. But here, if one of the cron hooks declared an exception, the run method keeps going. It logs and keeps going. And so this is a test to make sure that that's working correctly. Arguably doesn't belong here, but it's okay. So this is what testing of Doctor commands looks like.

[00:16:25] Okay. So getting back to sort of Drush and how we're moving over. When you want to move, like, your contrib module command and so forth, my recommendation is that you keep two command files, one for Drush and one for Doctor. You can share code between them, namely that service, so that really just the glue code is duplicated. And you either have to do no testing, because you're testing your service in the normal Drupal way, or if you want to, I'd say just test one of them. Test the Doctor one. You can leave the Drush as sort of a legacy command implementation.

[00:17:14] Some things that we lose by moving stuff to Doctor and away from Drush. Site aliases are a handy Drush feature. There's kind of lots of alternatives for people who are interested. Your hosting platform may already provide a way to run things on your site. Terminus, ACLI, and Lagoon are ones that I've used in my consulting work. They let you run things on each of your sites and so forth. And here are a couple sort of non-hosting-specific tools, PSSH and Fabric, that I found. Those are tools that let you run commands on a variety of SSH hosts. So you can pick each site, each of your many sites, and run on 10 of them or a hundred of them, and run whatever you're trying to do. You're trying to updatedb, or whatever you're trying to do, use one of these tools. That really was part of Drush's scope because this stuff didn't exist way back when. Now it does, and so we can kind of remove it from the scope of Doctor.

[00:18:36] Output formats are a cool feature of Drush that we don't have in Doctor. That's what lets you take, like, your module list and get it in JSON or YAML or CSV. Basically, each command can implement that on its own. It's not a core feature of the framework anymore. It's a little bit of a shame. It was a really good feature of Drush. We wanted more CLI tools to adopt this. We separated that functionality out into the consolidation/output-formats Composer package, but we really didn't get much uptake on that. So we're going to follow the Symfony way and just say commands that want to be available in multiple output formats should declare those options and go ahead and output that way.

[00:19:34] All right. So, ways you can help out. Code review is super helpful. And if you're going to nitpick, please go ahead and fix the thing that you, you know, found slightly lacking. That way we kind of keep things moving forward instead of lagging for months. These commands, in addition to passing their tests, it's really valuable when people decide to do manual testing. So go ahead and check out the branch, run the command, run it with output that is not what's expected. Make sure the error messages are what they're supposed to be. Try the different output formats, and comment on the issue saying, "I ran it. It looks great."

[00:20:24] If you want to author commands, you can go look at Drush and see what it has and make an equivalent in Doctor. It's not hard, and it is very valuable. There is a spreadsheet we made when we launched this initiative back in DrupalCon US. That was, like, tier one, tier two, tier three commands, a prioritized list of stuff we want to work on. So if you want to look that up, it's in the CLI in core Slack channel. One of the pinned items there is this spreadsheet. So if you want to look at the high priority stuff, please check that spreadsheet.

[00:21:01] Of course, all of the Drush commands that ship with contrib modules are going to eventually move to Doctor. It would be great if you guys helped out with that. And once you do, please blog about it, social media about it. Let's kind of get the movement going, so the Doctor knows how to do more and more and more things. And just echoing what I said before, my recommendation is to extract shared code into a service or into a trait or something like that, and just keep two command files around, one for Drush and one for Doctor.

[00:21:47] There are other ways to do it. I mean, these Symfony command files generally work in both runners. So you can take, like, the same exact file and symlink it in two places, and it will work. Or extend one in the other one, and it will work. So if that offends you less, do it that way. But I like to have two different command files.

[00:22:17] And just some future ideas for people who want to, like, really make Doctor better. Laravel Prompts is a really cool open source project that Drush uses that we have not brought into Doctor yet, but I think we ought to. It really makes gathering input from the user much more pleasant. You know, selecting from a list, type-ahead, and so forth are really implemented much nicer here than they are in the bare Symfony Console. Code generation: if you want a new controller, or you want a new service, or you want a new access handler, Drush gen handles that. We need the same thing for Doctor.

[00:23:09] There's opportunities for collaboration with the AI initiative and the tools module, because running a tool, you know, a tool, and running some service from the CLI are really similar concepts. And so, you know, maybe someone could do some deep thinking and see how we can come together more with the tools ecosystem. I haven't thought about that too much myself, but I think there is something there.

[00:23:42] TUIs are all the rage, text user interfaces. You know, people who are interested in command lines are generally interested in these things. So surely someone is going to write the TUI for Drupal, and you'll be able to quickly block users and change passwords and all that kind of stuff. And I think that'll be an interesting, another way to manage your site. You know, and, like, org-wide dashboards, something that will go into your different sites and aggregate data and show you the health of your system. That's kind of another way that TUIs can go for Drupal.

[00:24:26] So, just some ideas of how people can pitch in and make Doctor better and make Drupal better. So I'm actually out of time. It was like a 20-minute talk. But yeah, let's, you know, let's talk, in here or out in the hallways. And yeah, thank you to everyone for coming, and thank you for all that you give to Drupal. And this has really been a joy to participate in such an awesome community for so long. Thank you.