Scolta: Drop-in AI Search Without a Search Server

[00:00:04] All right, we are so excited to get you guys right after lunch when you're really awake and you're hitting the food coma time of the day. So we're excited to talk to you about, well, the stuff that you created, this new AI search on Share My Lesson. But before we jump in, we're going to introduce each other. So this is Jeremy. He's the man with the vineyards. Been working with the Tag1 team for over 10 years, and they rebuilt Share My Lesson in 2016, which we'll talk a little bit about in a bit. Jeremy founded Tag1 in 2007. If your Drupal site crashes and burns, Tag1 is who you call, kind of like Ghostbusters. It's probably who your hosting company calls, too. He's been contributing to Drupal since 2002, when Dries reached out to him. His site, KernelTrap, was one, Dries, sorry.

[00:00:59] KernelTrap was one of the early sites that made Drupal popular. Some of you have read it. Some of you are pretending like you didn't. I have actually no idea what that means, but again, I am a client, not the Drupal community. I've been working in a Drupal website, but these guys keep me sane. So he built Goose, a load testing framework in Rust. And I also don't know what that is. And now he's built Scolta, which I do know what that is. He lives in Tuscany. He speaks Italian, at least sort of. His daughter questions his Italian. Another Italian in the room also questions his Italian. And let's be clear, he owns some vineyards, but really it's his wife who owns those vineyards and runs those. So when I say trust Jeremy is, no, I'm kidding.

[00:01:49] When ChatGPT came out, Kelly did not panic. Okay, she panicked a little bit. And then she went and learned everything. Her worry was simple. If any teacher can generate a lesson in 10 seconds, what happens to two million teachers sharing lessons that they actually taught? So she did two things. She started teaching teachers how to use AI themselves through AI Educator Brain, so no startup could charge them $20 a month for a prompt. And she made sure that the real human-made lessons could be found. American teachers earn about a quarter less than other college graduates, and that's a much bigger gap than in most of Europe. So this is how I ended up asking if I could fix Share My Lesson search.

[00:02:36] And I said yes. But before we get started, we have some breaking news. Good afternoon, Rotterdam. I'm Kelly Booz.

[00:02:46] And I'm Jeremy Andrews. Here are today's top stories.

[00:02:49] A seventh grade teacher searched Share My Lesson for photosynthesis and got zero results. Our search engine called it working as designed.

[00:02:59] A Drupal site's Solr cluster celebrated a full year without an outage. That's because it was a staging environment.

[00:03:07] A nonprofit tech team rated AI search an XXL task and moved it to the backlog. It joins "fix the footer" and "find out who has the Twitter password."

[00:03:20] Is that X? Yeah. Formerly known as X. Yeah. Or formerly known as Twitter, I should say.

[00:03:26] In an exclusive report from the Tuscany Connection, a man in Tuscany says search works without a search server.

[00:03:33] And sources say he also owns vineyards, except I think it's his wife who owns them. We'll be right back after this pre-recorded, well, pre-recorded demo. But before we do that, a little bit more about us, and let's go into the demo. Remember, you're controlling. Yes. I'm on it.

[00:03:55] So, two quick demos. You're going to see all of this again. This demo was built from nearly 7,000 featured Wikipedia articles. And to start, we have search pointing to Solr. I'm going to have to speed up. If you want to find articles about Napoleon, but you don't know how to spell his name, you get zero results. Solr's never heard of him. If you spell it correctly, it has all this information about him. It's a common class of errors. It's a big problem for real websites, and it's a big reason why Kelly's here today.

[00:04:24] The second one is the exact same search, but this time hooked up to Scolta. Same problem. You can't spell Napoleon, but this time it helpfully suggests to you when you misspell the name. It will, are we pressing play? It, yeah, it's... All right. It was playing. Just trust us. I don't know the two-minute word. There we go. When you misspell it, it's able to make suggestions about what you were trying to say: Napoleon Bonaparte, French emperor, Napoleonic Wars. And so it finds all that information even with the typo, gives you a helpful summary. There's facets that you can use if you want to click and tune into, you know, what exactly it is that you're looking for. And of the two demos you saw, the better search truly does not have a search server.

[00:05:16] All right, so let me talk a little bit about Share My Lesson. Share My Lesson was created by the AFT in 2012. I've been with the site almost since, I should probably see, I wander and I talk. I've been with the site almost since the very beginning, starting in 2013. We have 2.2 million members on Share My Lesson. Some of them are part of the union, some of them are not. It is a free resource site for educators in the United States. We also have some international folks, but our target audience is really United States educators. We have over 420,000 resources. We offer a lot of professional development, which teachers in the United States need to get recertifications. So they're constantly doing what you guys are doing right now and getting some professional development, but then they get certificates and recertification.

[00:06:03] So one of the big things is that the human connection is super important to us on Share My Lesson. The content that's coming on Share My Lesson is coming directly from teachers who have tried a resource or tried a lesson, and they upload it to Share My Lesson to share with the community. And so, as you were saying, you know, when AI was coming, and I saw it coming, I really did quite panic, because I mean, anybody could just go to ChatGPT right now and create a lesson plan. So how do we stay relevant when we're going through this, and making sure, like, our goal is ultimately to save teachers time. They already have, I'm married to a high school teacher. I know how much time they have to spend preparing and grading and all this stuff. So to the extent we can help save teachers time in finding the content or getting professional development that they don't have to pay for is incredibly important.

[00:07:02] And you're nodding your head because I haven't said the last part. And Jeremy was right. Is that why you're waiting for me to say... What was that? Exactly. It's in the notes, and he's like, I was going to skip it, but... But yes, Jeremy was right at the beginning. I really was very concerned about how do we stay relevant in this world of AI, and how do we keep that human connection of educators working with educators.

[00:07:27] You didn't save me much time, so I'll be quick. These are some of the organizations that Tag1 works with. What they have in common is their sites can't go down. If something is slow or broken at scale, we're who they call. Some of the people who maintain Drupal core work at Tag1, and so a lot of what we learn and fix for clients also ends back up in Drupal. We'd love to put your logo up there.

[00:07:51] That's the sales pitch right there. Okay. So I want to talk about Share My Lesson and the search, and what we and Jeremy have come up with to help solve problems that we've had. So meet Miss R. It's Sunday night, 9:40 p.m. A seventh grade science teacher, let's call her Miss R, is planning for Monday's class. She types photosynthesis, like the example we had earlier, and spells it incorrectly into our search box, with zero results. She's a teacher, so she doesn't give up. So she tries again and types "how plants make food." Now she gets results: a garden club lesson, a school lunch nutrition guide, and a food drive flyer. This is all technically correct, but not at all helpful for what she was trying to do. So she closes the tab, and we lose this person from Share My Lesson, and she may not be coming back.

[00:08:43] And that's been our search for many, many years using Solr. So our search has been running on Solr up until more recently, and it did what Solr does, which was an exact match or nothing at all. So when it did find something, it matched the word, not what the teacher actually needed. And we had a wish list of filters we'd been told Solr just couldn't do for what we needed to support our educators. So, the XXL task. I'll just also mention, there we go, I'll also mention that our social media person gave up searching for resources that she wanted to promote on Share My Lesson, and actually would go to Google to search for the Share My Lesson resources because she couldn't find exactly what she needed.

[00:09:30] So in March of 2025, we wrote down kind of our, you know, big goal list in terms of our AI goals, and better search was at the very top. It was also rated XXL in the backlog. So, new infrastructure, new license, new costs. I'm hearing ding, ding, ding, dollar bills. A lot more budget asks that I have to go do. So it was the kind of thing that we, you know, wanted to do, but I didn't know if we'd ever have the budget to do. And then Jeremy proposed fixing all of this with Scolta, and turning off the search server completely at the same time. So, Jeremy, what is search without a server?

[00:10:14] Yeah, it's search where the index is just files, and search runs in the teacher's browser. So if you look at this picture on the left, it's probably familiar to everyone here. Every keystroke you make is a round trip to a search cluster, which should be three nodes if you're careful, or maybe it's just one if you accept downtime. If you run the cluster yourself, then you're familiar with making security patches, JVM tuning, disks filling up with index segments, and long reindex times with major updates. You can also choose to pay for hosted search from Elasticsearch, OpenSearch, another provider, but most of the complexity we talk about is still there.

[00:10:53] Scolta removes the cluster. And so instead, when Drupal publishes or updates a page, it writes the index as small compressed files. As a visitor is typing in their web browser, WebAssembly is running in the browser, and it's loading just the pieces it needs from the index, which typically, those pieces are tens of kilobytes, and the browser is able to cache them. They're static files, so they're also cached at the CDN level, and it scales just the same as images on your website.

[00:11:28] Under the hood, just to briefly talk about how it works, there's a technology called Pagefind. It's an open source tool that was actually built for static sites, for searching static sites. With Scolta, we also wrote an indexer in PHP, and it allows you to install this anywhere, anywhere that Drupal runs. It doesn't have to be a powerful server. It can index your site, and it can also update the index in place, something that Pagefind couldn't do. Scolta also adds a ranking layer, and the front end is actually written in Rust and then gets compiled to WebAssembly. Scolta also has AI, but we'll get to that later. Without the AI, it's just normal full text search, ranked relevance. There's nothing exotic. If you swap out Solr for Scolta and don't turn on the AI, you lose the server, but you gain speed.

[00:12:21] If you think about it as a two-story house, downstairs is search and upstairs is AI, which is optional. So the ground floor, wait, we're in Europe, the first floor is two pieces. When you publish something, Drupal builds the index as you go, as you publish, and the second is the browser searches as you type, using WebAssembly and the local cached files. So this can take 50 milliseconds for the search. If you stopped there, you have complete search, no server, but no AI.

[00:12:55] Upstairs, we've already put it up, there are three pieces. Three is query expansion. A fast, cheap model gets the search query that's typed and also a sentence describing the website, and then it returns, saying, okay, well, I know this is a lesson plan site, so it suggests words a teacher might use for that exact same search that they typed, allowing it to expand how it actually can find things. That gets passed back downstairs, where locally in the browser it searches on each of the expanded queries, and then it merges it all together. The fourth piece is the AI overview, where the results are then shipped back to a more powerful model that provides an AI summary based entirely on the results that were found on your website, so it's grounded in what your website says. The fifth is an optional piece, allowing for follow-ups so that you can actually discuss the results. You could discuss the AI summary and ask questions about it and better understand it.

[00:13:57] Yeah. Okay. So, but wait, you said no server, but you're talking about using a server. I'm confused.

[00:14:08] Yes. So with AI off, it is literally true that the index is just a folder of static files. The scoring happens entirely in the browser, and the search doesn't leave the visitor's machine. But if you turn on AI, there's still no search server. You already did that. Okay. There's no search server, but the browser talks to Drupal, which is the site you already have. Drupal has the API key. It caches the requests. It sets rate limits, and it calls the large language model to get the added information. One thing this changes is the cost model. If a site is idle, there's no cost. And a repeated search gets cached, so again, there's no cost for repeated searches. And the model's hosted, so there's nothing you have to do there. It's all external. So even though this is AI search, one last thing to understand is that there is no vector database. There's no embeddings. The intelligence is added when someone types the search. It's not when you build the index.

[00:15:12] Great. And so Share My Lesson is a big site. We have over 120,000 indexed pages on Share My Lesson, seven facet dimensions, and well over 1,000 taxonomies. Oh no. I think I need you. Okay, great. I must have touched... well over a thousand taxonomies covering every subject area, starting in pre-K through 12th grade, and then we've added higher education and a bunch of other audiences. And it's not just the grades and subjects, it's then subtopics that go underneath all of those, resulting in nearly 4,000 filter values.

[00:15:50] So hopefully the obvious question everyone's wondering is, we used to have a dedicated server holding this index. How is the browser now holding the entire search index, and how big is it? When it comes to Share My Lesson, it's compressed, but even compressed, the index is about 325 megabytes. The good news is nobody actually has to download the entire index. When you do your first search, it downloads the WebAssembly and it downloads a tiny little index into the index files, and then as you type, it figures out which of those files are actually needed to build the search, and it just downloads those, which are like tens of kilobytes. So your browser sees a tiny, tiny fraction of the index.

[00:16:35] Okay. Great. So, in theory. All right. So I'm going to show you, this is the before. This is our previous search on Solr. And so I'm intentionally mistyping photosynthesis to do the search as we go, and I get no results, which is annoying, and it's just like, maybe you misspelled it or spelled it incorrectly, which is annoying. So now I'm going to type "how plants make food" to see what those results come up with. And it will scroll down. And so there's some stuff here. Still not exactly what I was looking for, especially if I was trying to find something on photosynthesis. So we have that, and now I'm going to say I want to know "the president with a tall hat" and see what comes up. And it comes up with something that is completely irrelevant to that search. So you can imagine, like, we're having people being like, peace out, Share My Lesson. I don't want to use you, because this is not great.

[00:17:36] So now, this is the new one using Scolta. We're going to go through those same three prompts. And that's a typo. That's a yes, an intentional typo. And you can see not only does it understand what I was trying to do, I'm getting much more relevant results from the site. And thanks to some of this dream team from Tag1, they've made sure that all of the attachments are now being indexed on the site too, so you can find some good content. And so this is an example of the overview, the AI overview that Scolta gives. And it's, you know, serving up a lot more content that is much easier to find now. And this is an example of a lesson page we have on the site.

[00:18:21] Okay. So now I'm going to do "how plants make food." And check this out on the very top left. Not only that, it gives me related searches to look at. So photosynthesis, other options. I can scroll down and see, you know, what was showing up for "how plants make food." It looks like there's some great content here. In fairness, I'm a civics educator, social studies person, not a science person, so I like the next search better. But you can see this, and then I can go, you know, back to photosynthesis to see what is displayed.

[00:19:06] And now I am going to ask about a president with a tall hat. Let's see if something relevant, and more than one, comes up. And look at that. Abraham Lincoln shows up, a man with a tall hat. And so I can see, look, there's this great collection of curated resources for Presidents' Day, and I can see what's in these resources. And for my Americans in the room, yes, there should be powers and limitations based on our Constitution for the role of the president in the United States. And we have great lesson plans for that, if you need a civics lesson for anybody in the United States.

[00:19:43] And then I decided, I really love this. This is so great. So I typed "I love this." And the nerdy educator side of me really loves the results: you know, Shakespeare, different books, "How Do I Love Thee," more of those types of activities. I really completely enjoyed what was displayed. So, you know, this is still very new, and I can't wait to measure, I mean, maybe even a month, maybe not, I'm not sure. But yeah, we're going to get people back. They're going to be so excited and thrilled to be with us. And so I think I'm turning it over to Jeremy.

[00:20:28] So we were excited also to launch it, and we were demoing to them how great it was. And we're like, "Hey, check this out. We'll search for math," and it spun, and it spun, and a very simple word like math took 36 seconds, after all our claims about how fast it was. We tried a six-word query, and that one took almost a minute. Photosynthesis, spelled right or wrong, one second. So it wasn't the size of the index, but we did figure out that it was common words that were slower. So obviously the problem was AI. We turned it off and saved 13 milliseconds. So no, it wasn't the AI either.

[00:21:05] The real cause, it turned out, was how Pagefind works with faceting. It basically recomputes the facet counts on every single search. And that's fast. It's about half a millisecond or one and a half milliseconds for every result. But then it has to multiply that out by all of the facets and the taxonomies, and basically across 4,000 filter values, and it adds up to seconds and seconds of effort, which is what was so slow. On top of that, there was a bug in our code. I can't remember who wrote that bug, but it's probably me. Scolta was needing those counts, so it calculated them itself. So not only was Pagefind doing that, but then Scolta was. So, 15 lines of code to make Scolta use Pagefind's counts, and that got rid of half the time, which was still really, really slow. And in the end, what we did is we made Scolta actually cache the facet counts, so now it's just a lookup. And the long and the short is that math went from being a 36-second query to now a 228-millisecond query there.

[00:22:18] Okay. So you'll notice when I make the slides versus when Jeremy makes the slides. I have a little fun. So... What are you saying? I know, exactly. Yours are boring. Mine are way more fun. Jeremy, you know that I wear a lot of hats. I have noticed. As director of Share My Lesson, I'm also a privacy officer. I am a school board member, actually, I really am, in Alexandria. And when we decided to bring AI to 2.2 million educators, I found out I have quite a few personalities. I've met all of them. Yep. Mostly by Slack, usually at 11 p.m. his time, since he's in Italy and I'm in the US, in Virginia. So I want to introduce the many personalities and questions, and this is a different way to do the frequently asked questions.

[00:23:04] So let's start off with Priya, the privacy officer. She has read every terms of service and privacy terms, and she's the reason that laptop has a padlock on it. Priya wants to know, where does our data go? So the search itself does not leave the browser. Only the AI steps go to the provider that AFT has picked, and that uses AFT's account. There's no per-user log. So no cloud somewhere or anywhere? Well, it's your cloud, your key. And if you really wanted nothing to leave, you could host the model yourself, but that's complicated, and I wouldn't recommend it. That's a sales pitch there.

[00:23:44] Okay. So this is Leslie from Legal. She's never once said "sounds good" without an asterisk. Leslie wants to know, are they training on our content? Not if you don't want them to. You choose the provider, and most paid providers let you opt out of training. Obviously, if you did go with a self-hosted model, that would never even leave your infrastructure. Leslie would like that in writing, please. It's in the slide notes. Oh, there we go.

[00:24:12] All right. So Miss Rodriguez teaches eighth grade science. She has caught every student who has ever used AI on a lab report. Her question is, is it making stuff up? No. The overview and the follow-up questions come only from the results that the search found. So even if the model knows more, it's only going to reply with what it finds on the site. So it shows its work? Yes. Well, it gets creative only in one case, and that's on purpose. When it does query expansion, it's using its own knowledge, based on what it knows about your site, to figure out other ways to ask the same query. And I will also mention, too, if you do go to sharemylesson.com, it's a free site, so you can even play with search and search for different terms while we're here.

[00:24:58] Okay. So this is Barb. Barb is Barb in Budget. She has a red pen, a calculator, zero patience. She can find a $3 discrepancy from across the room. Barb wants to know, what does search cost? And Barb says, don't round. So, worst case, it's about a penny for each search that uses AI. Just a penny? Yes. And repeat searches come from the cache, so they're free. Anything that happens in the browser has no cost to you. That's great. Look, Barb's smiling. She never smiles.

[00:25:31] All right. And then we have Denise, the content person. Denise runs all the content. She has lots of sticky notes. She wants to know what teachers need before they may know that they need it. Denise's question is, can we see what people are searching for? So, by default, there is no logging. Everything happens in the browser. And that's something important to understand if you install it, because that probably will surprise you. What we can do, and what we recommend, is that we can send aggregated search terms and filters to whatever analytics package you use, like Google Analytics, if you need that information. Great. And we do. So, yes.

[00:26:04] All right. Stick with us. Last two. This is Miss Thompson. Miss Thompson has taught AP history for 31 years. She still has a chalkboard. Didn't even go the whiteboard or the Smartboard route. She loves it, and she wants AI nowhere near her or her students. So, can she turn it off? Yes, in the profile you can turn it off. She still gets the same search that you had when you had Solr, but there's just no AI layer on top anymore. So turning it off, no AI does not mean bad search. Exactly. There was a shrug there. Well, if you do a typo, you won't get any answers. There you go.

[00:26:42] All right. And finally, every new teacher: brand new teacher, first-time teacher, third coffee, tote bag full of papers, and she just wants to know the answer. She needs an answer, like, right now. Like, I need to search, I need my professional development certificate, or PD certificate. So Scolta has a concept called priority pages. And what it does, it can recognize searches. You can configure it to recognize certain search terms, and then it'll insert pages always when those searches are done, and that will help ground the responses. Yep. And the AI summary uses it too. Yes. It's actually added as a search result, so you can see it in the results, and when the AI summary writes, that's one of the results that it's working with. We do the same thing on the Tag1 website. We also have Scolta there, and it grounds, like, if you search for Drupal, it will pull up our Drupal resources to make sure that it says the right things.

[00:27:37] Great. Well, somebody get this teacher another coffee, or the bottle of rum that Martin talked about earlier this morning. All right. So here are the actual questions and answers, in case you didn't want to have the moving. But to sum it up: our content, our index, our key, our rules. Share My Lesson owns it all. You own it all.

[00:27:59] Okay. So basically, you know, you saw the demo in terms of what's changed. The Solr cluster is no longer part of the architecture. The servers will be decommissioned soon, I am told, once there's apparently a budget for it. Thanks. Which, frankly, is making, again, our social media person very happy, because now she can actually find what she needs. And so search has become a roadmap, search stopped being a complaint for us, and is now much more exciting. And we're still going to be looking at how do we better promote and tweak what we want to make sure that we're serving up. So for example, that Presidents' Day is a collection where humans curate some of the best content on Share My Lesson. So we want to make sure that if somebody's searching for president, we can elevate collections. And so the team is also starting to trust our search, which is also a big thing.

[00:29:00] Yeah. So now another quick demo, with the purpose of understanding that installing this is quick and easy. It's stock Umami, which you've probably used before. You pick an AI provider. We're using a built-in demo from amazee.ai, which, you just click a button that says "try it," and it lets you, with no email, without taking any information. You have to name the site and describe it in a sentence. Oops, I got ahead of myself. So yes, the amazee.ai, if you watch this really quickly, this is all there is to it to set up. You click that button and boom, it works, for a free trial for a little while. If it runs out, then you can choose to give them more information and pay for it, or use something else.

[00:29:39] Configuration is pretty straightforward as well. You have to add a site name and a description, and that's how query expansion works, and AI summarization. So it's important that you put useful information there for that, but you keep it nice and short. In this case, for Umami, we also have it index the recipes so that search actually can work on recipes. And beyond that, we tell it what kind of website it is, which is kind of a shortcut for tuning it as best as possible for, in this case, a recipe site. And that's it. Standard Drupal. There's nothing complicated there. There's no external services beyond the LLM. We save that, and then we build the index. It's instant because Umami is small. It's a few milliseconds per piece of content. And finally, we have to place the search block, and that's it. So everything you see here is familiar if you've done anything with Drupal at all.

[00:30:37] Once it's placed, we can test it out. And this is all running in your browser with Umami. Nothing special here. Search for chocolate cake. One result comes back, which is important if you actually search for chocolate cake. However, query expansion finds other things that qualify, which is handy. The AI summary there works like you would expect, and then we take a follow-up and say "mine needs to be dairy-free," and it says, oh yes, we have a brownie mix that's exactly what you want. Which is good, by the way, too.

[00:31:15] Oh, yes. So beyond what you saw there and what was demoed, just really quickly, a few things that it can do. Expansion can also understand what you're asking. And so if you typed "newest physics lessons for ninth grade," it can add the sort by date and pick ninth grade, so that it's ordered and filtered down by exactly what you're looking for. If that returns no results, then it'll view the unfaceted version. It also has configuration so that you can add additional guardrails. For example, we have healthcare clients, and on those sites the guardrails would say "don't diagnose." Scolta can also scrub things like phone numbers and tax IDs out of the query logs using regular expressions. There's a lot more work that needs to be done there. We've talked about it a bit. It needs to be improved.

[00:32:04] Multilingual search is basically free, because translation is the technology that LLMs were first built for. You can search in Spanish on an English site, or you can search in, you know, English on a Spanish site. Expansion handles it perfectly. The AI can also be configured to reply in the searcher's language or in your site's language, whatever makes more sense for your site. And finally, this week I've been working on a grounded chatbot, which is everything you see here, but instead it's just, you know, like the Drupal chat, where you can talk to the site instead of searching the site, if that's what you prefer. It's aware of where you are on the website, and yeah, it's currently a proof of concept. It should be in the next release of Scolta. And we want that already. Yeah.

[00:32:54] All right. Scolta is open source, all of it. The Drupal module and WordPress plugin are... everything else is MIT. You probably have noticed right there, not just Drupal. The same index, the same browser code, the same AI works also in WordPress, Laravel, Django, Wagtail, Next, Astro, multiple languages. And so you can use it in all your projects, and it works the same everywhere, too. It'll feel the same, and you configure it the same. Also, if you really just like Solr and you're happy with it, what Scolta does are ideas that you can still build on top of. You can still do query expansion. You can still merge the results. And you can get, you know, grounded AI answers in what the search has found. I'm happy to talk about, you know, how to get that working if you don't actually want to switch to client-side search. So you can find Scolta on drupal.org. The source is on GitHub under the Tag1 domain, and give it a try and come talk to us.

[00:34:02] Great. And I think we actually have time for question and answer, which both of us did not think we were going to have time for Q&A today, because when we were going through this yesterday, it was a lot longer. And so this is just a fun, like, cheers to getting Sunday nights back again. Our whole goal at Share My Lesson is to help teachers get the content that they need, when they need it, as fast as they need it. But Q&A, we've got maybe eight minutes or so left. I will walk around the mic.

[00:34:32] (From the audience: Hi. Thanks. Very exciting. Does it still charge when you switch off the AI?) Does it still what? (From the audience: Is there a rate when you switch off the AI? If you switch off the AI?) I'm missing the word. If... (From the audience: Does it...) Yes, there is... there does what? (From the audience: Charge. I mean, AI.) So the only cost to using this search is talking to the LLMs. So if you turn off AI, it's not talking to LLMs, and no, it's completely free, and it all happens in your browser. Thank you. Sorry about that.

[00:35:06] (From the audience: One question. What type of content types or media types is it possible to index? Like web pages, PDF, audio, video?) Yeah. So it is a text search. So anything that is text is trivial to index and to search. If you wanted to search images, you could have an LLM describe all your images so that you could still search them, and that certainly works. Kind of like the Dries note, how he was searching for Nonna with glasses. Same concept. There's no limit in what you can search if you go that way.

[00:35:42] Are there questions? All right, I'm coming to this side of the room. I see one here and one up here, so let's go here first. (From the audience: What's the size of the runtime and memory, and how does it perform on older phones and stuff?) I don't know the current size. Let me think. I should know the current size. It's about two megabytes, I think. So it's not huge. The runtime on the hardware is great. WebAssembly is extremely efficient, and so that's not the issue so much. But the problem with a phone, you caught me there, is that downloading the index files, especially that first two megabytes, can be slower if you're on a slow EDGE connection or 3G or something. So the only slowdown really would be in that network and getting the files. Once they're on your phone, do the same search, do similar searches, they'll be as fast as you would expect, because they're local.

[00:36:36] Yeah, there was one over there. Okay. (From the audience: Oh, okay. Thank you. Sorry. Can you hook into the indexing? So add extra data when it's indexing your content?) Yes. Yes, it's all written in PHP code. So the indexer is nothing special, and you absolutely can do some modifications or changes if you wanted to. Yeah.

[00:37:05] (From the audience: Oh, okay. You mentioned there, when you were asked about the images and all the rest, and you said it's possible to index PDFs, like read the text within PDFs. So that's possible with this, is it?) Yes. But you would use a tool to export the PDFs to a text format, and then you would index that exported text. So the metadata would know where the PDF is, and it can link to it. But Scolta and Pagefind itself is not indexing PDFs directly. (From the audience: Makes sense. All right. And so how does Scolta then do filtering in Views and all the rest? Is it just the same as if you were using Solr, let's say, for, like, you know, taxonomies, if you're filtering that way?) So in the demos there, you saw that there were facets. It's not a view that's actually building that search result, but it does have faceting, and it does have the ability to sort or to filter by them. (From the audience: Okay.) Yes. If it's not clear, let me know.

[00:38:13] It's super, super easy to install, and I would love to, yeah, if you have problems, ask them in the issue queue or directly, whatever. Yeah. And I will say, indexing attachments was really big for us, because the bulk of the content on Share My Lesson is attachments. And so a teacher may create, and they'll fill in the field of a title and a description, but it really does not represent what the resource is. So, thank you, motion team, for doing that. Okay, I think I'm going to come over here, and then I'll come back here.

[00:38:44] (From the audience: Thank you. So when we compare with Solr, how fast is it compared to Solr? Well, if I have created one piece of content right now, so if I compare with Solr, how would it be?) It's an interesting question, because it's not a fair answer per se. Solr is fast. There's nothing slow about Solr. It's the network that is the issue. You know, every key you type is going across the network to Solr, gets a response back, and it does that. Also, if you had a lot of search traffic happening, it's all hitting a central server, right? So with Scolta, it's extremely fast as well, except it's local in your browser, so there's no round trip. And when you play with it, one of the feedbacks we get first is that, oh, this can't be working, because it's too good to be true. It really is that fast.

[00:39:35] (From the audience: But if your back-end content gets changed very often, like maybe every 30 minutes, in that case?) So that's interesting. The indexing speed is going to depend on your web server, because indexing is literally happening in PHP. We have some more complicated clients that have set up, like, a front end and a back end, so that the search indexing is on a dedicated back end, but as long as it writes to an index it can access, that works great. Pagefind: we originally were going to have Pagefind do all the indexing, but ran into some problems with that. One is you can't just install arbitrary Rust code on all web hosts, and this installs anywhere. That was, like, requirement number one. Two, Pagefind rebuilds the indexes any time there's a change, because it's designed for static websites. You deploy, it rebuilds the indexes. The PHP Scolta one that we've done actually does update the indexes in place. It's not perfectly fast. It does take time. I don't remember what the current time is for Share My Lesson, but it's something like 45 minutes, the last I checked, on how long it takes to index the whole site. It's optimized to only update where the changes are, though. And the slowdown I've seen is more in rendering, so that it can properly index, versus the actual index update itself.

[00:40:53] (From the audience: And this compressed file would be cached at the CDN as well, right?) You asked if it's compressed? (From the audience: The file, which...) Yes, of course. Yes, it is a compressed JSON format that's written on disk, and that's what's transferred, and yes, it gets cached. One cool consequence of that is that your entire search index can live on a CDN. So if your site goes down, you not only can cache every page, but search itself can continue functioning the entire time your site's down, and no one has to know your site's down. (From the audience: Yeah. Interesting. Thank you.)

[00:41:28] I just want to know, that difficult client, you weren't talking about me, were you? Never. All right. I think I saw some other hands. I'm going to come to you in one second. Let me just go over here, and then I'll come back. What? Okay.

[00:41:42] (From the audience: Let's see. There were a few questions, but I'll take one of them. The multilingual support, is it there without the AI on?) Let me rephrase that. You can search multilingual content without AI, absolutely. If you want to search in different languages, it can support that. We've got some demos on the Tag1 website that include a Git resource on how to use Git, and it's in many, many languages. And so yes, the search is multilingual in that sense. The sense where it's not going to work without AI is it's not going to understand your search in another language and translate it for you. (From the audience: Yeah, and it's not going to support stemming and lemmatization in that case, then?) Stemming is actually built in. Yes, there is some stemming. Pagefind has a stemmer built in, and that was ported into our indexer, so stemming it does do. (From the audience: Fuzzy search?) Sorry? (From the audience: Fuzzy search.) Fastly search? (From the audience: Fuzzy, fuzzy.) Oh, fuzzy search. No. That's where you're going to be relying on... (From the audience: Because that's just a query change in Solr. So that's maybe the big change, then.) Yeah. Okay.

[00:42:56] Thank you. I think we have time for maybe one, possibly two, depending on how short Jeremy is in responding. Any questions? Talking too much. Thanks for that. Yes.

[00:43:10] (From the audience: The examples we've seen that you've demonstrated are both quite clear, single-page, single-purpose content. You know, one was a lesson plan, so you've got, like, one lesson plan per page. The other one was recipes; you've got, like, one recipe per page. How does it do with content that's kind of, you know, more document style, so covering quite a lot of different things, maybe on the same page?) I'm not entirely sure what you're asking. I mean, it will be able to find text on... it doesn't matter how complicated a page is. It's certainly able to find that. And in the search results, it'll highlight in yellow what was matched. So it wouldn't matter how complicated the page is. Great.

[00:43:53] Any other questions? All right. Last question. Make it good. (From the audience: Thank you. Would it be possible to have different indexes? For example, if our website has a job search, a clinical trial search, or a product search, instead of putting everything into one big block, having them separated so that we have different search blocks and they match against different data sets?) Yes, without a doubt. Nothing prevents it. Out of the box, that's not currently a feature. But what's interesting, not only can you do what you describe, but it also supports federated search. So you can have them separate, or you can join them depending on where you are on the website and search all the indexes together, and everything you saw here still works the same, where it only grabs the pieces it needs and, yeah, uses those. Great. Thank you, everybody. Thank you, everyone. Thanks for coming.

[00:44:49] (From the audience: Thank you, Tag1.) I also just, one point of privilege, I have to just say, like, the Tag1 team, I have worked with plach here, who ditched us not too long ago to work on some other clients, since the very beginning. So many of these other folks, they have done yeoman's work and really believe in our mission of supporting educators. So thank you to all the Tag1 team that's here and not here right now. Thank...