BARTMOSS ST. CLAIR
USING OPEN SOURCE LLMs IN LANGUAGE
FOR GRAMMATICAL ERROR CORRECTION (GEC) BARTMOSS ST. CLAIR holds maths and physics degrees from Heidelberg University. He’s an AI researcher who previously worked at Harman International and Samsung, and has been a guest researcher at the Alexander von Humboldt Institute in Berlin. Currently, Bartmoss is Head of AI at LanguageTool. He’s also the creator of the open source community Secret Sauce AI.
How did you go from maths and physics at Heidelberg into data science and AI? I was a physicist at university, but I found academic life just didn’t suit me. Through a connection with a colleague, I got involved in a very interesting project with natural language processing, dealing with automating content governance systems for banks. Then I founded a company to build up solutions for five or six different languages, and that’s when I discovered my passion. Now you’re at the cutting edge of using LLMs in the business world. Can you give us an overview of what you’re doing at LanguageTool? So at LanguageTool, the one use case for us is grammatical error correction. Someone writes a sentence or some text, and they want their grammar checked, then the tool replaces it with the correct grammar. Of course, that’s a very basic use case. LanguageTool has
existed for about 20 years, but back then we didn’t use AI or machine learning. As head of AI, one of the things I wanted to do was create a general grammatical error correction system (GEC) to catch all kinds of errors for all languages possible, and correct them. So initially, we needed a model that would correct the text someone has written. We had to decide whether to take an existing LLM like GPT-3 or GPT-4 and just use prompting, or to create our own model. And we found that creating our own model was cheaper, faster, and more accurate. There was a lot to consider when creating a model that will run for millions of users: we had to make sure there was a good trade-off between the performance, speed, and price. Decoder models have become extremely popular, and we’ve seen a lot of scaling behind them. But sometimes the question is: how big do you need the model to be for your purpose? And do you have to benchmark and test that to see how well that works?
32 | THE DATA SCIENTIST
