Start

Home / Big ideas

Idea6-min read+5 pts

What Is an LLM? The Engine Inside Your Chat Tool

With Josh Schultz ·

Answer. An LLM, short for large language model, is the engine inside chat tools such as Claude, ChatGPT, and Copilot. It is a program trained on a huge amount of text to predict the next words, and the chat tool is the app built around that engine.

The picture

Diagram: a big box labeled "The app you open" contains a smaller box labeled "The model (the engine)". Around the model sit smaller boxes labeled "Your chat", "Settings", "Files", and "Safety rules". The model box is circled.

The large box is the app you open on your screen, where you type, attach files, and read replies. Inside it, at the center, is the model, which is the part that actually writes the text. The smaller boxes around it, such as your chat history, the settings, your files, and the safety rules, belong to the app and not to the engine. The circle on the model shows what this lesson is about.

Think of it like this

Think of a delivery truck and the engine under its hood. You drive the truck, with its seat, its steering wheel, and its radio, but the engine is what moves it. Two trucks from different makers can feel quite different to drive, even if the engines are similar, and the same maker can swap in a newer engine without changing the cab.

In AI, the app is the truck and the LLM is the engine. When people say "I use ChatGPT" or "I use Claude," they usually mean the app. When people say "the model," they mean the engine, and the company can change which engine sits behind the app over time. That is why a tool can feel different from one month to the next even though the screen looks the same.

Why does it matter at work?

Knowing the difference helps you ask better questions and avoid a few wrong assumptions. If a result is poor, the cause could be the engine, but it could just as easily be the app, such as the settings, the files you attached, or how much of the conversation it could still hold (see what a context window is).

It also helps when someone at your company says "we use an LLM." That tells you very little by itself. At Mesquite Ridge Metal Works, Jen might use a chat app, while IT builds a small tool that calls the same kind of engine in the background to read invoices. The engine is similar in both, but the safety rules, the data handling, and the price can be very different, so the useful questions are about the app: which tool, which plan, and what happens to our data. Check your plan's data settings and ask IT about the rest.

How does it work?

There are three parts to know. The model reads a lot of text, it learns to predict the next words, and the app wraps it so you can use it.

It reads a huge amount of text. The "large" in the name refers to two things, the amount of text it studies and the number of adjustable settings inside it, which are called parameters. You can think of parameters as the many small dials that get tuned during training so the model's guesses improve.

It learns to predict the next words. Anthropic's glossary says these models are pretrained "to predict the next word, given the previous context of text in the document." That sounds modest, but predicting the next word well across millions of documents means picking up grammar, facts, tone, and many common patterns. For more on this, see how AI writes.

Diagram: the words "Please find the attached purchase" lead to a model box, which points to likely next words: order, orders, invoice, agreement, contract. "order" is circled.

In the picture, the words so far go into the model, and out come a few likely next words, with "order" the most natural one. A whole reply is this step repeated, with each new word added to the words so far.

The app wraps it so it behaves. A model fresh from reading is not very good at following instructions. The glossary says Claude "is not a bare language model; it has already been fine-tuned to be a helpful assistant," and mentions the human-feedback step explained in reinforcement learning. The app then adds the chat window, your files, and its own safety rules. Text is measured in small pieces called tokens, which is how limits and prices are counted.

Where do people get it wrong?

Most mistakes come from treating the engine as a database or as a person. It is neither, and these are the three I see most.

  1. Thinking it looks things up. An LLM writes the likely next words from what it learned, which is why it can make things up with a confident tone.
  2. Mixing up the app and the model. A feature like file upload or web search belongs to the app, so a different app on the same engine may not have it.
  3. Assuming it knows your company. The model learned from public text and not from your ERP, unless you give it your documents in the chat. Does AI learn from my data? covers what the vendors do with what you type.

Try this in 2 minutes

This test shows the engine's one real skill, which is guessing the next word. You can use any chat tool.

  1. Open a new chat and paste this:
Finish this sentence with the 5 most likely next words, as a plain list, one word each, no numbers: "Please find the attached purchase". Then in two plain sentences explain how that guess is the same job you do for every reply.
  1. What you should see is a list of five likely words, and then a short explanation.
  2. Notice how many of the words are ones you would have guessed yourself.

A chat reply listing five words, order, orders, invoice, agreement, contract, then two sentences explaining that every reply is the same guess repeated.

When I tried this, the five words were "order," "orders," "invoice," "agreement," and "contract." Then the reply said: "A full answer is just that prediction repeated one token at a time, with each new token added to the context before the next guess."

That one sentence is the whole engine. Everything a chat tool writes, from a summary to a supplier email, is that guess made over and over, and the app around it decides what you see.

Frequently asked questions

What does LLM stand for?

It stands for large language model. It is an AI model trained on a very large amount of text so that it can read and write language.

Is ChatGPT an LLM?

ChatGPT is an app, and it runs on LLMs made by OpenAI. The same is true of Claude from Anthropic and Copilot from Microsoft. Vendors can change which model sits behind an app.

Is an LLM the same as AI?

No. AI is the wide field, and an LLM is one kind of AI, the kind that works with language. Types of AI, explained shows where it fits.

Does an LLM understand what it writes?

It is very good at producing text that fits the situation, but it works by predicting patterns and can be wrong with full confidence. Check anything important, as with a new hire's first draft.

Use it today

From my training: the site teaches it with the truck and the engine, so people stop asking "which AI is best?" and start asking "which app, on which plan?"

Ideas are worth 5 points. Guides are worth 10 and up.

Get your first win this week.

One minute. Three guides picked for your job.

Start my first win