What is a large language model?
A large language model (LLM) is a type of artificial intelligence trained on vast amounts of text to understand and generate human language. LLMs power AI chatbots, writing tools, and AI assistants, letting them answer questions, summarize documents, draft content, and hold natural conversations.
The word "large" refers to two things: the enormous amount of text the model learns from and the size of the model itself, which can contain billions of adjustable values called parameters. LLMs are a form of generative AI, because they produce new text rather than only analyzing existing text.
This guide explains how LLMs work, what tokens and context windows are, and what LLMs can and cannot do well.
How do large language models work?
An LLM works by predicting the next piece of text in a sequence, over and over, based on patterns it learned during training. That simple process, repeated at very large scale, lets it write fluent sentences, follow instructions, and answer questions.
Training an LLM
Pretraining. The model reads a massive collection of text, such as books, articles, websites, and code. For each passage, it practices predicting the next word and adjusts its parameters whenever it guesses wrong. Over billions of examples, it learns grammar, facts, writing styles, and how ideas relate.
Fine-tuning. The pretrained model is trained further on smaller, carefully chosen examples so it follows instructions, answers helpfully, and avoids harmful responses. This often includes feedback from human reviewers.
Inference. Once trained, the model is put to work. When you type a prompt, it generates a response one token at a time. Learn more about training and inference in our machine learning guide.
Transformers and parameters
Most LLMs are built on an architecture called the transformer. A transformer uses a technique called attention, which lets the model weigh how every word in a passage relates to every other word. That is how it understands that "it" in a sentence refers to a specific noun mentioned earlier.
Parameters are the internal values the model adjusts during training. They store what the model has learned. Models with more parameters can generally capture more knowledge and nuance, but they also need more memory and computing power to run.
What are tokens in an LLM?
Tokens are the small units of text that a large language model reads and writes. A token can be a whole word, part of a word, a punctuation mark, or a space. Before an LLM processes your prompt, it splits the text into tokens and converts each one into a number it can work with.
Common short words are often a single token, while longer or less common words are split into pieces. For example, a word like "tokenization" might be split into "token" and "ization."
How many words is a token?
For English text, a useful rule of thumb from OpenAI is:
Tokens | Approximate equivalent |
|---|---|
1 token | About 4 characters, or ¾ of a word |
100 tokens | About 75 words |
About 2,048 tokens | About 1,500 words |
Source: OpenAI Help Center. Exact counts vary by model, and other languages and computer code often use more tokens per word.
Why tokens matter
Length limits. A model's context window and maximum response length are measured in tokens, not words.
Cost. Many AI services charge for usage based on the number of tokens processed.
Speed. Generating more tokens takes more time, so long responses take longer to complete.
What is a context window?
A context window is the maximum amount of text, measured in tokens, that a large language model can consider at one time. It works like the model's short-term memory. Everything in the window, including your prompt, any documents you share, earlier messages in the conversation, and the model's own reply, has to fit within that limit.
When a conversation or document exceeds the context window, the model can no longer see the earliest material. That is why a long chat can "forget" details mentioned at the start, or why a tool may ask you to shorten a very large file.
How big are context windows?
Context windows have grown quickly. Early public chatbots could hold only a few thousand tokens, roughly a few pages of text. As of July 2026, the median context window across the AI models tracked by BenchLM is 256,000 tokens, about 24% of those models offer 1 million tokens or more, and the largest reaches 10 million tokens (BenchLM).
Using the token rule of thumb above, a 1-million-token window holds roughly 750,000 English words, or several full-length novels.
Does a bigger context window mean a better model?
Not necessarily. A larger window lets a model work with longer documents and conversations, but models do not always use all of that space equally well. Information buried in the middle of a very long input can be easier to miss. For important tasks, it often helps to put key instructions at the start of your prompt and to share only the material the model needs.
What can large language models do?
Large language models can handle almost any task that involves reading or writing language. Common uses include:
Answering questions and explaining complex topics in plain language
Summarizing long reports, articles, emails, and meeting transcripts
Writing and editing emails, posts, product descriptions, and reports
Translating text between languages
Writing and explaining code, from short scripts to debugging help
Analyzing documents, such as pulling key points from a contract or comparing two proposals
Powering AI assistants and agents that can plan steps and complete tasks on your behalf
Many newer LLMs are multimodal, meaning they can also work with images, audio, or files. For example, you can share a photo of a chart and ask the model to explain what it shows. Learn how these capabilities show up in everyday tools in our AI assistants guide and AI for business guide.
Limitations of large language models
LLMs are powerful, but they do not think or verify facts the way people do. Knowing their limits helps you use them safely.
Hallucinations. An LLM can produce information that sounds confident but is wrong or invented, such as a fake statistic or citation. This happens because the model predicts likely text rather than checking a source. Always verify important facts.
Knowledge cutoff. A model only knows what was in its training data, which ends at a certain date. Unless it is connected to web search or other current sources, it may not know about recent events.
Bias. LLMs learn from human-written text, so they can reflect stereotypes or one-sided views found in that data.
Context limits. Information that falls outside the context window is not available to the model during that conversation.
Inconsistency. The same prompt can produce different answers, and small wording changes can change the result.
Privacy. Prompts entered into cloud-based LLMs may be stored by the provider. Avoid sharing sensitive personal or business information, and see our AI privacy and security guide for safe practices.
LLMs vs. small language models
A small language model (SLM) is a compact language model designed to run efficiently on devices like laptops and phones. SLMs have far fewer parameters than the largest LLMs, so they need less memory and power, but they are also less capable at complex tasks.
Large language model (LLM) | Small language model (SLM) | |
|---|---|---|
Where it usually runs | Cloud data centers | On your device, often on the NPU |
Size | Very large, often tens to hundreds of billions of parameters | A few billion parameters |
Best for | Complex reasoning, research, long documents | Quick rewrites, summaries, and everyday tasks |
Privacy | Data is sent to a provider | Data can stay on your device |
Works offline | No | Yes |
Many AI PCs use both. A built-in small language model handles quick, private tasks on the device, while larger cloud LLMs take on more demanding requests. For example, Microsoft's Phi Silica is a small language model of about 3.3 billion parameters that runs on the NPU in Copilot+ PCs (Neowin). Learn more in our on-device AI vs. cloud AI guide and NPU guide.
Large language model frequently asked questions
What does LLM stand for?
LLM stands for large language model. It is a type of AI model trained on large amounts of text to understand and generate human language.
Is ChatGPT a large language model?
ChatGPT is an AI chatbot powered by large language models developed by OpenAI. The chatbot is the app you interact with, and the LLM is the underlying model that generates its responses. Other assistants, such as Microsoft Copilot, Google Gemini, and Anthropic's Claude, also run on LLMs.
What is the difference between an LLM and generative AI?
Generative AI is the broad category of AI that creates new content, including text, images, audio, and video. An LLM is one type of generative AI that focuses on language. Learn more in our generative AI guide.
What is a parameter in an LLM?
A parameter is an internal value the model adjusts during training to store what it has learned. A model with more parameters can usually capture more knowledge and nuance, but it needs more memory and computing power to run.
What happens when you exceed the context window?
The model can no longer see the oldest part of the conversation or document. Depending on the tool, earlier messages may be dropped, summarized, or the tool may ask you to shorten your input. Starting a new conversation or sharing only the most relevant material can help.
Why do LLMs make things up?
LLMs generate the most likely next words based on patterns, rather than looking facts up in a database. When the model lacks reliable information, it can still produce a fluent answer that is wrong. This is called a hallucination. Tools connected to search or trusted documents can reduce, but not eliminate, the problem.
Can an LLM run on my laptop?
Smaller language models can run on a laptop, especially an AI PC with an NPU and enough memory. The largest LLMs still run in the cloud because they need far more memory and computing power than a personal device provides.
Do LLMs learn from my conversations?
It depends on the provider and your settings. Some services may use conversations to improve their models unless you opt out, while business and on-device tools often do not. Check each tool's privacy settings before sharing sensitive information.
Related AI glossary guides
AI glossary: Definitions of key AI terms in one place
Generative AI: How AI creates text, images, and code
Machine learning: How models learn from data through training
AI assistants: The everyday tools powered by LLMs
On-device AI vs. cloud AI: Where language models run and why it matters
AI privacy and security: How to protect your data when using LLMs
