When you ask a chatbot a question, a large language model (LLM) breaks your text into pieces, uses patterns learned during training to predict what text comes next, and generates a response one piece at a time. Those pieces are called tokens. A token can be a whole word, part of a word, or punctuation.
Most LLMs use a design called a transformer, which helps the model relate different parts of the text it receives. Training on large amounts of text helps the model produce useful answers. During training, the model adjusts its internal numerical settings, called parameters, to improve its predictions. The basic mechanism is prediction; it does not guarantee that an answer is true.
How text becomes model input
Before processing a prompt, a tokenizer splits it into tokens and encodes each token as a number. The model turns those numbers into vectors, or lists of values that represent information the model can work with. It also receives information about token order, so “dog bites person” and “person bites dog” do not look identical.
A transformer uses attention to determine how strongly tokens relate to one another in context. For example, in “Maya put the book on the table because it was uneven,” attention helps the model relate “it” to the surrounding words. Multiple attention heads let the transformer examine different relationships at the same time. The Cornell notes on attention and transformers describe how these mechanisms help process relationships among tokens.
How training teaches predictions
During pretraining, the model sees text and learns to predict a missing next token from the tokens that came before it. If it reads “The capital of France is,” it assigns likelihoods to possible next tokens. Training compares those predictions with the text that actually follows and adjusts the model’s parameters to improve future predictions. Repeating this across a large collection of text teaches the model patterns in language, facts, and styles of writing. Databricks’ overview of large language models describes this training and transformer-based approach.
Pretraining gives a model broad language abilities. Developers can then fine-tune it, which means training it further on a more focused set of examples. Instruction-focused examples, for instance, teach a model to respond to requests in a useful format. Fine-tuning changes how a model responds, but it does not make every answer accurate.
How an LLM generates a response
When you submit a prompt, the model processes its tokens and predicts a likely next token. It adds that token to the text, then predicts another using the prompt and its own new output as context. This repeats until the model reaches an ending condition, such as producing an end-of-response token.
The model does not always choose the same next token. Generation settings influence how it chooses among likely options, so the same prompt can produce different wording. A chat interface can also add instructions or relevant information to the prompt before the model processes it. Those additions affect the response because they become part of the text the model considers.
Why fluent answers can still be wrong
An LLM generates text based on learned patterns and the context it receives. Unless it is connected to a tool that retrieves and checks information, the prediction process itself does not verify whether a claim is true. A model can therefore produce a confident-sounding answer that contains an error or invented detail.
OpenAI’s discussion of why language models hallucinate argues that evaluation practices that reward guesses over expressions of uncertainty contribute to the problem. In practice, verify important claims against dependable sources, especially when the answer concerns health, law, finances, or current events. Treat a fluent response as a starting point for checking, not as proof.