If you want to chat with an AI model without sending each prompt to a hosted chatbot, Ollama lets you download and run language models on your own computer. It includes a command-line tool to download and manage models, and a local API for other programs.
To get started, install Ollama. Make sure your computer has enough memory and storage for your chosen model, and connect to the internet to download it. Once installed, run the model locally. This tutorial covers installation, a first chat, basic model management, and a simple API request.
Install Ollama
Ollama has installers for macOS and Windows, plus installation instructions for Linux. Choose the instructions for your operating system from the Ollama project.
On Linux, the documented install command is:
curl -fsSL https://ollama.com/install.sh | sh
After installation, open a terminal and check that the command is available:
ollama --version
Ollama can use supported graphics hardware to speed up model generation. Its hardware support documentation describes supported NVIDIA and AMD GPUs, Apple Metal, and Vulkan. A GPU is not required, but model size and hardware affect how quickly responses appear.
Download and chat with a model
Run a model by name:
ollama run llama3.2
If the model is not already on your computer, Ollama downloads it before starting the chat. At the prompt, enter a question. Type /bye to leave the chat.
The available model names and tags change over time. Check the Ollama model catalog before choosing one, and select a model that fits your computer’s memory and storage. Larger models generally require more resources. If a model responds too slowly or fails to load, try a smaller one.
Manage downloaded models
Use ollama list to see which models are stored locally. To download a model without starting a chat, run ollama pull followed by its name:
ollama pull llama3.2
ollama list
Use ollama show llama3.2 to inspect a model, and ollama ps to see models currently running. To stop a running model, use ollama stop with its name. To remove a downloaded model and free disk space, run:
ollama rm llama3.2
The Ollama CLI reference explains these commands and others, including ollama create.
Customize a model with a Modelfile
A Modelfile is a text file with instructions for building a customized model from an existing model. For example, create a file named Modelfile with a base model and a system prompt:
FROM llama3.2
SYSTEM "You are a concise writing assistant."
Build and run the customized model:
ollama create writing-helper -f Modelfile
ollama run writing-helper
The SYSTEM line sets default instructions for the model. Ollama also supports Modelfiles for other configurations, including importing supported model files and adapters.
Send a request to Ollama’s local API
Ollama runs a local service that lets programs send prompts to a model. For example, this curl request sends a chat message to the /api/chat endpoint:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "Explain how solar panels work." }
],
"stream": false
}'
Ollama returns the response as JSON. Ollama’s API also includes endpoints for listing local models and other tasks; the Ollama repository links to API details.
Keep the API accessible only from your computer unless you have secured it for broader access. Check the address Ollama listens on, and restrict it to 127.0.0.1 if only local programs need to connect. If you expose the API to other devices without authentication, others can send requests that use your computer’s resources.