How to Install and Use Ollama on Ubuntu 26.04 and 24.04

Running a large language model does not require sending every prompt to a hosted API. Ollama packages models so you can run them on your Ubuntu machine, use them offline after downloading, and access them from terminal commands or local applications.

When you select a local model, inference and prompt processing stay on your machine. Ollama also supports optional cloud models, which are separate from the local workflow covered here. This guide explains how to install Ollama on Ubuntu 26.04, run your first local model, manage downloads, check GPU acceleration, and use the built-in API. The same steps work on Ubuntu 24.04.

Quick Reference #

Step 1: Install Ollama #

The official installation script detects the system architecture, installs the Ollama binary, and configures a systemd service. Install curl and zstd first so Ubuntu can download and extract the current package:

sudo apt update
sudo apt install curl zstd

You can then run the official installer:

curl -fsSL https://ollama.com/install.sh | sh

The script adds the ollama command to your path, creates a dedicated ollama system user, adds your account to the ollama group, and enables the background service when systemd is available. On a machine without a supported GPU, the last lines of the installer output look like this:

>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

The warning is expected on CPU-only machines and most virtual machines. Ollama still works and runs models on the processor.

If you prefer to inspect remote scripts before running them, download this one first:

curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
less ollama-install.sh
sh ollama-install.sh

Confirm that the command is available:

ollama --version

ollama version is 0.40.11

The command prints the installed Ollama version. The number changes as new releases become available.

Running ollama with no arguments opens an interactive menu for chat and coding-tool integrations (since version 0.32). To list the available subcommands, use ollama --help.

Step 2: Verify the Ollama Service #

Ollama listens on 127.0.0.1:11434 by default. Check that its systemd service is running:

systemctl status ollama --no-pager

The relevant part of the output should look similar to this:

● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
     Active: active (running)

The enabled state means the service starts during boot. active (running) confirms that the local server is ready to accept commands and API requests.

Step 3: Run Your First Model #

The ollama run command downloads a model the first time you use it, then opens an interactive chat. Llama 3.2 is a practical first model because its default 3B variant is about 2 GB and answers quickly even without a GPU:

ollama run llama3.2

After the download finishes, enter a prompt at the >>> marker:

>>> Explain what a reverse proxy does in one sentence.
A reverse proxy receives client requests and forwards them to one or more
backend servers, often handling TLS, caching, or load balancing.

>>> /bye

Type /bye or press Ctrl+D to leave the chat. Enter /? inside the session to see the available interactive commands.

Step 4: Manage Models #

The Ollama model library lists available models, variants, file sizes, and context limits. Download another model without opening a chat by using ollama pull. The part after the colon selects a variant, here the 2B version of Qwen 3.5:

ollama pull qwen3.5:2b

List the models stored on the machine:

ollama ls

NAME               ID              SIZE      MODIFIED
qwen3.5:2b         0689d44085e0    2.7 GB    2 minutes ago
llama3.2:latest    a80c4f17acd5    2.0 GB    7 minutes ago

Qwen 3.5 is a thinking model. It writes out its reasoning before the answer, which can take a minute or more on a CPU. Pass --think=false to get the answer directly:

ollama run --think=false qwen3.5:2b "Explain what a reverse proxy does in one sentence."

Use ollama show to inspect a model and ollama ps to see which models are loaded in memory:

ollama show llama3.2
ollama ps

Ollama keeps a recently used model loaded for five minutes by default. Unload it immediately with ollama stop:

ollama stop llama3.2

Remove an unused download to recover disk space:

ollama rm qwen3.5:2b

Models installed by the system service are stored under /usr/share/ollama/.ollama/models. When you pick a model, match its size to the memory you have. A small 1B to 3B model runs comfortably on a CPU, while larger models want more RAM or GPU memory and feel much faster once a supported GPU handles them.

Step 5: Use the Ollama API #

The local API is available at http://localhost:11434/api. Send a prompt to the generate endpoint with curl, setting stream to false so the server returns one completed JSON response:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Name three Linux distributions.",
  "stream": false
}'

The JSON response contains the generated text in the response field, along with timing and token-count information. Because the request stays on your machine, local models need no API key or authentication.

The same local service also exposes chat, embeddings, model-management, and OpenAI-compatible endpoints, so most applications and browser front ends can talk to Ollama without any extra setup.

Step 6: Check GPU Acceleration #

If no supported GPU is present, Ollama falls back to the CPU and still works, just more slowly. When the Linux installer detects supported NVIDIA or AMD hardware, it pulls in the matching GPU libraries for you. You still need a working GPU driver on the host, since Ollama relies on it to reach the card.

For NVIDIA hardware, verify the installed driver:

nvidia-smi

For live utilization, memory use, and process details while a model runs, see our nvidia-smi command guide .

Ollama does not require the full CUDA Toolkit for inference. It supports NVIDIA GPUs with compute capability 5.0 or newer and driver version 550 or newer. Cards with compute capability 5.0 through 6.2 need driver 570 or newer. See the official hardware documentation for the full list of supported cards.

For supported AMD hardware on Linux, Ollama uses ROCm and requires a compatible ROCm 7 driver. The installer downloads the additional ROCm package when it detects an AMD GPU.

On x86_64 systems, Ollama also ships a Vulkan backend. It covers Intel GPUs and AMD cards that ROCm does not support. Vulkan uses the Mesa drivers from the mesa-vulkan-drivers package, which Ubuntu Desktop installs by default. On a server, install it with sudo apt install mesa-vulkan-drivers.

Run a model, then check which processor it uses:

ollama run llama3.2 "Respond with the word ready."
ollama ps

On a machine without a supported GPU, the output looks like this:

NAME               ID              SIZE      PROCESSOR    CONTEXT    RUNNER      UNTIL
llama3.2:latest    a80c4f17acd5    2.9 GB    100% CPU     4096       llamacpp    4 minutes from now

The PROCESSOR column shows where the model runs. 100% GPU confirms GPU inference, 100% CPU means no GPU is used, and a split such as 48%/52% CPU/GPU means the model did not fit in video memory. SIZE is the memory the loaded model uses. It is larger than the 2.0 GB download because it also holds the 4096-token context window shown under CONTEXT.

If a machine with a GPU shows 100% CPU, inspect the service logs and verify that the GPU driver can see the card:

journalctl -e -u ollama

Step 7: Expose Ollama on the Network #

Ollama binds to the loopback address by default, which prevents other devices from reaching it. Before making a GPU server available to a trusted workstation, configure a firewall to allow only trusted clients.

If you use UFW, first check its status and default policies:

sudo ufw status verbose

Do not proceed unless the output shows Status: active and deny (incoming). UFW is disabled by default, and adding an allow rule does not enable it. If it is inactive or uses a different policy, follow the Ubuntu UFW guide to configure it first. Allow your configured SSH port and any other required services before enabling the firewall.

With UFW active and incoming connections denied by default, allow the trusted subnet to reach Ollama:

sudo ufw allow from 192.168.1.0/24 to any port 11434 proto tcp

Replace the subnet with the address or network that should have access. Review the existing UFW rules and remove any broader rules that allow untrusted clients to reach port 11434.

Once the firewall is configured, create a systemd override:

sudo systemctl edit ollama

Add the following setting:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Reload systemd and restart Ollama:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Update Ollama #

When a new release lands, you do not need a special upgrade command. Run the same installation script again and it updates Ollama in place:

curl -fsSL https://ollama.com/install.sh | sh

The installer replaces the program files and restarts the service without deleting downloaded models.

Uninstall Ollama #

The installer does not include an uninstall command. Stop and disable the service, remove any systemd overrides, then delete its unit file and reload systemd:

sudo systemctl stop ollama
sudo systemctl disable ollama
sudo systemctl revert ollama
sudo rm /etc/systemd/system/ollama.service
sudo systemctl daemon-reload

The revert command removes custom settings such as the OLLAMA_HOST override from Step 7, so they do not carry over to a later installation.

Remove the binary and its libraries:

sudo rm /usr/local/bin/ollama
sudo rm -r /usr/local/lib/ollama

The last step deletes the ollama user, its group, and its home directory /usr/share/ollama. That directory holds every downloaded model, so copy out anything you want to keep before running these commands:

sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollama

The installer added your account to the ollama group, so userdel keeps the group and prints userdel: group ollama not removed because it has other members. The groupdel command removes it.

Troubleshooting #

apt cannot find the ollama package
Ollama is not packaged in the Ubuntu repositories, so sudo apt install ollama fails with E: Unable to locate package ollama. Install it with the official script from Step 1. The ollama snap in the Snap Store is a community build, not an official Ollama release.

Ollama cannot connect to the local service
Start the service with sudo systemctl start ollama, then inspect its status with systemctl status ollama --no-pager.

The service starts but model commands fail
Read the recent logs with journalctl -e -u ollama. For live output while reproducing the problem, use journalctl -u ollama --no-pager --follow --pager-end.

A model runs out of memory
When the system runs out of RAM, the kernel kills the model runner and ollama run stops with Error: Post "http://127.0.0.1:11434/api/generate": EOF. The service log shows Failed with result 'oom-kill'. Unload other loaded models with ollama stop MODEL, then choose a smaller model or a lower-precision variant from the model library.

Ollama uses the CPU instead of an NVIDIA GPU
Run nvidia-smi and update the NVIDIA driver if it cannot see the card. After a Linux suspend and resume cycle, reloading nvidia_uvm or rebooting may restore GPU detection.

Ollama does not detect an AMD GPU
Confirm that the host uses a supported GPU and ROCm 7 driver. Check /dev/kfd permissions and the Ollama logs for ROCm or GPU-discovery errors.

Conclusion #

Ollama provides a direct way to run local language models on Ubuntu through the terminal or an HTTP API. Start with a small model, confirm its memory and processor use with ollama ps, and keep port 11434 restricted to trusted systems.

About the authors

Dejan Panovski

Dejan Panovski is the founder of Linuxize, an RHCSA-certified Linux system administrator and DevOps engineer based in Skopje, Macedonia. Author of 1000+ Linux tutorials with 20+ years of experience turning complex Linux tasks into clear, reliable guides.

View author page