【2026】Ollama Complete Setup Guide: Running Local AI on a Mini PC

「I want to run AI on my own PC without relying on cloud API billing」

The tool that makes this wish come true is Ollama. Ollama is a tool for running LLMs (large language models) in a local environment. Download a model with one command and chat with AI offline.

This article fully explains how to run Ollama on a mini PC (GMKtec M8 / Minisforum UM690L), covering both Windows and WSL2.

Bottom Line: Ollama Runs AI on a Mini PC

Question	Answer
Can a mini PC run AI?	Yes. 7B models run at practical speed
Required specs?	16GB+ RAM, Ryzen 5+ CPU recommended
Do I need a GPU?	Not required. CPU works, but GPU is faster
Is setup hard?	Installing Ollama takes 5 minutes. 3 terminal commands
What can I use it for?	Programming help, writing, translation, summarization

What Is Ollama

Ollama is an open-source tool for easily running LLMs on a local PC. Its feature: manages models like Docker.

Key Features

Download & run models with one command: ollama run llama3.2
Supports many models: Llama 3, DeepSeek V4, Mistral, Qwen, Gemma, etc.
REST API: callable from other applications
OpenAI-compatible API: use existing OpenAI libraries as-is
Multi-platform: macOS / Linux / Windows
GGUF quantization: plenty of lightweight models for low-memory environments

Required Environment

Assumes a 16GB-memory mini PC like the GMKtec M8 or Minisforum UM690L.

Item	Recommended	Minimum
Memory	32GB+	16GB
CPU	Ryzen 7 / Core i7+	Ryzen 5 / Core i5
Storage	50GB+ (for models)	20GB+
GPU	NVIDIA RTX 4060+ (eGPU OK)	Integrated GPU OK
OS	Linux (Ubuntu 24.04)	Windows 11 + WSL2

With 16GB memory, 7B model Q4 quantization is the realistic line. With 32GB, 13B models become viable.

Setup Steps (By OS)

Method A: Windows + WSL2 (Recommended)

Running Ollama on a Linux environment via WSL2 is the most stable.

Step 1: Install WSL2

# Run in PowerShell (admin)
wsl --install -d Ubuntu-24.04

After install, launch Ubuntu and set up a username and password.

Step 2: Install Ollama

# Inside WSL2 (Ubuntu)
curl -fsSL https://ollama.com/install.sh | sh

Step 3: Download and run a model

# Lightweight model (3B) — start with a smoke test
ollama run llama3.2:3b

# Practical models (7B)
ollama run llama3.1:7b
ollama run deepseek-v4:7b
ollama run qwen2.5:7b

The model downloads on first launch (2–10GB depending on model). After download, it works offline.

Method B: Windows Native (No WSL2)

An official installer is available for Windows.

Download the Windows version from the Ollama official site
Run the installer
Open Command Prompt or PowerShell
Run ollama run llama3.2:3b to verify

Almost no difference from the WSL2 version. Choose this if you want to use the GPU directly from the host OS.

Method C: Linux Native (Ubuntu / Debian)

If you've installed Linux directly on the mini PC:

# Official install script
curl -fsSL https://ollama.com/install.sh | sh

# Check service status
systemctl status ollama

# Run a model
ollama run llama3.2:3b

Setup Flow Diagram

Recommended Models & Performance Reference

Based on measurements with the GMKtec M8 (Ryzen 5 6650H, 16GB).

Model	Size	Params	Speed	Best For
Llama 3.2	2.0GB	3B	🟢 Blazing	Smoke test, simple chat
DeepSeek V4 Lite	4.5GB	7B-Q4	🟢 Smooth	Programming, translation
Llama 3.1	4.7GB	8B-Q4	🟢 Practical	Writing, summarization
Qwen 2.5	4.5GB	7B-Q4	🟢 Practical	General Japanese tasks
Mistral	4.1GB	7B-Q4	🟢 Practical	Code generation, reasoning
DeepSeek V4	8.5GB	14B-Q4	🟡 Tight memory	Not recommended on 16GB

First pick: DeepSeek V4 Lite (7B). Great balance of programming, Japanese, and reasoning — smooth even on the GMKtec M8.

Common Commands

# Run a model (auto-downloads if missing)
ollama run <model-name>

# List downloaded models
ollama list

# Model details
ollama show <model-name>

# Delete a model
ollama rm <model-name>

# Start server mode (API)
ollama serve

# Search available models
ollama search <keyword>

Use It from the Browser with Open WebUI

Ollama is CLI-based, but installing Open WebUI gives you a ChatGPT-like browser UI.

# On WSL2 / Linux
pip install open-webui
open-webui serve

After launch, visit http://localhost:8080 to see the chat UI. Chat with local AI just like ChatGPT.

Ollama vs LM Studio

Item	Ollama	LM Studio
Install	CLI (one command)	GUI installer
Operation	Command line + API	Mouse (GUI)
GPU usage	Auto-detected	Manual setup
Model management	Auto-download (easy)	Manual download
API server	Built-in (OpenAI-compatible)	Built-in
Windows support	WSL2 recommended	Native
Linux support	✅ Full	⛔ Not supported
GPU offload	Auto-setup	Manual setup

Recommendation: LM Studio for beginners, Ollama once you're used to it. Once familiar, Ollama is overwhelmingly more efficient.

Troubleshooting

Q. Getting "ollama: command not found"

Restart your shell after install. Run exec $SHELL or reopen the terminal.

Q. Model won't load due to memory

Try a smaller quantization. Append :q4_K_M or :q3_K_M to the model name.

ollama run llama3.1:8b-q4_K_M  # 4-bit quantization (recommended)
ollama run llama3.1:8b-q3_K_M  # 3-bit quantization (lighter)

Q. GPU not being used

Check GPU recognition with nvidia-smi in WSL2. On native Linux, you may need ROCm setup.

Q. Model download is slow

Models are 2–10GB. On a gigabit connection, a few minutes.

Summary: Start Local AI with Ollama

With Ollama, a ¥50K-class mini PC becomes a practical local AI environment.

Step	Content
Step 1	Install WSL2 (Windows users)
Step 2	Install Ollama (curl one-liner)
Step 3	Pick a model and run (ollama run <model>)
Step 4	Use from the browser with Open WebUI

The appeal of local AI: no billing, no internet needed, and you get a private AI assistant. The mini PC + Ollama combo is the perfect entry point.

GMKtec M8 Mini PC Review — detailed review of the mini PC used in this guide
GMKtec M8 vs Minisforum UM690L Comparison — how to choose a local AI mini PC

This site contains affiliate links. We may earn a commission from some services we recommend.