How to Run a Local AI Model with Ollama on Windows 11

Install Ollama’s native Windows application, open a new PowerShell window, download a small local model with ollama pull gemma3:1b, and start it with ollama run gemma3:1b. The official Windows documentation confirms that Ollama runs in the background, makes its command available in Windows terminals, and serves its API on localhost.

Choose the first model conservatively, because its download size is not the complete runtime memory requirement. After the model answers a prompt, keep it loaded and run ollama ps in a second PowerShell window; Ollama’s processor guidance explains how the result identifies GPU, CPU, or split placement.

1. Record the PC’s memory and GPU details

Before installing anything, open Settings > System > About and record the installed RAM. Then open Task Manager > Performance, select the GPU, and note its exact name and dedicated GPU memory. These figures will guide model selection and help distinguish an unsupported GPU from a model that is simply too large for available VRAM.

Windows 11 falls within Ollama’s supported Windows range because the current system requirements specify Windows 10 22H2 or newer. That page also lists Windows-specific NVIDIA and AMD driver requirements, while the separate hardware page tracks supported GPU families and backends; consult both before buying a GPU or changing drivers.

1. Install pending Windows updates and restart the PC if required.
2. Update the graphics driver through NVIDIA or AMD.
3. Check free space on the drive that will hold Ollama and its models.
4. Record whether the PC has only integrated graphics or also has a discrete GPU.

The application installation requires at least 4GB of disk space according to Ollama, and a collection of downloaded models can occupy tens or hundreds of gigabytes. If the system drive is already crowded, choose a separate model directory before pulling several tags.

A supported dedicated GPU is useful but not required for the basic workflow. The independent Windows Central setup guide documents CPU-and-RAM operation when no dedicated GPU is available and describes small 1B, 3B, and 4B models running on comparatively modest hardware. Treat that as a starting point, not a promise of a particular speed.

2. Choose a model that leaves memory headroom

Parameter count is useful for comparing variants within a family, but it does not determine memory use by itself. Quantization, context length, runtime buffers, and processor placement also matter. Start with the download size shown for the exact tag, then leave capacity for Windows, other applications, and Ollama’s runtime allocations.

The official Gemma 3 catalog lists gemma3:1b at 815MB, gemma3:4b at 3.3GB, gemma3:12b at 8.1GB, and gemma3:27b at 17GB. These are downloaded-model sizes displayed by the catalog, not minimum RAM or VRAM specifications.

Use this editorial worksheet for the first attempt:

• 8GB of system RAM without a discrete GPU: begin with a 1B-class quantized model and close memory-heavy applications.
• 16GB of system RAM: try a 1B model first, then a 4B model if the smaller tag runs reliably.
• 32GB of system RAM: validate a 4B model before experimenting with a quantized 12B tag.
• A discrete GPU: compare the tag’s download size with currently free VRAM, but do not assume that a tag fitting on disk will fit entirely on the GPU.

These ranges are cautious setup estimates, not Ollama hardware guarantees. If a model does not fit completely in VRAM, it may use system memory and the CPU, which can change responsiveness substantially. Measure placement with ollama ps instead of inferring it from parameter count.

Context also consumes memory. Ollama’s context documentation states that the default context window is 4096 tokens and explains how to override it. Keep that default during initial validation so a large custom context does not obscure whether the basic configuration works.

3. Install Ollama’s native Windows application

Download OllamaSetup.exe from Ollama’s official Windows download path and run it under your normal account. The installer instructions say the standard application installs per user without administrator rights, places its binaries in the user profile by default, and adds their directory to the user PATH.

1. Run OllamaSetup.exe and complete the installation.
2. Start Ollama from the Start menu if it is not already running.
3. Open a new PowerShell window so it receives the updated PATH.
4. Run ollama --help.

If the help text appears, PowerShell has located the executable. If the command is not recognized, close all terminal windows, confirm that Ollama is running, and open PowerShell again. The documented binary location is %LOCALAPPDATA%\Programs\Ollama, which can also be checked directly.

To place the application on another drive, Ollama documents the installer form OllamaSetup.exe /DIR="D:\Ollama". Model storage is configured separately: create a user environment variable named OLLAMA_MODELS with a directory such as D:\OllamaModels, quit Ollama from the taskbar, and relaunch it. Changing the binary directory does not automatically relocate downloaded models.

4. Pull and run the first model

Use an explicit tag so you know which variant is being downloaded. The official CLI reference documents commands for pulling, running, listing, stopping, and removing models.

1. Download the example with ollama pull gemma3:1b.
2. Confirm its presence with ollama ls.
3. Start it with ollama run gemma3:1b.
4. Enter a short prompt such as Explain the difference between RAM and VRAM in three bullet points.
5. Type /bye when you want to leave the interactive session.

The first pull needs an internet connection to retrieve the model files. Once the local tag is downloaded, prompts sent to that local model can be processed on the PC. Do not confuse a locally installed tag with a cloud model: Ollama’s current product also supports cloud-hosted models, which follow a different data path.

If a partial or corrupted download is suspected, run the pull command again before manually modifying the model directory. To unload a running model from another terminal, use ollama stop gemma3:1b. To remove the stored tag later, use ollama rm gemma3:1b.

5. Verify the background service and API

Open http://localhost:11434 in a browser as a quick service check. For a structured test, run Invoke-RestMethod http://localhost:11434/api/tags in PowerShell. The API introduction identifies http://localhost:11434/api as the default local base URL, and its tags endpoint returns information about installed models.

If the request fails, confirm that Ollama is running and inspect %LOCALAPPDATA%\Ollama\server.log. The Windows troubleshooting documentation identifies that file as the current server log and lists nearby logs for the application and updater.

You can inspect the listener directly with Get-NetTCPConnection -LocalPort 11434 -State Listen | Select-Object LocalAddress,LocalPort,OwningProcess. The Microsoft cmdlet reference confirms that Get-NetTCPConnection reports local addresses, ports, states, and related connection properties.

6. Confirm GPU use instead of assuming it

Keep the model active, open a second PowerShell window, and run ollama ps. According to Ollama’s Processor-column definitions, 100% GPU means the model is loaded entirely into GPU memory, 100% CPU means it is in system memory, and a CPU/GPU percentage split means both are involved.

A CPU result does not by itself indicate a failed installation. Possible explanations include an unsupported GPU, an incompatible driver, insufficient free VRAM, or selection of a different backend. Run the 1B example before diagnosing a larger model: if the smaller tag reaches the GPU but the larger one does not, capacity is a more likely issue than basic device detection.

Compare the exact device with Ollama’s current hardware-support tables. The page distinguishes NVIDIA CUDA support, AMD acceleration through ROCm, and additional Windows support through Vulkan; Vulkan behavior and supported devices can change, so the live table is more reliable than a static compatibility list.

1. Update the GPU driver and restart Windows.
2. Relaunch Ollama and run ollama run gemma3:1b.
3. Repeat ollama ps while the model remains loaded.
4. Inspect server.log for GPU discovery or backend errors.
5. If only a larger model falls back or splits, return to a smaller tag and the default context.

7. Restrict Ollama to the local PC

Ollama’s network configuration says the server binds to 127.0.0.1 on port 11434 by default. For a single-PC installation, leave that default unchanged: do not set OLLAMA_HOST to a LAN-facing address, configure router port forwarding, or add an inbound firewall rule that exposes the service.

Run the Get-NetTCPConnection check again and inspect LocalAddress. A loopback value such as 127.0.0.1 confines the listener to the machine; a LAN address or wildcard listener requires investigation. Check both user and system environment variables for OLLAMA_HOST, remove an unintended override, quit Ollama, and restart it.

Review Windows Security > Firewall & network protection > Allow an app through firewall if Ollama previously triggered a network-access prompt. A loopback-only setup does not need unsolicited inbound access from public or private networks. Firewall settings are an additional safeguard, but the listener address remains the clearest check of what interface the server is using.

The local API itself has no authentication, as stated in Ollama’s authentication reference. This is expected for the loopback service, but it is also why exposing the port to a LAN or the internet should be a deliberate security decision rather than an installation shortcut.

For stricter local-only operation, create a user environment variable named OLLAMA_NO_CLOUD with the value 1, then restart Ollama. The official local-only instructions say this disables cloud models and web search and records Ollama cloud disabled: true in the logs. Download any required local models before disconnecting the PC from the internet.

8. Update and expand the setup methodically

Ollama on Windows automatically downloads application updates, which can be applied with Restart to update from its taskbar item; Ollama’s update instructions also permit manual installation of the latest version. Update before troubleshooting a newly released model or recently added GPU, because model and hardware support changes over time.

Use ollama ls to review stored tags and ollama rm MODEL_NAME to remove one cleanly. If you configured OLLAMA_MODELS, remember that the Windows uninstall documentation says the installer will not remove models from that custom location.

Finish with the same validation sequence after every important change: ollama --help, ollama ls, ollama run MODEL_NAME, ollama ps, and the loopback-listener check. Once a small model passes all five stages, move up one tag at a time and observe processor placement. That process establishes a practical limit for the actual PC without relying on unverified benchmark speeds or treating parameter count as a memory specification.

Also read:

• Alibaba’s Qwen3.8-Max Preview: What the 2.4T Model Means for AI Buyers
• OfficeCLI: Let Your AI Agent Run Word, Excel, and PowerPoint from a Single Command Line