From Windows 11 to Ubuntu for my Local AI Workstation

Disclaimer: I create this content entirely on my own time, and the views expressed here are mine alone (not my employer’s). Because I love leveraging new tech, I use AI tools like Gemini, NotebookLM, Claude, Perplexity and others as a “digital team” to help research and polish these articles so I can share the best possible insights with you!

If you need help with the use of AI, automating your workflows, increasing your productivity and making you your life easier, send me a note

It is a strange feeling to look at a piece of hardware—all shiny, new, and theoretically capable—and feel the quiet frustration of it underperforming. You know the specs. You’ve seen the benchmarks. Yet, when you prompt an LLM, the cursor blinks for a beat too long, and your CPU usage jumps while your GPU sits there, cold and inert. It’s like owning a high-performance sports car but being told you can only drive it in a suburban cul-de-sac.

After spending weeks debugging my Ryzen-powered mini PC on Windows 11, I realized that the “AI revolution” isn’t just about hardware availability; it’s about the narrow, often precarious bridge between your silicon and your software.

The Hardware Promise vs. The Windows Reality

My setup is built around the MINISFORUM AI X1 Pro-370, featuring the AMD Ryzen AI 9 HX370. On paper, it’s a beast: 12 Zen 5 cores, 80 TOPS of AI performance via an integrated NPU, and the RDNA 3.5-based Radeon 890M graphics. With 64GB of DDR5 RAM available, I fully expected my local RAG pipelines and coding assistants to fly.

Instead, I hit a wall of silent bottlenecks. My system wasn’t crashing; it was just… lazy.

Component	Specification
Processor	AMD Ryzen AI 9 HX370 (12 Cores/24 Threads)
Graphics	AMD Radeon 890M (RDNA 3.5)
Memory	64 GB DDR5 RAM
Storage	1 TB NVMe SSD
Connectivity	WiFi 7, Bluetooth 5.4, Dual USB4, Dual RJ45
OS	Ubuntu 24.04 LTS (Post-migration)

When I looked at my logs, the reality was stark. Even with what I believed was a perfect configuration, the inference engine was defaulting to the CPU. My GPU, designed for these exact matrix multiplications, was doing almost nothing.

The workloads I targeted were specific: Ollama, llama.cpp, Open WebUI, GGUF-quantized models, vector databases for RAG, local coding assistants, and automation agents built with n8n. All of these should have benefited from GPU acceleration. Almost none of them did on Windows.

The NPU Reality Check

Before going further, let me address the elephant in the room: that 80 TOPS NPU.

The NPU is real, and it’s genuinely impressive silicon. But here’s the catch—most open-source LLM runtimes (llama.cpp, Ollama, vLLM, etc.) do not currently use it. The XDNA NPU requires specific SDK integration that isn’t broadly available in the local AI ecosystem. Today, NPU acceleration is mostly limited to Windows Studio Effects, background blur, and a handful of Microsoft-specific features.

For LLM inference on AMD APUs right now, the practical path is GPU acceleration via Vulkan, not the NPU. There are experimental projects (RyzenAI-SW, Lemonade SDK) that aim to change this, but they’re not production-ready. This is important context because marketing materials often highlight NPU TOPS, while actual LLM workflows lean on the integrated GPU.

The Disconnect of the Software Ecosystem

In the world of professional computing, we often think of hardware as the primary variable. But in the world of local AI, the runtime is king.

For years, the industry has been effectively “NVIDIA-polarized.” Most of the optimization work, the low-level shims, and the bleeding-edge drivers are built around CUDA. When you step outside that ecosystem into AMD’s world, you aren’t just installing a driver—you are navigating a fragmented landscape of ROCm, Vulkan, and DirectML.

Think of it like the early days of automotive travel. You might have a high-performance engine, but if there’s no standardized fuel pump at every gas station, you’re forced to jury-rig a solution just to get down the road. On Windows, that “fuel” (GPU acceleration) was inconsistent. It would often appear to be working until you pushed it, at which point it would silently throttle or hand the workload back to the CPU to avoid an error.

What Performance to Actually Expect

One of the most common questions after switching was: “How fast does it actually run?”

Here are rough baselines for the Ryzen AI HX 370 with Radeon 890M using llama.cpp + Vulkan on Ubuntu 24.04:

Model	Quantization	Approx. Tokens/sec
7B (e.g., Mistral, Llama 3)	Q4_K_M	~20–30 tok/s
13B (e.g., Llama 2 13B)	Q4_K_M	~10–15 tok/s
Mixtral 8x7B (MoE)	Q4_K_M	~12–18 tok/s

These numbers will vary based on context length, KV cache quantization, and system tuning. For a quick verification, run:

./build/bin/llama-cli -m model.gguf -ngl 99 -p “Hello world”

If you see GPU utilization in radeontop and tokens flowing steadily, you’re on the right track. If everything still falls back to CPU, the troubleshooting section below will help.

Why I Switched to Ubuntu

Once I realized the hardware wasn’t the problem, the choice to move to Linux became clear. Moving a dedicated workload from a general-purpose OS like Windows to a specialized Linux build is akin to shifting from an all-purpose city street to a dedicated racing circuit: the asphalt is flatter, the corners are banked, and there are far fewer pedestrians wandering into your path.

The Mesa and AMDGPU drivers on Linux are simply more mature. When you use Vulkan on Ubuntu, you aren’t fighting a layer of abstraction designed for gaming or UI rendering; you are interacting with a compute stack that the AI community has battle-tested for years.

Architecting the Optimal Linux AI Stack

I didn’t want to just “run Linux.” I wanted an architecture that prioritized stability and performance, which led me to a hybrid approach.

Layer	Runs	Why
Inference (llama.cpp, Vulkan)	Native host	Direct GPU access, lowest overhead
Frontend (Open WebUI)	Docker	Easy upgrades, isolated config
Databases (PostgreSQL, Qdrant, ChromaDB)	Docker	Reproducibility, clean volumes
Automation (n8n, LangFlow, Flowise)	Docker	Rapid experimentation
Caching/Queues (Redis)	Docker	Service isolation

Running inference natively bypasses the overhead of container networking and resource isolation, giving the model direct, unobstructed access to the Radeon 890M’s compute units. Everything else benefits from Docker’s modularity.

As part of this shift, I also started reconsidering Ollama. It’s excellent for quick setup, but on AMD iGPUs, llama.cpp offers more direct control, clearer visibility into Vulkan behavior, and easier debugging of GPU offloading. Ollama isn’t bad—it’s just less transparent for this specific use case.

Should You Switch to Linux? A Quick Guide

This isn’t the right move for everyone. Here’s a rough decision framework:

• Switch if: You’re building a dedicated AI server, comfortable with the terminal, hitting silent CPU fallback on Windows, or want maximum hardware utilization.
• Stay on Windows if: You need the machine for daily productivity, rely on Adobe/Office workflows, or are still experimenting casually with smaller models.
• Consider dual-booting if: You want both worlds, but be aware of BIOS/EFI complexity and partition management overhead.

The Build Process (With the Details I Wish I’d Had)

Step 1: Back Up Everything

This step is easy to skip. Don’t. Before wiping anything, save:

• Model files (.gguf, .safetensors)
• Ollama model manifests (~/.ollama)
• Docker volumes (docker volume ls then back up each)
• SSH keys (~/.ssh/)
• Environment configs and .bashrc/.zshrc
• Any custom scripts or systemd services

Step 2: Install Ubuntu 24.04

Create a bootable USB with Ventoy or Balena Etcher. Boot, install, update fully.

Step 3: Configure BIOS

Before installing Vulkan or anything else:

• Update to the latest firmware
• Increase shared GPU memory to at least 8–16 GB if the option exists
• Disable Secure Boot (some Mesa/Vulkan paths require it)
• Enable Above 4G Decoding if available

Step 4: Install Vulkan Support

sudo apt update
sudo apt upgrade -y
sudo apt install mesa-vulkan-drivers vulkan-tools -y

Verify with vulkaninfo. You should see the Radeon 890M listed.

Step 5: Build llama.cpp with Vulkan

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON
cmake --build build -j

Step 6: Run Inference

./build/bin/llama-cli -m model.gguf -ngl 99

The -ngl 99 flag attempts to offload all layers to the GPU. If you see errors, reduce it (e.g., -ngl 20) to find the maximum your system can handle.

Troubleshooting: When Things Still Don’t Work

If you’ve followed the steps and inference is still slow, run through this checklist:

• Ollama logs show compute = cpu: Vulkan isn’t being picked up. Confirm vulkaninfo lists your GPU, and check that Ollama was built with Vulkan support (ollama --version should mention it).
• vulkaninfo returns nothing: Mesa drivers aren’t installed correctly. Reinstall mesa-vulkan-drivers and check that your kernel is recent enough for RDNA 3.5 support (6.8+ recommended).
• Partial GPU offloading (e.g., -ngl 20 works, -ngl 99 fails): You’re running out of VRAM. With shared memory, you have headroom, but the kernel may need vm.dirty_bytes tuning or the model may be too large.
• Token generation stalls under load: Check for thermal throttling. The 890M is fast but compact; sustained workloads may need fan curve adjustment.
• Driver regression after kernel update: Pin to a known-good kernel or use ukuu/mainline to manage versions.

What’s Next: The NPU Horizon

Here’s the honest part: GPU + Vulkan is the best path available today, but it’s not where I want to stay.

The Ryzen AI HX 370 was designed with hybrid compute in mind: CPU for orchestration, GPU for graphics-adjacent parallel work, and NPU for sustained, low-power AI inference. Right now, only two of those three are pulling meaningful weight in my local LLM pipeline.

What I’m watching:

• RyzenAI-SW and Lemonade SDK — AMD’s own efforts to bring LLM inference to the XDNA NPU. Still early, but promising.
• ONNX Runtime with DirectML Execution Provider (DML EP) — Microsoft’s path that includes experimental NPU support on Windows.
• llama.cpp NPU backends — Community experiments are starting to surface. If they mature, this becomes the killer combo for AMD APUs.
• Strix Halo and future APUs — Rumors suggest significantly more TOPS and a more developer-friendly NPU ISA. Worth waiting for if you’re buying new hardware.

The reason I designed the hybrid architecture (native inference + Docker infrastructure) was precisely so I could swap the inference layer when NPU support stabilizes—without rebuilding the rest of the stack. That’s the real payoff of the Linux move: not just better performance today, but a flexible foundation for whatever comes next.

Looking Back

There was never anything wrong with the hardware. The MINISFORUM AI X1 Pro-370 and Ryzen AI 9 HX370 are genuinely impressive. The limitation was the software ecosystem—specifically, the current state of AMD acceleration on Windows.

After digging through Vulkan behavior, CPU fallbacks, GPU utilization, and runtime quirks, the conclusion became clear: Linux is a better fit right now for dedicated AMD local AI systems. Not because Windows can’t do it—but because Linux does it more reliably, more transparently, and more efficiently.

If you’re building a dedicated AMD-powered AI workstation today, Ubuntu paired with Vulkan-based inference is one of the most practical and performant paths available. And if you’re aiming for NPU acceleration eventually, you’ll want that Linux foundation in place long before the NPU tooling catches up.

As I look at this setup, I find myself wondering: at what point does the complexity of “optimizing for hardware” outweigh the benefits of using a system that “just works”? And when NPU support finally lands, will it feel like unlocking a third engine—or discovering that the car was always capable of more than the road allowed?

References & Further Reading

• Unlocking the NPU: FastFlowLM
• llama.cpp GitHub Repository
• AMD ROCm Documentation
• Mesa 3D Graphics Library
• Vulkan SDK Overview
• MINISFORUM AI X1 Pro-370 Specifications