Guide to Local AI: Picking a PC in September 2025

To learn more about Local AI topics, check out related posts in the Local AI Series 

The future of AI isn’t just in the cloud; it’s right on your desktop. Running AI models locally offers unparalleled privacy, speed, and creative freedom. But with so many options from AMD, Intel, and Apple, how do you choose the best “Local AI PC” for you? Let’s break down the leading architectures and what they bring to the table.

Why Go Local with AI?

Before we dive into the hardware, let’s quickly recap why local AI is so compelling:

• Privacy: Your data never leaves your machine. Perfect for sensitive information or personal projects.
• Speed & Latency: No internet connection needed, no server queues. Instant responses.
• Cost-Effectiveness: Once you have the hardware, running models is free.
• Customization: Full control over your models, fine-tuning, and experimentation.

Now, let’s explore the Options

AMD: The Rise of Ryzen AI & Radeon Power

AMD has made significant strides, particularly with its “Ryzen AI” initiative.

The All-in-One: Ryzen AI Processors (APUs)

• What it is: These are not just CPUs. AMD’s Ryzen AI Max+ series (e.g., the 395 model) are “Accelerated Processing Units” (APUs) that pack a powerful CPU, an integrated Radeon GPU, and a dedicated Neural Processing Unit (NPU) onto a single chip.
• The Game Changer: Shared Memory (VRAM): This is where AMD directly challenges Apple. The integrated GPU and NPU can share a large pool of fast system memory (LPDDR5X) with the CPU. Why is this huge for AI? Large Language Models (LLMs) need vast amounts of VRAM. By sharing system RAM (up to 128GB in some mini PCs), these APUs can load and run very large models (e.g., 120B parameters) that discrete GPUs with less VRAM simply can’t handle.
• Ideal for: Compact mini PCs and laptops. If you want a small, power-efficient, yet powerful AI workstation that can run significant models, these are a top choice.
• Example: A mini PC with a Ryzen AI Max+ 395 and 64GB+ of LPDDR5X RAM.

The Powerhouse: Custom Desktops with Discrete Radeon GPUs

• What it is: For maximum raw performance, a traditional desktop build with a separate AMD Ryzen CPU and a powerful discrete AMD Radeon GPU is the way to go.
• VRAM is King: When it comes to local AI, the GPU’s dedicated VRAM is the most critical factor. More VRAM means you can run larger, more complex AI models.
• Top Picks: Look for GPUs from the Radeon RX 7000 series (like the RX 7900 XTX with 24GB VRAM) or newer Radeon AI Pro cards (up to 32GB VRAM). These chips feature dedicated AI accelerators.
• Software Ecosystem: AMD’s ROCm platform provides the necessary tools and libraries to run popular machine learning frameworks like PyTorch and TensorFlow, with growing support for consumer cards.
• Ideal for: Enthusiasts and professionals who need the absolute highest performance for demanding AI workloads, stable diffusion, and model training, and don’t mind a larger desktop footprint.

Intel: Core Ultra & Arc Graphics Step Up

Intel is also heavily invested in the local AI experience, especially with its recent architectural shifts.

The Integrated Solution: Intel Core Ultra Processors

• What it is: Intel’s answer to AMD’s APUs, the Intel Core Ultra (especially Series 2) processors also integrate a powerful CPU, an Arc Graphics integrated GPU, and a dedicated Neural Processing Unit (NPU) on a single chip.
• How it Works: The NPU is designed for sustained, low-power AI tasks, offloading work from the CPU and GPU. This is excellent for features like real-time video effects, but also for running small-to-medium AI models efficiently.
• Performance: The latest Core Ultra Series 2 chips boast significant AI performance, with up to 120+ total platform TOPS (tera operations per second) across all components.
• Ideal for: Laptops and compact NUC-style mini PCs. Great for users who want an efficient AI PC that excels at daily tasks and can handle a wide range of AI applications.
• Software Ecosystem: Intel’s OpenVINO toolkit and the Intel AI Playground app provide a robust environment for optimizing and running AI models.
• Example: An ASUS NUC powered by an Intel Core Ultra Series 2 processor.

The Discrete Option: Intel Arc GPUs

• What it is: For those building a more powerful desktop, Intel offers its Arc series of discrete graphics cards.
• VRAM Matters: Cards like the Arc A770 (16GB VRAM) offer a solid entry point for local AI, providing dedicated hardware for AI acceleration.
• Ideal for: Budget-conscious builders or those already in the Intel ecosystem who want to add dedicated GPU power for AI tasks.

Apple: The Unified Memory Powerhouse (M3, M4 Architectures)

Apple’s M-series chips offer a fundamentally different approach, built on an ARM architecture.

The Unified Memory Advantage

• What it is: Apple’s Silicon uses a Unified Memory Architecture (UMA). This means the CPU, GPU, and Neural Engine all share a single, high-speed pool of RAM.
• Why it’s a Game Changer for AI: This is Apple’s killer feature for local AI. Because there’s no separate “VRAM” that the GPU is limited to, your entire system RAM (which can be up to 128GB on an M3 Max, and a staggering 512GB on an M4) is available to the AI model. This allows Macs to run massively larger AI models than many discrete GPUs from AMD or Intel that might only have 24GB or 32GB of VRAM. A Mac with 64GB of unified memory can easily handle a 60B or 120B parameter model.
• Performance & Efficiency: Apple’s M-series chips are legendary for their performance-per-watt. The M4, with its 38 TOPS Neural Engine, is incredibly fast and efficient for AI tasks, especially in portable form factors like the MacBook Air or iPad Pro.
• Software Ecosystem: Apple’s tightly integrated Core ML and Metal frameworks ensure highly optimized performance for AI applications developed within their ecosystem.
• Ideal for: Users who prioritize power efficiency, silent operation, and the ability to run very large AI models on a laptop or compact desktop. Perfect for those already in the Apple ecosystem.
• Example: A MacBook Pro with an M3 Max (64GB unified memory) or the latest M4 iPad Pro.

Comparing the Options: A Quick Look

The NVIDIA Option (New!)

(Promising but locked to NVIDIA software-stack – I not yet tried)

While AMD, Intel, and Apple are all making big strides, NVIDIA is also entering the local AI PC space with a unique and powerful approach. Their GB10 “Grace Blackwell” Superchip powers a new class of devices often referred to as “GB10 PCs” or “AI mini-supercomputers.” These systems, exemplified by NVIDIA’s own “DGX Spark” and Dell’s “Pro Max” desktops, combine a high-performance, ARM-based CPU with a powerful Blackwell-architecture GPU and a massive amount of unified memory (up to 128GB). The key here is the integration: the CPU and GPU are linked by a high-speed NVLink connection, offering blazing-fast communication and allowing the GPU to access the entire pool of system memory. This architecture is designed to handle immense AI models—up to 200 billion parameters—that would be impossible to run on most consumer PCs, all in a compact, power-efficient form factor that sits right on your desk.

As for the operating system, these specialized systems typically run on a custom-built version of Linux. NVIDIA provides a full-stack AI platform, with software and drivers highly optimized for their hardware. These devices, often part of the NVIDIA DGX family, use the NVIDIA AI Enterprise software platform, which is built on a Linux distribution like Ubuntu to provide a robust and secure environment for AI development and deployment. This allows developers to leverage NVIDIA’s extensive ecosystem of tools and libraries, like CUDA-X, with an experience tailored for AI

Which is Right for You?

• For the Ultimate Desktop Power User: If you’re building a desktop and want the absolute highest performance for AI tasks (and perhaps gaming), an AMD Ryzen CPU with a high-end discrete Radeon GPU (e.g., RX 7900 XTX) remains a formidable choice, especially if your models fit within the GPU’s VRAM.
• For a Balanced, Compact Windows PC: An AMD Ryzen AI Max+ mini PC or laptop or an Intel Core Ultra Series 2 laptop or NUC offers an excellent blend of performance, efficiency, and the ability to run significant AI models thanks to their integrated NPU and shared memory concepts.
• For Unrivaled Large Model Capacity & Efficiency in a Compact Form: If running the absolute largest AI models (e.g., 60B, 120B+ parameters) is your priority, especially in a laptop or compact form factor where power efficiency is key, Apple’s M3 or M4 Pro/Max chips with generous unified memory configurations are incredibly hard to beat.

The local AI landscape is rapidly evolving, and all three companies are pushing the boundaries. The “best” choice truly depends on your specific needs regarding model size, performance demands, portability, and existing ecosystem preferences. Happy AI exploring!

What Do I Use?

I need to learn and explore different options and test drive different things. I use an Apple Mini M4 for some of my primary lab and development and also an old Dell Precision Workstation with an Quatro NVIDIA GPU with 256Mg of RAM which I plan to upgrade soon.

Connect with me if you would like to chat about your use case and that might be the best PCs for your Local AI needs!