Self-Hosted AI Infrastructure

A practical private AI development environment that connects a Windows workstation to Apple Silicon inference through a private network and exposes local models to coding tools through OpenAI-compatible endpoints.

OllamaMLXllama.cppGGUFApple SiliconTailscale

Abstract flow from a development workstation through a private network to an Apple Silicon local LLM server

Context

The Problem

AI-assisted development needed a private, locally controlled inference path that development tools could reach remotely, while working within Apple Silicon unified-memory limits and the capabilities of local model runtimes.

System flow

Architecture

Windows 11 Development Workstation

Tailscale / Headscale

Mac mini M4

Ollama / MLX / llama.cpp

Local LLM

OpenAI-Compatible API

VS Code / OpenCode / Continue / Coding Agents

Implementation

Engineering

Experimented with Apple Silicon inference using Ollama, MLX, llama.cpp, Metal acceleration, and both GGUF and MLX-native models.
Tested models from the Qwen, Gemma, and DeepSeek families without treating model installation as the end of the infrastructure work.
Exposed local inference to remote development machines through a Tailscale/Headscale private network and OpenAI-compatible endpoints.
Integrated local models with VS Code, OpenCode, Continue, coding agents, and MCP-based tooling.
Explored context-window tuning, prompt caching, tool-capable models, and memory/performance tradeoffs.

What required care

Challenges

Selecting quantization and context settings within unified-memory constraints.
Balancing model capability, memory use, and practical responsiveness without relying on unsupported benchmark claims.
Troubleshooting remote inference and coding-tool compatibility across the private network.

Technology

Stack

Status: Experimental

OllamaMLXllama.cppGGUFApple SiliconTailscale