Skip to content
Build Perch
Browse documentation

Running local models with Ollama

Short answer: Install Ollama, pull a model, and Build Perch detects it automatically: no API key, and nothing leaves your machine. Model size and your GPU’s VRAM determine how well it runs; Build Perch shows you a fit estimate before you commit.

What is Ollama?

Ollama is a free, separate application that runs AI models on your own hardware. Build Perch doesn’t bundle it: you install it yourself, pull a model through Ollama’s own tooling, and Build Perch detects and lists whatever you’ve downloaded. Chat with a local model and nothing about the conversation ever reaches a server. It needs no key, account, or network connection once the model is downloaded.

This is one of three providers alongside OpenRouter (remote, keyed) and Ollama Cloud (remote, keyed subscription). Local Ollama is the only fully offline option.

How do I run a local model with Ollama?

  1. Click Download Ollama from Build Perch’s Connect AI Model dialog (or go to ollama.com/download directly) and install it.
  2. Pull at least one model using Ollama’s own interface or CLI.
  3. Reopen the Connect AI Model dialog in Build Perch. It detects the running Ollama instance, waits for it to come up if it was just started, and lists every model you’ve downloaded.
  4. Pick a model. Build Perch shows its size, quantization, context window, and whether it supports tools and vision (sourced live from Ollama itself) right in the picker.
  5. Name the chat and connect. You can switch to a different local model, or to a remote one, at any time from that same chat.

Good to know

  • Your hardware determines which models are usable. As a rough starting point, a 7–8B model wants about 8 GB of RAM; larger models need a dedicated GPU with matching VRAM. Build Perch surfaces a live “Suggested for your GPU” recommendation and a fit status (comfortable / tight / exceeds your GPU’s VRAM) based on your detected graphics card; it doesn’t set a hard limit, since Ollama still runs an oversized model, just slower.
  • The context-window size is a per-chat slider. Open the Local Model control beside a local chat’s Mode pill to set how much the model can hold in memory at once (4K up to 64K, clamped to what the model actually supports). Higher uses more GPU memory; the popover shows the estimated memory cost and fit status live as you move the slider. Tool-enabled chats need more headroom: a few thousand tokens go to tool definitions before any conversation fits, so 4K leaves little room once tools are in play.
  • Small context windows get a slimmer default toolset. When a local chat’s context window is 16K or smaller, Build Perch holds back the heaviest skills (document creation, templates, media, chat recall, live workspace snapshots) and the Office document tools on each send so the prompt fits with room left for the conversation; Chat Configuration → Skills shows a note whenever this is active. The AI can still request a held-back capability mid-chat when a task needs it, toggling any skill in that panel puts the chat under your manual control instead, and raising the context-window slider past 16K restores the full set.
  • No GPU is required by Build Perch itself. The app’s own built-in AI features (voice dictation, semantic search) run on CPU. A GPU only matters if you’re running a local Ollama model, and that’s entirely optional.
  • Local models cost nothing to run, unlike OpenRouter or Ollama Cloud; there’s no per-token rate to track.
  • Response quality and speed vary widely by model and depend entirely on your hardware; Build Perch surfaces the capability metrics, but the tradeoffs are Ollama’s and your machine’s, not something the app can smooth over.