Guides

Local models with the edge runner

Run Ollama, vLLM or LM Studio behind your firewall without opening a port.

Handoff's servers cannot reach your laptop, and a tunnel to an unauthenticated model server is an open GPU faucet. The edge runner is the alternative: a small, dependency-free Node process that runs next to the model, opens an outbound HTTPS connection to Handoff, long-polls for model requests, calls the local model and posts the response back.

  1. Settings → Providers → Add, kind OpenAI-compatible, transport “Customer-side runner”, key empty.
  2. Copy the enrollment command from the provider card. It carries a token scoped to that binding.
  3. Run it where the model is:
HANDOFF_URL=https://qaexpert.radiatus.com/api \
HANDOFF_EDGE_TOKEN=… \
LOCAL_BASE_URL=http://127.0.0.1:11434/v1 \
npx handoff-edge-runner

The card shows connected state, measured tokens per second and a projected wall-clock per twelve-step test. A loop that takes forty seconds on a hosted model can take ten minutes on a laptop; the projection is there so that is not a surprise.

Nothing listens on your side, no inbound firewall rule is needed, and page observations for that binding never leave your network.

← Bring your own modelPlaywright export →
Local models with the edge runner · Handoff