Skip to content
→ All work

Case study 33 / 37

ModelHarbor

A self-hosted dashboard for every configured LLM provider: live health and latency per model, task-aware routing with live failover, and a usage-cost ledger.

Status
In development
Period
2026
Domain
AI · Web
Language
TypeScript
Last push
26 OCT 2026
License
MIT
Source of claims
README.md, CHANGELOG.md, docs/PRODUCTION_READINESS.md, server/src/index.ts, server/test/setup.ts, .github/workflows/ci.yml; mock-mode server run from a fresh clone 2026-10-25
ModelHarbor dashboard with provider health cards, routing panel and cost totals

01/The problem

Teams running several LLM providers cannot see which models are up right now, routing lives in if-statements nobody trusts, and the monthly invoice is a surprise.

02/The system

A zero-framework Node server and a Vite + React dashboard. Each model is health-checked on a TTL; each task type has a ranked pool of models; a request that fails at call time fails over to the next candidate, and every attempt is recorded as a usage event with token counts and cost. A deterministic mock provider runs the whole thing with no API keys.

Server modules (README)7 components
  • Dashboard to api.ts
  • api.ts to health.ts
  • api.ts to router.ts
  • api.ts to execute.ts
  • health.ts to Providers
  • router.ts to health.ts
  • execute.ts to Providers
  • execute.ts to costStore.ts

03/Implementation

  1. 01Health: every configured model is checked on a TTL and classified healthy, degraded or down by latency, with the error surfaced when a check fails.
  2. 02Routing: each task type (code, chat, fast, long-context) has a ranked candidate pool; the router picks the highest-priority healthy model.
  3. 03Execution: if the chosen model's call fails, the request fails over live to the next candidate and records every attempt.
  4. 04Cost: every executed request, success or failure, becomes a usage event, aggregated by model, provider, task and day, optionally persisted to JSONL.
  5. 05Adapters for OpenAI-compatible endpoints, native Ollama and a deterministic mock used by CI and the quickstart.

04/Engineering

Deciding is separate from doing

GET /api/route answers from the health cache with no side effects, so it is safe to poll; POST /api/route is the only path that runs prompts and writes usage events.

Keys stay on the server

/api/config drops API keys and strips credentials from base URLs; provider error text has keys, bearer tokens and sk- strings replaced before it reaches the dashboard.

Unauthenticated, so local by default

The server binds 127.0.0.1, sends no CORS headers unless one exact origin is configured, requires JSON content-type on POST and rejects bodies over 1 MiB.

05/Interface

The dashboard with health for 8 mock models across 3 providers (one degraded, one down), task-aware routing and the usage and cost panel after 9 routed requests.
The task-aware routing panel choosing aurora-coder-large for a code task and showing the mock response it executed.
The usage and cost panel: total requests, total cost, token counts, cost by model and task, and recent routed events.

Real screenshots of the running software. Captions state the data shown.

06/Tech stack

  • TypeScript
  • Node.js
  • Vite
  • React
  • Vitest
  • npm workspaces

07/Result

Verified outcomes

  • 144 Vitest tests with coverage gates (90% statements, lines and functions; 85% branches), per the README; a network guard fails any test that reaches a non-loopback host.

Known limitations

  • The live OpenAI-compatible and Ollama adapters are tested only against stubbed fetch, not live endpoints.
  • No authentication: anyone who can reach the API can spend on your providers.

08/Links

GitHub repository ↗

Related work

Next case study

PacketLens →