gpu-federation-monitor — Integration API

Why this exists / when to use what

gpu-federation-monitor (a.k.a. gpumon) exposes four integration surfaces that other repos (fraud-heuristics, edgar-cik-cli, claude-mem batches, etc.) call into. This directory is the integrator's reference — not the internal architecture, not the deployment guide. The master design doc lives at docs/specs/federation-queue-architecture.md; the docs here are the public-facing contracts derived from that spec and from the actual service source under services/.

Pick the path that matches your use case from the decision tree below. Then read the corresponding file for headers, body shapes, and copy-paste examples (curl + Bun fetch).

Decision tree — which surface for which use case

I want to …
│
├── call an LLM (chat, completion, vision, embedding)
│   └── ──► gpumon-ingress (HTTP)         ◀── ingress.md
│           POST /v1/chat/completions
│           POST /v1/embeddings
│
├── record telemetry about an LLM call I already made / am about to make
│   └── ──► gpumon-writer (HTTP)          ◀── writer.md
│           POST /llm-calls/start  (returns request_id)
│           POST /llm-calls/finish (outcome)
│           NOTE: if you call through ingress, ingress writes this for you.
│
├── register a project, workload, or cooldown
│   └── ──► gpumon-writer (HTTP)          ◀── writer.md
│           POST /api/projects/register
│           POST /workloads
│           POST /cooldown/:key
│
├── OCR a PDF (vision → markdown + text, with IPFS persistence)
│   └── ──► gpumon-ocr-api (HTTP)         ◀── ocr-api.md
│           POST /jobs (multipart upload or raw PDF bytes)
│           GET  /jobs/:id
│           GET  /jobs/:id/result.{txt,md,json}
│           Behind the scenes: publishes envelope to `ocr.jobs` queue.
│
├── publish a unit of cross-machine work that another worker will pick up
│   └── ──► federation broker (AMQP)      ◀── broker.md + envelope.md
│           amqp://federation@node-eleven:5672/federation
│           exchange: federation.work (direct)
│           queues: ocr.jobs, fraud.rows, edgar.ciks, edgar.filings
│
├── consume work from a queue (write a new worker / scale an existing one)
│   └── ──► federation broker (AMQP)      ◀── broker.md + client-pattern.md
│           queue-bound consumer with prefetch + heartbeat=600s
│
├── live-tail what's happening on the fleet
│   └── ──► gpumon-writer (SSE)           ◀── writer.md
│           GET /stream
│
└── read aggregated GPU + LLM telemetry for a dashboard
    └── ──► gpumon-writer (HTTP read APIs)◀── writer.md
            GET /api/gpu/*  /api/llm/*  /api/health-summary  /api/workloads

Surface map (one line each)

All addresses use LAN IPs (per global routing rule); Tailscale is not on the data path. In-swarm callers use the overlay names (gpumon-ingress, gpumon-writer, etc.) — same service, different DNS.

End-to-end walkthroughs

Two complete examples that span multiple subsystems live near the end of this directory:

• "I want to OCR a PDF" → see ocr-api.md — ocr-api → broker (ocr.jobs) → worker → writer (/api/ocr/jobs/finish) → poll GET /jobs/:id/result.md. Three subsystems, one call from the caller's perspective.
• "I want to record an LLM call's outcome" → see writer.md — ingress receives headers → ingress calls /llm-calls/start → real LLM call happens upstream → ingress calls /llm-calls/finish → writer emits SSE → dashboard renders the row.

Hard rules for integrators

1. Send the gpumon headers on every LLM call (x-gpumon-client-id, x-gpumon-project, x-gpumon-pool). Without them attribution breaks and your project will not appear in the dashboard. See ingress.md.
2. Envelope v1 is forever-compatible. Producers MUST emit {version: 1, id, ts, attempt, payload}. Consumers MUST accept v1 and ignore unknown top-level keys. See envelope.md.
3. AMQP heartbeat MUST be 600s on consumers that run long-tail work (OCR pipeline, SEC downloads). The 60s default kills the connection mid-job. See broker.md and the Cane audit lesson referenced in client-pattern.md.
4. Workers manage their own retries. Republish with attempt+1 for transient failures; nack(requeue=false) after MAX_ATTEMPTS (default
   5. so the broker DLX-routes the envelope to the per-domain DLQ.
5. Idempotency. /api/ocr/jobs/{start,finish,fail} are idempotent by id. Terminal states are sticky (a retried start cannot un-finish a job).

Versioning

• All HTTP endpoints under /api/* are stable. Endpoints without /api/ (/llm-calls, /samples, /workloads, /stream, /health) are also stable — the prefix is historical, not a quality signal.
• Envelope version: 1 is the long-lived contract. A version: 2 would be additive (new fields) and v1 consumers MUST keep working.
• New pools and queues are additive. Removing or renaming a pool / queue is a coordinated breaking change tracked in the master spec.