gpumon · integration API

broker · writer · ingress · ocr-api

gpumon · architecture

gpumon is the federation's shared LLM, OCR, and telemetry plane. Every other repo (fraud-heuristics, edgar-cik-cli, future projects) routes LLM calls through gpumon-ingress, queues batch work on the federation broker, persists telemetry through gpumon-writer, and submits OCR jobs through gpumon-ocr-api.

Services

Service hashes are deterministic 4-hex slugs derived from the service name. Use them when referencing a service in a ticket, commit, or design discussion — [21a6] is unambiguous even when the same word appears in multiple repos.

Service Hash Role Host
gpumon-writer [21a6] Sole SQLite writer · HTTP ingest · SSE fan-out node-eleven (swarm)
gpumon-rollup [fa10] 5s → 1m roll-ups + retention; co-located with writer node-eleven
gpumon-collector [edf9] Polls nvidia-smi over SSH every 5s on each GPU host node-eleven (host network)
gpumon-ingress [25e6] LLM proxy: pool resolver, per-key RPM, cooldown node-eleven, port 4010
gpumon-rabbitmq [87d1] Federation AMQP broker — vhost federation, 7 queues node-eleven, port 5672
gpumon-ocr-api [15c7] Producer for ocr.jobs queue · HTTP POST /jobs node-eleven, port 2295
gpumon-ocr-worker [1e6d] Consumer for ocr.jobs · pdftoppm → vision → cleanup node-eleven (×N replicas)
gpumon-dashboard [1912] SPA + nginx proxy to writer / ingress / ocr-api node-eleven, port 2290

Data flow (SVG, canonical)

Centered diagram below is the canonical architecture view. See also the text representation for terminals / accessibility readers.

SVG diagram (default)

The diagram embedded at the top of the docs overview is the source of truth — same SVG, same flow.

Text representation (fallback) 
            ┌─────────────────────────┐
            │ your service [client]   │
            │ fraud-publisher · edgar │
            └────────────┬────────────┘
                         │ HTTP
        ┌────────────────┼────────────────┐
        ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ ingress 25e6 │  │ writer  21a6 │  │ ocr-api 15c7 │
│ /v1/chat/... │  │ /llm-calls   │  │ POST /jobs   │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       ▼                 ▼                 ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ pool-*       │  │ gpumon.db    │  │ ocr-worker   │
│ vLLM·NIM     │  │ WAL + WAL ck │  │ 1e6d (×N)    │
└──────────────┘  └──────────────┘  └──────────────┘
                         │
                         ▼
                  ┌──────────────────────────────────┐
                  │ federation broker 87d1           │
                  │ ocr.jobs · fraud.rows · edgar.*  │
                  └──────────────────────────────────┘

Integration paths (decision tree)

You want to … Use
Call an LLM (chat, embed, vision) gpumon-ingress HTTP — see ingress.md
Persist a row of telemetry / project / workload gpumon-writer HTTP — see writer.html
OCR a PDF (single shot or batch) gpumon-ocr-api HTTP — see ocr-api.html
Push batch work into a domain queue gpumon-rabbitmq AMQP — see broker.html and envelope.html
Watch a dashboard or stream events gpumon-dashboard + /stream SSE

Hard rules (NON-NEGOTIABLE)

  • No bulk data in envelopes. AMQP messages are ≤ 4 KB. PDFs/blobs live in IPFS (payload.ipfs_cid) or filesystem; the worker re-fetches.
  • Single-writer SQLite. Only gpumon-writer writes to gpumon.db. Everything else goes through HTTP endpoints. Consumers POST results back; they do NOT bypass the writer.
  • Heartbeat 600s on every AMQP connection. Default 60s breaks during long handler runs. See services/ocr-worker/src/amqp.ts for the canonical client.
  • Envelope v1 is forever-additive. Every consumer accepts {version:1, id, ts, attempt, payload} and ignores unknown top-level keys. New fields are additive, never breaking.

Related

rendered from docs/api/architecture.md  ·  v0 reference: writer hand-coded catalog ↗ (the layout that inspired this style guide)