1 · Introduction
Move understanding out of the answer path.
Connectors deliver partial backfills, renamed fields and values that mean something other than they appear to. The runtime agent discovers these mid-answer, in front of users, after spending an estimated 30–120 s per question listing tables, describing them and trying queries. Existing checks stop at logs: the volume check only logs too_low, and coverage drift only posts to Slack.
Venus Colony moves that understanding out of the answer path (Figure 1). Leaves own slices of records, internal nodes hold roll-ups of their children, and the root holds an always-on summary that every node inherits. When a question arrives, the runtime agent hands it to the node that already knows the relevant data. This document specifies the node and its limits (§3); one mechanism for ingestion and growth (§4); the mechanisms that keep the tree stable (§5); how information and authority flow (§6); and the evidence so far, a pilot plan and the open decisions (§7–8).
Today · discovery inside the answer
est. 30–120 s per question, in front of users
- Question
- List tables
- Describe them
- Try queries
- Answer
Venus Colony · understanding already built
code checks every record ·agents read what changed
- Question
- Hand to the node that knows
- Answer
2 · Design principles
One tree, one rule, and information that flows up.
- P1One tree.Every node has exactly one parent, and depth varies by area.See §4 →
- P2One rule.No node may exceed its limits; exceeding one creates a sub-node.See §5 →
- P3Information flows up.No peer messages; parents settle conflicts with evidence, not votes.See §6 →
- P4Private territories.Only a node and its sub-agents see its state. People and the audit log can read everything.See §3 →
- P5Records stay at the source.Nodes hold record IDs and their own notes, never copies.See §3 →
- P6Code proposes, agents decide.Code clusters, routes and checks; agents name, split and judge.See §4 →
- P7Cost follows change.Agents read what is new or surprising, and code checks the rest.See §4 →
- P8Organisation-wide data only.Nothing user-scoped; a task thread only if all of it is org-visible.See §8 →
3 · The node
Every node, from the root to a single-record leaf, has the same structure.
Its main agent sets priorities, decides when to split and chooses what to report, making it the node’s regulator. The KV cache library keeps the node’s prompt as a prefix tree of a base prompt and overlays chosen per query type. The memory palace consolidates memory within the node’s limit, promoting, demoting or discarding items, and writes the roll-up its parent receives. The data representation sub-agent owns the node’s membership (record IDs and a routing rule). It keeps one note per record (embedding, checks passed, whether it has been read, which beliefs cite it) and writes beliefs with SQL checks.
← read by ID
Node e.g. Invoices 2025
Limits
fills → split
Private state
- membershipIDs and a rule
- record notesone per record
- beliefs+ SQL checks
- memorybounded
- promptand overlays
outside caller ⇢
trusted code: permissions · org-wide data only · limits · quotas
Main agent · regulator
Sets priorities, decides when to split and chooses what to report, making it the node's regulator. It watches the three limits.
- agents
- code
- stored data
Reports and privacy
A report carries the claim, the SQL that tests it and its result, the record versions it rests on (raw_event IDs), the node’s confidence and what depends on it. The parent keeps only what fits within its own limit, and its roll-up lists exactly which children it came from. When a newer version of a cited record arrives, every belief resting on it is re-checked.
Nothing inside a node is visible to other agents, and a parent sees only what it is told. People and the audit log can read everything, and every answer carries its SQL. Nodes propose logical views; physical database views are created only after approval.
Report · Invoices 2025 → Finance
Checked- Claim
- Invoice INV-2025-0418 has two payments allocated to it.
- SQL that tests it
select invoice_id, count(*) as payments from invoice_payments where invoice_id = 'INV-2025-0418' group by invoice_id having count(*) > 1;
- Result
- 1 row · payments = 2
- Rests on
- raw_event re_7q2k · v3raw_event re_7q2m · v1
- Confidence
- 0.86
- Depends on it
- Finance roll-up · ageing report overlay
- 1The leaf writes the belief with the SQL that tests it, and reports it to its parent.
- 2A newer version of a cited record arrives: the second payment was reversed.
- 3Every belief resting on that record is re-checked by running its SQL again.
- 4The claim no longer holds, so the belief is withdrawn and what depends on it is re-checked in turn.
One request extends exactly one path. Everything before its own task tokens is already cached: the root summary for every node, the roll-up for the subtree, the node's own segment and the reconciliation overlay.
4 · Ingestion and growth
One mechanism for the first load and every later change.
The colony grows like a B-tree[3] built by clustering. Sources land in raw_events and the canonical tables. Code profiles every record and embeds it with a small frozen encoder, so every record has been processed before any agent reads one. A single root then owns everything. It is over its limit, so code proposes clusters, and the root’s agent names them and writes each child’s charter. Children repeat this in parallel until every leaf fits. Top-level splits follow sources and tables, and deeper splits follow values such as client, period or type.
- 1 connectsources → raw_events
- 2 profile + embedevery record, in code
- 3 root splitsagent names clusters
- 4 recurseuntil every leaf fits
- 5 learn, then watchagents read the new
- 6 route changesdown from the root
1 · code
Sources land in raw_events and the canonical tables. One mechanism handles the first load and every later change, for a database of any size.
A single root owns every record. It is far over its limit.
- root (L3)
- internal node (L2)
- leaf (L1)
- over the limit
A leaf learns its slice by reading representative and unusual records until further reading stops changing its beliefs. It then watches: code checks every new record, and the agent reads only failures, new kinds of record and a small random sample. Parents build roll-ups from their children’s reports.
Learning: the leaf reads representative and unusual records until further reading stops changing its beliefs.
Steady state
High update volume does not mean reclustering.
Updates do not move records.
Splits follow what a record is (source, table, client, period), not its state (status, balance), so a paid invoice re-runs the checks in its own leaf.
New records fill leaves.
Code routes them down from the root, and a full leaf splits locally. The tree is never reclustered as a whole.
Misfits wait.
Records that fit nowhere wait at the level where they stopped fitting, and become a new child once there are enough of them. A newly connected source lands as a new branch.
Drifted nodes re-split.
A node that collects many poor fits is re-split from its own records, which also corrects the order sensitivity of incremental trees.
An invoice in Invoices 2025 is paid. Its status changes; what it is does not.
- internal node (L2)
- leaf (L1)
- code
- over the limit
The shape is close to BIRCH[1], with capacity-limited nodes, root-down insertion and local splits. It borrows COBWEB’s create, merge and split operators[2], and agents, not fixed distance rules, make each decision. Because code splits in parallel below the first level, building the tree costs about one split per level. The root’s agent reviews cluster summaries and representative records, and never reads the database itself.
| Token Atlas (September 2026) | Venus Colony |
|---|---|
| L3: a brief always in the prompt (the “root prefix”) | The root's summary, first in every node's prompt |
| L2: roll-ups with exact child lists | Internal nodes' roll-ups of their children's reports |
| L1: digests of N records | Leaves: up to C records, with notes, beliefs and checks |
| Three levels, made variable on 30 September | Depth grows per area from the limit rule |
| Embedding lookup as a shortcut to detail | Embeddings for clustering, routing and jumping to a leaf; never the memory itself |
depth = 1 + ⌈log32(1 million / 1,000)⌉ = 3
- Depth
- 3 levels
- Leaves
- 1,000
- Nodes
- 1,033
- root1
- level 232
- leaves1,000
Within the cap: every node fits, with 962 to spare.
5 · Stability
One rule and one cap keep the tree well formed.
The rule is the limit rule. A parent’s children’s reports count against its own memory, so the rule also bounds fan-out: no parent can gather fifty children, and a parent left with one child merges with it. The cap is a maximum node count, set per tenant from serving capacity. Node cap × leaf capacity is roughly the most records the colony can watch closely, and dropout decides where that attention goes.
A leaf over its limit splits into children.
- internal node (L2)
- leaf (L1)
- over the limit
Dropout. When the cap is reached, the least important leaf is dropped. Candidates are shortlisted by recent use, verified findings, dependants, data churn and cost, never by call frequency alone, and new nodes get a grace period. A fold-in test then replays a sample of the candidate’s own past tasks with its parent answering. The node is dropped only if the parent does about as well, so a rarely used but important node survives. This is pruning by ablation rather than random dropout.
1 · Shortlist
Candidates are shortlisted by recent use, verified findings, dependants, data churn and cost, never by call frequency alone. New nodes get a grace period.
- recent use
- verified findings
- dependants
- data churn
- cost
- call frequency alone
- new nodes: grace period
7 · Evidence and pilot
What is known so far, and how the pilot will find out the rest.
- The roll-up hierarchyNever builtNever built end to end.
- Token Atlas runTimed outThe only run timed out before producing results.[6]
- Learned memory against retrievalLost every testLost to retrieval with a frozen encoder in every test so far, so embeddings remain an index and not the memory.[7, 8]Learned memory8/192Retrieval, frozen encoder182/192
An example from the tests: embeddings remain an index, not the memory.
- Open-weight serving smoke test, 28 OctoberPassedNative tools, prefix caching and yes/no logprobs passed: the mechanics work, which is not yet evidence that the agents are useful.
- Prefix caching on candidate node modelUnprovenNot yet shown to work.
- Factory throughputNever measuredHas never been measured.
- UK dedicated GPU qualificationIn progressQualification runs on dedicated UK GPU capacity.
| Correlated errors | Evidence and blind re-checks. |
|---|---|
| Lossy roll-ups | Exact child lists and drill-down. |
| Prompt injection in records such as CRM notes | Read-only tools, and outputs limited to reports and triage items. |
| Parents as bottlenecks for cross-territory questions | The pilot measures this, and it bears on decision 1. |
8 · Decisions
Eleven decisions to freeze the specification.
| # | Decision | Proposed default |
|---|---|---|
| 1 | What the top-level split follows | Content plus links (shared IDs, foreign keys), so the same client from Xero and HubSpot lands together. Needs entity matching up front. |
| 2 | Records stay at the source | Yes. Nodes hold record IDs and notes; contents are read from the canonical tables. |
| 3 | Runtime agent routing | As an outside caller. Capability-card search is the baseline, and the one-token yes/no broadcast is tested against it. |
| 4 | Models | A fast open-weight model for node agents on the factory, with our validated private checkpoint for the pilot while UK capacity qualifies. The runtime agent stays on its configured model. |
| 5 | When the colony runs | Factory batch windows, interactive work first. Always-on is a separate decision. |
| 6 | Where findings go | A triage item with evidence, and dependent views marked impaired. Never notify-only. |
| 7 | How much gets re-checked | A blind re-check for every flag that reaches people; the rest by sample. |
| 8 | Task threads as input | Not in the pilot. Added later under the whole-thread rule (P8). |
| 9 | Dropped nodes' evidence | Archived in cold storage, so the node can be re-created cheaply. |
| 10 | Pilot data and residency | A read-only role on an isolated research copy, with processing in the UK only. |
| 11 | Success, kill criteria and budget | More verified findings per GPU-hour than the single-investigator baseline, with no loss of interactive latency. Budget to be set. |
Set now, tuned in the pilot
- Leaf capacity C
- Per-node memory and KV limits
- The node cap
- Split and drop thresholds, with a grace period
- The learn-to-watch rule
- The watch-mode sample rate
- Dropout weights
- The embedding model
The frozen Arctic encoder the Twin already uses is the first choice of embedding model.
Approvals before production, not needed for the pilot
- Explicit governance for agents that keep private state between tasks
- No notify-only monitoring: every finding creates an inspectable triage item
- A verified tenant-isolation proof across node state and storage
- A dedicated model-gateway caller class and quota for the colony
References
- [1]T. Zhang, R. Ramakrishnan and M. Livny. BIRCH: an efficient data clustering method for very large databases. Proc. ACM SIGMOD, 1996.
- [2]D. H. Fisher. Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2(2):139–172, 1987.
- [3]R. Bayer and E. McCreight. Organization and maintenance of large ordered indexes. Acta Informatica, 1(3):173–189, 1972.
- [4]R. G. Smith. The Contract Net Protocol: high-level communication and control in a distributed problem solver. IEEE Transactions on Computers, C-29(12):1104–1113, 1980.
- [5]Project Venus. Earlier multi-agent roster and staged implementation study. Project Venus research note, October 2026; partly superseded by this specification.
- [6]Project Venus. Venus Token Atlas hierarchy and calibration study. Project Venus research study, September 2026.
- [7]Project Venus. Fast Weight Memory Campaign. Project Venus research study, October 2026.
- [8]Project Venus. Territory memory and frozen-encoder retrieval comparison. Project Venus research study, October 2026.