On-Prem AI for Finance: A Practical Path for Regulated Teams

“On-premises” can answer one useful question: where does this component run?

It does not answer:

• which external services the component calls;
• where model prompts and outputs are processed;
• who operates the database and worker;
• how identities and secrets are managed;
• what an agent is allowed to query or change;
• which business definition produced an answer;
• how releases, backups, incidents, and audit evidence are handled.

For a regulated finance team, those questions are the deployment.

MLX's architecture separates product contracts, connector configuration, model routing, and runtime services. That is useful groundwork for supporting more than one topology. It is not evidence that every topology is available today.

Current MLX targets are managed deployments, and current configured model routes use external providers. A customer-operated VPC or on-premises target should therefore be treated as a deployment project with explicit acceptance criteria, not as a checkbox already proven by the repository.

Start with a Boundary Map

Before installing anything, map every component and external dependency:

• client applications and authentication;
• control plane and canonical Postgres state;
• task worker and Electric runtime;
• connector runner and source endpoints;
• secrets and configuration authority;
• model providers and the context sent to them;
• logs, metrics, backups, and release artefacts;
• operator and support access.

For each connection, record the initiator, protocol, data class, authentication method, retention behaviour, and owner.

This prevents a common mistake: moving the worker into a private network and declaring the whole system private while prompts still travel to an external model provider or logs still leave through an observability integration.

An on-premises topology changes some trust boundaries. It does not collapse them to one.

Separate Four Acceptance Questions

1. Can the target be operated?

A customer-operated target needs more than a successful build:

• reproducible installation and upgrade procedures;
• migration ownership and rollback policy;
• release provenance for every running service;
• health checks that cover the real runtime path;
• backups and tested restoration;
• capacity, queue, and failure monitoring;
• a documented support-access path;
• a patch and incident-response process.

“The container started” is not production evidence. A regulated team needs to establish which release is active, whether workers claim and complete runs, whether canonical state is durable, and whether the deployment can recover from failure.

2. Is the data path acceptable?

Connector evidence, runtime state, model context, and observability data can have different paths. Assess each separately.

Approved MLX connectors land organisation-scoped evidence inside the selected deployment boundary. Current model routes use external providers, so the context supplied to the selected model follows that provider path. A customer-controlled worker does not change that fact.

Any future in-boundary inference route would need its own implementation and acceptance evidence: supported model contract, serving runtime, capacity, routing integration, quality evaluations, monitoring, and failure behaviour. Having a GPU or downloading open weights is not enough.

3. Is business meaning governed?

Private infrastructure cannot decide what “active customer,” “cash available,” or “contribution margin” means.

The Twin lifecycle provides that control:

1. connect scoped evidence;
2. discover a candidate business product;
3. review mappings, joins, grain, fields, and freshness expectations;
4. publish an approved version, stable relation, contract, and policy/routing snapshot;
5. retain validation evidence and, where defined, an evaluation suite;
6. let approved agents query the activated product through the governed read-only route.

Human review remains part of publication. The model does not promote its own proposed interpretation into company truth.

4. Are capabilities constrained?

The session requests capabilities, and the control plane grants the allowed intersection with tenant policy. That is stronger than asking a prompt not to write.

But a broad capability name does not prove that a particular connector supports safe writeback. Connector tools have their own implementation and permission boundary. Some current finance adapters are read-only.

Any write path should be evaluated independently from the hosting topology, with:

• a real connector implementation;
• narrow typed inputs;
• identity and permission checks;
• approval policy;
• idempotency and duplicate handling;
• audit events;
• bounded test fixtures;
• recovery and reconciliation procedures.

On-premises does not make a write safer. It only changes where the unsafe write could originate.

A Phased Adoption Plan

Phase 0: Qualify

Document the required residency, network, identity, support, recovery, and audit outcomes. Compare them with the exact target that can be supplied today. Record gaps as gaps.

The output is a responsibility and data-flow map, not a generic architecture slide.

Phase 1: Prove the target

Deploy the minimum runtime and establish:

• active release identity;
• database migration state;
• service and worker health;
• authentication and organisation scope;
• secrets delivery;
• outbound network behaviour;
• logs and metrics;
• backup and restore.

Do not connect production finance data until these controls have owners and evidence.

Phase 2: Connect one read-only source

Choose a bounded source and approved fixture. Verify the connector layer by layer:

• source access succeeds with the intended identity;
• raw events or records land in the correct organisation scope;
• freshness advances;
• errors are visible rather than silently skipped;
• no unsupported mutation tool is exposed.

Connector installation is not the same as successful ingestion.

Phase 3: Publish one Twin product

Use the connected evidence to derive a narrow product such as receivables ageing or account balances. Review its mappings and business meaning with the finance owner, then publish a versioned relation and contract.

Validation is required. Add product-specific evaluations where the use case warrants them.

Phase 4: Evaluate governed use

Test representative questions through the actual agent path. Confirm:

• the intended product was activated;
• queries stayed inside the published contract;
• organisation scope was enforced;
• the answer cited the product, version, relation, and freshness watermark;
• unsupported fields or drill-downs were refused;
• requested and executed model identities were recorded;
• the human owner accepts the output for its intended use.

Only rendered answers and runtime evidence prove the workflow. A successful deployment script does not.

Phase 5: Broaden deliberately

Add sources, products, users, or model routes one at a time. Re-run the relevant acceptance checks as the boundary changes.

Treat writeback as a separate programme. Do not infer it from a general platform capability or from the fact that read-only governed queries work.

What to Demand Before Calling It On-Prem

Ask for evidence against the exact target:

1. Which components run inside the customer boundary?
2. Which components remain vendor-operated?
3. Which outbound endpoints are required?
4. Which model providers receive context today?
5. Where are canonical state, connector evidence, logs, and backups held?
6. How are releases identified, upgraded, and rolled back?
7. How are secrets, user authority, and organisation scope enforced?
8. Which Twin products and evaluations passed on this target?
9. Which connectors are read-only, and which write tools actually exist?
10. Who responds when the worker, database, connector, or model route fails?

If those answers are not available, the topology is not yet an operating model.

The Takeaway

On-premises can be the right requirement for a regulated finance team. It is not automatically the safest posture, and it is not a current MLX capability merely because the architecture can be decomposed.

The credible path is evidence-led: qualify the boundary, prove the target, connect narrow read-only evidence, publish and validate a Twin product, evaluate the real governed workflow, and expand only when the next control is ready.

That is slower than putting an “on-prem” badge on a diagram. It is also how a deployment becomes defensible.

---

Related: What “Customer-Controlled AI” Actually Means for Finance. A four-plane framework for separating deployment, data, model, and knowledge control.

Architecture: LLM-Agnostic by Design. The current external-provider surface and the evidence gate required for model changes.