# LLM-Agnostic by Design: Why Finance AI Shouldn't Be Locked to One Vendor

> Model optionality is useful only when the business contract stays stable, routes are inspectable, and every material model change is evaluated before rollout.

Published: 2025-11-20
Updated: 2026-09-10
Author: Archie Norman (Founder, MLX)
Category: Architecture
Tags: twin, llm-routing, llm-agnostic, model-selection, architecture
Canonical URL: https://mlx.systems/blog/llm-agnostic-by-design

## TL;DR

- The durable contract is the published Twin product (its meaning, fields, relation, validation, and evidence) rather than the behaviour of one model.
- Current MLX runtime contracts support multiple external provider families; the providers and models actually available still depend on target configuration.
- Changing a model is a controlled migration: run the relevant evaluations, inspect quality and cost evidence, canary the route, and retain an explicit rollback.

---

A finance team should not have to redefine revenue because it changes model provider.

That is the practical value of being LLM-agnostic. The model can change while the business definition, permissions and evidence requirements remain stable. The replacement still has to prove it can do the work.

## Keep the Definition Outside the Model

Imagine comparing two models on an overdue-receivables review. If each chooses its own invoice statuses, ageing buckets and currency treatment, the comparison mixes two questions: which model works better, and which definition happened to produce a more convincing answer?

A published Twin definition holds the second question steady. Both models use the same fields, reporting date, currency basis and source evidence. The governed query tool validates their requests and compiles read-only queries against the permitted data.

That does not force identical answers. One model may explain a movement more clearly or handle an unsupported request more reliably. Those differences are precisely what the evaluation should reveal.

A task skill can guide the procedure, but it cannot provide this boundary on its own. Different models may interpret the same instruction differently. Shared definitions and enforced tool contracts give the comparison a stable foundation.

## What Model Choice Means in Practice

MLX separates model routing from the governed business contract. Availability depends on the routes configured and permitted for the organisation; support in an integration does not establish that a particular customer can use it.

There are also different kinds of model choice. Using an external provider, operating an inference server and connecting a consumer AI application are separate decisions. [Project Venus](/research) covers our work on models running on infrastructure we control. [Connection availability](https://docs.mlx.systems/docs/connect/availability) describes the supported ways people and applications reach the Twin.

For an actual deployment, ask which model will run, where it runs, what context it receives and how its identity is recorded. Those answers matter more than the length of a provider list.

## Compare One Workload at a Time

Finance work is not one benchmark. A bounded receivables query, extraction from a complex contract and a draft board narrative place different demands on a model.

For a replacement route, keep the product, tool permissions and test questions fixed. Then compare:

| Check | A useful test |
| --- | --- |
| Product selection | Does it choose the definition intended for the question? |
| Numerical accuracy | Does the narrative preserve the returned values, dates and currencies? |
| Missing evidence | Does it identify an unsupported forecast or drill-down? |
| Tool use | Does it recover appropriately from a rejected query? |
| Practical performance | What are the observed latency, usage and cost? |

Record failures as well as successes. A model that writes better prose but omits a material freshness caveat may be the wrong choice for that workflow.

## A Model Change Is Still a Change

Switching a configuration value can be easy. Establishing that the new route is suitable takes evidence.

Start with representative cases and known failure modes. Run the available product and workflow evaluations, compare results, then try the route on a limited set of approved work before wider use. Retain a way to return to the previous route.

Check which model actually executed. A requested model name alone does not prove what served the run. Likewise, missing usage or cost evidence should remain unknown rather than appearing as zero.

Repeat the relevant checks when the model, product definition or tool boundary changes. A result from last month's configuration is useful history, not permanent certification.

## Keep the Option Useful

The aim is not to change models constantly. It is to make a change possible without moving company knowledge back into an agent's prompt.

A team may switch for better capability, lower cost, a different data boundary or continuity when a provider retires a version. In each case, the valuable asset is the definition and evaluation evidence the organisation keeps.

The model earns its place by using that shared business context well.