The Token Is a Narrow Feedback Channel

The most interesting idea in Full-bandwidth transformer is not that a language model can “think without words”. It is that standard autoregressive generation throws away an already-computed route back through the model's depth.

A Transformer processes each new token in two directions. Horizontally, attention gives it access to earlier positions. Vertically, the token passes through every layer of the model. The horizontal channel is broad: the KV cache preserves representations of the previous sequence. The vertical return channel is much narrower. Once the model samples a token, that discrete token embedding becomes the next input at the bottom of the stack, while the previous top-layer hidden state does not.

The paper describes this as a bandwidth problem. A $D$-dimensional state containing uncertainty, partial results and other non-verbalised computation is compressed into one symbol before it can receive another full depth budget. Intermediate states are not entirely erased; many remain in the KV cache. They are depth-frozen: a representation produced high in the stack cannot return to an earlier layer on the next token. The model must either reconstruct that state or externalise it into a chain of tokens.

That framing made me reconsider what chain-of-thought is doing mechanically. It is not only explaining a result. It is also carrying computational state through a channel the architecture already exposes.

Widening the Return Path

The proposed full-bandwidth Transformer adds a second channel. At decoding step $t$, it combines the sampled token embedding $e_t$ with the previous top-layer state $h^L_{t-1}$:

$$u_t = W_U h^L_{t-1} \odot \sigma(W_G e_t)$$

The hidden state supplies the value, while the token supplies a multiplicative gate. The fused vector $u_t$ then enters layer zero as the next input.

The asymmetry matters. A simple residual addition could allow the model to ignore the hidden-state path and recover ordinary token embeddings. Here, discarding the state also discards the input value, so the model must learn to use the feedback channel. The token still determines the gating pattern and the model still emits ordinary text; latent feedback augments token generation rather than replacing it.

It does not inject information which was absent from the token history. The state is itself a deterministic function of that history. The proposed gain is computational accessibility: information which would otherwise need to be reconstructed by a fixed-depth pass can re-enter the bottom of the stack already processed.

The paper evaluates three inference regimes:

Regime	Prompt prefill	Generated-token input	Additional cost
Standard	One ordinary pass	Token embedding	Baseline
Soft	One ordinary pass	Token-gated hidden state	Two $D \times D$ projections per generated token
Fused	One ordinary pass plus one feedback pass	Token-gated hidden state	Approximately double prefill plus the Soft projections

The authors report less than one per cent additional per-token decoding cost for Soft. This does not mean that every use of the method is free. Fused decoding pays for a second full prompt pass, and the model itself must first be trained to accept hidden states in its input. A conventional pretrained checkpoint cannot simply have the switch turned on at inference.

The Training Problem Was the Real Problem

Exact latent feedback is sequential. The state at one position depends on the completed state at the position before it. Training it token by token would lose the parallel teacher forcing that makes Transformer pre-training practical.

The paper instead uses several full-sequence passes. Pass one is ordinary next-token prediction. On each later pass, the previous pass's hidden states are shifted one position, fused with the original token embeddings and processed again in parallel. This approximates a recurrent unroll while paying sequential cost across a small number of passes rather than across every token.

Most of pre-training remains single-pass. Feedback is introduced later through a schedule. For the 200-billion-token and 400-billion-token runs, 75 per cent of batches use one pass, 22 per cent use two and three per cent use three. A random plain-token prefix also teaches the model the boundary it will see during generation: a prompt processed with ordinary embeddings followed by recurrently fused output tokens.

The most informative training result is a failure. A model trained with only one- and two-pass batches worked at its trained depth but became unstable when the recurrence was iterated further. Validation loss rose and the hidden-state update oscillated. Adding three-pass training to only three per cent of batches changed the long-horizon behaviour: the update began to contract towards a stable fixed point and remained stable through far more passes than it had seen in training.

This is important because the useful recurrence is not merely one which fits the next training pass. It must survive being composed with itself hundreds of times during generation. The small three-pass mixture did not just improve a score; it changed the learned transition from a locally successful map into one which remained usable outside its training horizon.

It also makes the efficiency claim more precise. The 200-billion-token run consumed 256 billion token-equivalents of forward-pass compute, while the 400-billion-token run consumed 512 billion. The method appears more data-efficient than an ordinary Transformer, but it is not obtaining the gain without additional training computation.

What the Experiments Establish

The authors trained 1B-parameter models on up to 400 billion tokens, with standard baselines extending to one trillion tokens. Across validation loss and a ten-task, five-shot language-model evaluation, one or two feedback passes during prefill moved the recurrent models towards standard baselines trained on substantially more data. The 200-billion-token full-bandwidth model with feedback approached the 400-billion-token standard baseline in this setting.

The decoding comparison is stronger because it holds the recurrent model's weights fixed. On the base 200-billion-token model, Soft decoding raised MATH-500 Pass@1 from 0.27 to 0.37. It improved over Standard on GSM8K, HumanEval and MBPP as well. After long-context extension and instruction tuning, the 200-billion-token model still improved from 64.52 to 67.93 on GSM8K and from 42.54 to 45.92 on HumanEval when moving from Standard to the best latent-feedback regime.

The state-tracking probes support the proposed mechanism. With ordinary prefill, a probe at layer zero was near chance because the final colon token had not yet reconstructed the state of the preceding sequence. After one recurrent prefill step, layer-zero accuracy reached 99.6 per cent for completion tracking and 100 per cent for delayed memory. The previous top-layer representation had made global state immediately accessible to the bottom of the stack.

The authors correctly separate accessibility from use. A linear probe showing that information is present does not prove that the model will use it to choose a better output. The downstream generation results are necessary evidence alongside the probes.

The result on reasoning length is similarly bounded. Base models often produced shorter mathematical solutions under Soft decoding at equal or better accuracy. That effect disappeared after instruction tuning. The paper's explanation is plausible: post-training examples were generated in the verbose policy of ordinary token-by-token reasoning, so imitation training restored that style. It remains evidence about a base model, not yet evidence that production assistants will become both shorter and better.

This Is a Research Direction, Not a Drop-In Upgrade

Full-bandwidth Transformer is part of a much wider return to recurrence and latent computation. Feedback Transformer previously let future positions attend to a mixture of layer representations, although its training remained sequential over tokens. Two closely timed 2026 preprints make the overlap clearer. Latent Recurrent Transformer injects an upper-layer state from the previous token through lightweight KV and residual pathways. T2MLR feeds a later middle-layer representation into an earlier middle layer and reports that localised recurrence can outperform recurrence across the full stack.

The full-bandwidth paper acknowledges that these methods are similar in spirit and that no controlled large-scale comparison yet establishes which injection point is best. Its distinctive choices are the external top-layer-to-input gate, the two-matrix serving path and a larger free-form evaluation across models trained on up to 400 billion tokens. T2MLR, in contrast, has an official Apache-licensed implementation and reports a retrofit experiment on an existing 1.7B checkpoint.

Several boundaries therefore remain:

• the reported full-bandwidth language-model experiments remain at the 1B-parameter scale;
• the feedback-pass schedule is heuristic rather than derived from a general stability rule;
• the free-form coding results select temperature separately for each regime and estimate Pass@3 from ten rollouts per problem;
• the claimed serving compatibility is described through a vLLM modification, but the paper's Hugging Face record currently links no official weights, code or model;
• the optional Fused regime roughly doubles prompt-prefill computation, which matters for long agent contexts;
• shorter reasoning traces are not retained by the instruction-tuned models.

None of these invalidate the result. They define what would need to be reproduced before calling the architecture efficient at a larger model scale or useful in a deployed agent.

What It Changes for Agents

The distinction I find most useful is between computational state and organisational state.

Latent feedback changes how a model carries computation from one generated token to the next. It does not create durable memory between sessions, establish which business definition is approved, prove that a tool result came from the authorised source or explain why an action was permitted. A hidden vector may help the model reach an answer, but it cannot serve as the receipt for that answer.

This matters for Project Venus. A model which can preserve partial computation without verbalising it may require fewer generated tokens between a request and a constrained warehouse action. That could improve latency and reduce the pressure to turn every intermediate state into prose. It would not remove the need for explicit actions, deterministic validation, broker-executed queries and evidence attached to the final claim.

I would therefore test this architecture as a model intervention, not an authority intervention. A relevant Venus experiment would hold the training data, tool surface, runtime and evaluation deck fixed; compare Standard and latent-feedback decoding under the same weights; and measure action correctness, recovery after tool errors, final-answer support, generated-token count and end-to-end latency. Any gain which reduced visible evidence or weakened the broker boundary would be a regression, even if a benchmark score improved.

The paper does not show that language is unnecessary for reasoning. It shows that the sampled token need not be the only route by which computation crosses decoding steps. That is a narrower claim, but a more useful one. The token can become less of a computational bottleneck without becoming less of an accountability boundary.