Percepta Transformer VM: Exact Code, Not Hallucination-Free AI

Five months after its March 11, 2026 debut, Percepta’s Transformer VM is best understood as a public, code-backed proof of concept—not a hallucination-free language model. Percepta’s original technical post introduced the attempt to execute programs inside a transformer with a faster path for long execution traces.

The meaningful update is that the underlying implementation can now be inspected and reproduced. The public Transformer VM repository contains analytical weight construction, a WebAssembly interpreter, reference traces, tests and a C++ inference engine. That makes the work more substantial than an unsupported demonstration, but its scope is still much narrower than the phrase “the end of hallucination” suggests.

What Percepta actually put in the weights

Percepta did not train a conventional large language model until it spontaneously learned to behave like a computer. It constructed the weights of a small, standard softmax-ReGLU transformer analytically so that its attention and feed-forward layers implement the operations of a WebAssembly machine. In other words, the network is closer to a deliberately compiled digital circuit expressed with transformer operations than to a language model that discovered an interpreter through ordinary pretraining.

The repository exposes two related execution modes. In the universal mode, WebAssembly bytecode appears in the input sequence and the fixed transformer interprets it. A specialization mode can instead bake one selected program into feed-forward weights, eliminating the program prefix but producing a model dedicated to that program.

C source code is therefore not executed directly merely because it enters an LLM prompt. It must first be compiled to WebAssembly and converted into the project’s token representation. The current interpreter implements 35 WebAssembly opcodes; several harder operations, including multiplication, division and bitwise instructions, are expanded into sequences of supported operations during compilation.

During inference, generated tokens record changes in machine state: the instruction cursor, stack, locals, memory and output. Attention retrieves earlier state, while feed-forward components perform the fixed arithmetic and conditional operations. The transformer is executing a prescribed transition system, not improvising a textual answer about what the code might do.

Why its long traces can run quickly

The project addresses a genuine obstacle to program execution through autoregressive transformers. A conventional attention lookup compares a new query with an increasingly long history, so the amount of retrieval work grows with the trace. That becomes prohibitive when a solver produces hundreds of thousands or millions of state tokens.

Percepta’s cache exploits the special geometry of this constructed model. Its lookup heads use two-dimensional keys and queries with effectively hard-maximum attention. The winning key must lie on a maintained convex hull, allowing the inference engine to insert and retrieve entries in logarithmic rather than linear time.

This is not a universal replacement for standard attention. The optimization depends on two-dimensional lookup heads, hard routing and the structured access patterns of the compiled interpreter. It does not establish that an arbitrary language model can exchange its normal attention mechanism for the same cache without losing its linguistic capabilities.

The repository reports roughly 30,000 generated state tokens per second for its C++ path and describes a Sudoku execution trace of about 900,000 tokens. Those tokens are compact pieces of machine state, however, so their throughput should not be compared directly with prose tokens emitted by a large conversational model. Model size, token semantics and workload are fundamentally different.

Independent replication strengthened one claim and qualified another

A later, independently authored replication and reduction report reconstructed the analytical model and matched reference WebAssembly traces on six out of six test programs. Its longest reported run was a 1,055,417-token Sudoku trace, and its measured mean throughput was 18,049 tokens per second across approximately 1.29 million tokens—below Percepta’s roughly 30,000-token figure but within the same order of magnitude.

The difference is important because it turns a promotional performance number into a result with an observed replication range. It does not necessarily imply a contradiction: throughput depends on hardware, build settings and implementation details, and the two reports do not establish identical test environments. The safer conclusion is that million-step execution has been reproduced, while the headline speed should remain attached to its particular configuration.

The replication also found that the model is highly engineered rather than broadly learned. Its weights are analytically computed, its routing behaves like saturated digital switches, and many nominal attention slots are exactly unused. Removing those zero components preserved outputs in the report’s equivalence checks, but ordinary low-rank compression failed because tiny numerical perturbations could change discrete routing decisions.

Why this does not end hallucinations

Hallucination is a failure of generated content to remain grounded in facts, evidence or the user’s actual context. Percepta’s artifact demonstrates exact execution inside a transformer for a supported deterministic machine. It does not demonstrate that a general language model will select the correct program, provide correct inputs, interpret the output properly or avoid inventing a factual premise.

A perfectly executed wrong program still produces the wrong result. Likewise, a future hybrid model could calculate an invoice total without error while misreading the contract that determines the rate. Deterministic execution can remove one class of arithmetic and state-transition mistakes; it cannot by itself solve retrieval, source verification, ambiguity or probabilistic language generation.

The present artifact is also a standalone executor, not a finished combination of conversational intelligence and internal computation. There is no demonstrated general-purpose model that moves reliably between open-ended language reasoning and this virtual-machine mode, and no evidence that existing chatbots can acquire the capability by importing these weights.

What the distinction means for creator tools

For creators, the most plausible value lies in workflows that already contain deterministic stages. A future system might keep media metadata transforms, spreadsheet calculations, layout constraints or validation rules within a neural execution substrate while using a language component to understand instructions. That could reduce orchestration overhead in some architectures, but Percepta has not yet delivered such a creator product.

Factual content work would still need citations, provenance and human review. Running a calculation internally does not verify the source numbers, and executing a transformation exactly does not establish that the requested transformation was appropriate. The prototype is therefore relevant to reliable computation inside AI systems, not evidence that generated articles, captions or research summaries have become trustworthy by default.

The result that the prototype genuinely establishes

Percepta has supplied a concrete answer to a narrower and technically interesting question: a standard transformer architecture can be assigned analytically constructed weights that make it simulate a WebAssembly interpreter, and a specialized attention cache can keep very long execution traces practical. Public code and later replication give that claim more weight than it had at launch.

What remains open is the harder product question—whether exact internal execution can be integrated with a trained, general-purpose language model without sacrificing flexibility, efficiency or trainability. Until that bridge is demonstrated, Transformer VM is an experimental computer represented as a neural network, not the mechanism that ends hallucination.

Percepta’s Transformer VM Runs Code—but It Hasn’t Ended Hallucinations