A language model that answers through DRAM

Ask this system "What is the capital of France?" and, some minutes later, it answers "Paris." Nothing exotic computed that answer. The multiplications inside the language model were carried out by ordinary DDR4 memory sticks — the same commodity parts in any desktop — coaxed, by breaking their timing rules on purpose, into behaving like a massively parallel analog computer. This page is the guided tour: what the machine is, what actually happens in the silicon, how a neural network's matrix multiply turns into memory-row tricks, and what we learned that we did not expect to learn. It assumes you know roughly what a transformer and a DRAM chip are, and nothing about this project.

Stop 1The machine on the desk

The hardware is a tower PC with an FPGA card (a Xilinx VU9P on a BCU1525 board) in a PCIe slot. The FPGA does not compute anything interesting. It is a programmable memory controller: it holds four DDR4 DIMM sockets, and it issues raw DRAM commands — activate this row, precharge, read — at precisely controlled times, including times the DDR4 standard forbids. The open-source instrument for this is DRAM Bender (from the SAFARI group at ETH); this project drives a four-controller build of it.

The host talks to the FPGA over PCIe (XDMA): it downloads little command programs (up to 8,192 instructions), says "run", and reads back whatever rows the program read. That's the whole interface. Everything else — the LLM, the request protocol, the calibration data, the laws in Stop 6 — is software built around that primitive.

Stop 2DRAM in three minutes — and the one distinction that matters

A DRAM bit is a capacitor: charged = 1, empty = 0. Bits are organized into rows (thousands of bits that share a wordline); rows into subarrays (512–1024 rows that share local sense amplifiers); subarrays into banks. To read anything, the chip activates a row: every capacitor in the row dumps its tiny charge onto its bitline, and the sense amp — a cross-coupled amplifier — snaps the ambiguous voltage to a clean 0 or 1 and drives it back. Reading is destructive: the act of sensing drains the capacitor, and the sense amp then restores it.

Capacitors leak. Left alone, a cell forgets its bit in seconds. So DRAM constantly refreshes: it re-activates each row on a schedule, letting the sense amp top up every capacitor.

Stop 3Breaking timing on purpose: one primitive, three operations

The DDR4 standard says: after activating a row, wait at least tRAS before precharging; after precharging, wait at least tRP before the next activate. Those waits exist so every analog transient settles. The entire compute substrate here is one command envelope that violates them deliberately:

• RowClone (copy) — long t1, tiny t2. The sense amps fully latch row R_F and are still driving the bitlines when R_S's wordline opens. R_S's capacitors have no say: they get overwritten with R_F's data. A row-to-row copy, thousands of bits at once, without a single byte crossing the memory bus.
• Multi-row activation (broadcast) — because the row decoder is built from predecoder stages that each latch part of the address, a fast PRE→ACT can leave stale predecoder state latched, so the second ACT opens many wordlines at once. Copy one row into sixteen.
• MAJ3 (the compute step) — both gaps tiny. Now no row gets fully latched first: many capacitors dump charge onto each bitline simultaneously, the voltages mix, and the sense amp snaps the mixture to whichever value the majority of the contributing cells held. Charge sharing is an analog vote. That's a majority gate — and a majority gate with one input pinned to 0 computes AND.

On real chips a bare 3-row vote is too fragile (process variation moves each sense amp's threshold), so each logical input is replicated: this project's calibrated shape opens 16 rows per vote — five copies of one operand, five of the other, five zeros, and one reference row conditioned to sit near half-charge so it leans on nobody. Which 16 rows open together, and how reliably they vote, is a per-chip empirical question — the project maintains per-DIMM calibration files discovered by multi-day sweeps (until Stop 6's laws made much of that computable instead).

Stop 4From a majority vote to a matrix multiply

BitNet's weights are ternary: every weight is −1, 0, or +1. That is the whole trick that makes DRAM arithmetic plausible, because a ternary × integer dot product needs no multiplier at all:

1. Split the weight vector into two bitmasks: pos (where w=+1) and neg (where w=−1).
2. Quantize the activation vector to int8 and slice it into 8 bitplanes (all the 1s-bits, all the 2s-bits, …).
3. For each (mask, bitplane) pair: AND the mask with the bitplane — that is the MAJ3 in DRAM — and popcount the result (count the 1s).
4. Recombine on the host: weight each bitplane's count by 2^b (MSB negative for two's complement), subtract neg-counts from pos-counts, apply the model's two scale factors. That integer is the dot product.

Everything the DRAM does is step 3, done 2,048 bit-positions at a time per row. Everything else is bookkeeping. A full 2560→2560 projection decomposes into a few thousand MAJ operations spread across four banks; a token through all 30 layers is a few hundred thousand. The model's answer is the accumulation of a very large number of analog votes, each individually ~99.9%+ reliable on screened columns, with the model's own quantization margins absorbing the residue.

Stop 5One request, end to end — and where the time actually goes

Follow one projection through the stack:

1. PyTorch calls the substituted linear layer with an activation vector.
2. The Python client quantizes, packs bitplanes (content-keyed cache — the adjective matters, Stop 7), and sends masks + bitplanes to the server. Weights it already delivered at load time stay resident in DRAM pool rows.
3. The C++ server emits a SoftMC program per batch of votes: refresh the subarray's rows (charge!), RowClone the resident weight into the compute tuple, write the activation slots (or, with the fused-coset lever, deposit them with two lattice-addressed doubleACTs instead of five writes), condition the reference row, fire the MAJ3, read the result row back.
4. The FPGA streams the commands into the DIMM and DMA's the read rows to the host.
5. The server popcounts the returned rows, accumulates, replies; the client rescales and hands PyTorch a tensor. The model never learns where the math happened.

Where does the time go? Not in the DRAM operations — a vote is tens of nanoseconds. It goes into round-trips and readout: every program launch costs host↔FPGA latency, and every vote's result is a full 8 KiB row that crosses PCIe so the host can reduce it to a single count. Measured on this rig: a packed row-read costs ~34 µs; the MAJ itself ~2.6 µs. The system is a memory-bandwidth machine being used as a compute engine, and its own readout is the bottleneck — which is why the two "readout killer" roads exist (Stop 8), and why per-token time is quoted in seconds, not milliseconds. On a real PIM controller without the FPGA round-trip, the same command stream would be orders of magnitude faster; that projection is stated as a projection, never blended into the measurements.

Stop 6The laws: when calibration becomes computation

The project's scientific center of gravity moved in July 2026 from measuring this particular silicon to predicting it. Three closed-form laws now describe what the address decoder does, and each one retired a pile of empirical screening:

The co-activation lattice (whose rows open)

When a doubleACT pairs rows A and A⊕d, the candidate rows that can open are exactly the XOR lattice {A⊕S : S ⊆ bits(d)} — a consequence of predecoder stages latching independently. The "spread" that once looked like mysterious corruption is this lattice seen from the source side: the sense amps drive every open row to the source's value, including rows nobody meant to open. The same physics is a hazard (it can overwrite neighbors, and even a MAJ's own operands) and a tool (one doubleACT can deposit a value into a chosen set of rows at once — a free fan-out).

The selection law (which lattice members fire)

Only some lattice members actually open, and the rule is exact: the ten in-block address bits fall into predecoder units ({1,2}{3,4}{5,6}{7,8}, bits 0 and 9 alone), and member A⊕S fires iff every unit contributes either nothing or its entire share of d. Verified member-by-member: 1,691/1,691 observations on each of two dies, zero exceptions, invariant across all tested timings — and byte-identical between dies, meaning it is a property of the chip design, not the specimen. Physically: each predecoder unit is a latch that either holds the old pattern or takes the new one whole; it cannot split. The law's scope was itself measured — it lives in 1024-row decoder blocks (the 640-row "subarrays" are sense-amp segments, a different partition), and one address bit turns out never to deposit at all.

The clone-dead law (which rows can feed a tuple)

Loading a compute tuple by RowClone fails for ~17% of source rows — the copy lands diluted, because the pair offset drags too many lattice siblings open and the restore charge splits too many ways. The rule is arithmetic: count the predecoder units the offset touches; five or more is dead, with one exactly-characterized four-unit exception family. Its validation is the project's proudest procedural artifact: predictions for an unseen tuple's 624 rows were committed to a file before the silicon ran — 2,496/2,496 correct, and 2,494/2,496 on a second die.

Drift (the lattice, seen over time)

The last law is dynamic: rows sitting resident in a subarray drift away from their loaded content under that subarray's compute traffic — a one-shot transition to a saturated level that is ordered by the same coupling classes (strong-coupling rows drift most). Refresh cannot help, because refresh restores charge, not content (Stop 2). Idle decay is a separate, additive term. This physics was characterized in controlled arms — and then predicted a production failure: the fused kernel's resident all-ones/all-zeros constant rows had been placed on deposit-safe rows, which are exactly the strong-coupling, fastest-drifting class. Protocol tests passed (fresh constants); the full model babbled (drifted constants). The fix — rewrite the constants' content each request, right after the refresh — restored byte-identical-to-control output.

Stop 7Why a passing test can hide a failing model

The campaign's hardest week produced its most transferable lesson. After the new bitstream landed, every protocol test passed — bit-exact — and the full model produced garbage. The causes were four stacked host-side bugs, none visible below full-model scale:

• A relative config path silently fell back to an unscreened default pool — all weights loaded into wrong rows, while every stat looked plausible. (Rule since: explicit config that can't be honored is fatal, never a fallback.)
• A cache keyed on a Python object's memory address: the allocator recycles addresses, so later token positions read earlier positions' cached bitplanes — deterministic, and dependent on the prompt's allocation choreography, so short prompts dodged it.
• Scratch rows for streamed requests were drawn over the resident-weight pool — harmless in every small test (which fit in the pool with room to spare), destructive at layer 15 of a real run.
• A global config window mis-scoped the voting calibrations' pools — every "extra" vote voted garbage, silently.

Stop 8Two lanes and two roads — the honesty architecture

The project runs two deliberately separated efforts. Lane 1 is our system: the BitNet ternary LLM, free to borrow any idea that helps. Lane 2 is a reproduction of someone else's paper — MVDRAM (arXiv:2503.23817), the closest peer system, which ran GeMV kernels for standard LLMs inside unmodified DDR4. Lane 2 runs their models (Llama-2/3, Phi-4, via llama.cpp), their conventions, their dataflow, and is never allowed to quietly benefit from Lane-1 machinery that their paper doesn't describe — because a reproduction that borrows from its own conclusions isn't one. By July 20 the reproduction had run their four-model protocol at declared sampled scope (36 DRAM-verified operations inside real generations, 99.58–99.99% exact) and produced its first bit-exact fp32 outputs on real Llama-2-7B tensors — while also finding, and stating, that the paper's own partial-sum granularity cannot produce exact fp32, and that its performance rests on a command-streaming shape our per-program rig deliberately does not fake (a measured ~10⁵ gap, named as the one open item). The full story is its own deck (the MVDRAM reproduction explainer).

Within Lane 1, the same honesty pattern governs the readout bottleneck. Road A kills it in DRAM: a carry-save tree of majority-gate full adders reduces K product rows to log₂K result rows before anything crosses the bus (faithful to what a real processing-in-memory chip could do; ~99% per-lane exact; 213× fewer readout rows at production K — though on this FPGA-round-trip rig, not yet a wall-time win, which is stated in the same breath). Road B kills it in the FPGA: a popcount accumulator in the readback path returns 4 bytes instead of 8 KiB — bit-exact, 2048×, but it proves nothing about DRAM. The standing decision (ADR-005): keep both, label both, publish the comparison, never blend them into one number.

Stop 9The scoreboard, read honestly

13.3× in one campaign, with output quality rising — not because speed causes correctness, but because both came from the same source: replacing "noise" with named mechanisms. The remaining named levers: the Road-B bitstream (in flight), a residency campaign whose economics were measured and found not-yet-worth-it (a decision recorded with its numbers, like everything else here), and the streaming execution shape that separates this rig from a real PIM part.