MMI: testing the router before building the memory system

I wanted to test whether a geometric index could make long-term AI memory more efficient while preserving the original records. After an AI-assisted Rust prototype and a small set of controlled comparisons, I stopped expanding the proposed router. The useful outcome was a working research CLI, a benchmark harness, and evidence for a bounded engineering decision.

Separate preservation from retrieval

Multipole Memory Index, or MMI, proposed organizing embeddings into a hierarchy whose parent nodes stored aggregates of their descendants: collective weight, position, and spread. The hypothesis was that those aggregates could guide retrieval toward promising branches without examining most of the underlying records. Original events would remain available for direct retrieval rather than being permanently replaced by summaries.

That combines two questions. Can the system preserve the original evidence? Can this geometry retrieve it more efficiently? A working storage layer helps answer the first. It does not establish the second.

Give the routing hypothesis a budget

The experiment had a cap of ten engineering days, with an earlier stop if the router failed to show a meaningful advantage. With AI assistance, I built a Rust CLI with SQLite storage and interchangeable retrieval backends. Monopole and diagonal second-moment estimates ordered traversal through a covering-ball tree. Only conservative geometric bounds could authorize pruning.

This was a test of multipole-inspired traversal heuristics, not a complete fast multipole method. The tree variants shared topology and retained summaries so the comparison could isolate traversal order.

The comparisons included an ordinary tree, an exhaustive correctness reference, USearch’s optimized exhaustive scan, and its HNSW index. Every backend received the same normalized f32 coordinates. Including the optimized scan mattered: outperforming a validation-heavy reference implementation would be a weak reason to build a new index.

What the public-text screen showed

The diagnostic used 1,459 embeddings from 110 Markdown files in a pinned version of the Rust book, with 384 dimensions and 32 distinct authored conceptual questions. Each query requested ten neighbors. Mean strict recall@10 measured agreement with the exhaustive reference’s neighbor IDs, not semantic relevance.

The recorded October 24 run produced the following selected results. Each entry gives mean recall@10, followed by p95 warm candidate-search latency, rounded to the nearest microsecond:

• Ordinary tree, exact: 100.00% recall; 675 µs.
• Ordinary tree, 1,024-vector budget: 94.06% recall; 554 µs.
• Monopole ordering, 1,024-vector budget: 99.38% recall; 533 µs.
• Diagonal-moment ordering, 1,024-vector budget: 75.94% recall; 588 µs.
• USearch optimized exhaustive F32 scan: 100.00% observed recall; 110 µs.
• USearch HNSW, search expansion 64: 100.00% observed recall; 60 µs.

The strongest result was a work count: every exact tree variant evaluated all 1,459 corpus vectors on every public query. The tested bounds eliminated no leaf-vector evaluations on this corpus. That finding does not depend on a noisy latency percentile.

Monopole ordering did provide a useful signal. At the same 1,024-vector budget, it raised mean recall from 94.06% to 99.38%. But it was slower than both the optimized scan and HNSW64 on all 32 corresponding queries. Its two missing neighbor slots were concentrated in one query with recall 0.8; the average was not a per-query guarantee. The diagonal-moment extension performed worse.

Synthetic clusters provided a control. The ordinary exact tree pruned well on tightly separated groups, evaluating about 7–8% of vectors on average. Field ordering did not explain that success: exact monopole visited slightly more vectors, and the diagonal moment substantially more. On isotropic synthetic data, all exact tree modes again evaluated every vector.

What these measurements can carry

These were small, exploratory screening runs on a Linux x86_64 AMD EPYC runner. Each configuration had one timed sample per query after up to four warmups, with fixed variant order and uncontrolled host activity. Thirty-two queries cannot establish production tail latency.

The timer covered warm candidate search, including input checks and result materialization. It excluded embedding, index construction, source loading, lexical fusion, and evidence assembly. The CLI rebuilt indexes on each command. The roughly ninefold p95 candidate-search gap between budgeted monopole and HNSW64 is therefore not a ninefold difference in complete application latency.

The tree used scalar f64 distance calculations; the native baselines used optimized F32 kernels. These measurements compare implementations. Native recall of 100% means observed ID agreement on this sample, not a formal exactness certificate under the reference’s arithmetic and tie policy. The work counts and common-tree ordering comparison are more direct evidence about this routing design.

Backend allocation estimates were not isolated resident-memory measurements. No memory-efficiency claim follows from them. Nor does nearest-neighbor recall establish that an agent retrieves relevant evidence, recognizes a correction, answers correctly, or completes a task. The public book was not a held-out coding-agent memory workload.

Stop this mechanism, retain the useful work

I stopped expanding the monopole and diagonal-moment router. The source-preserving storage, ordinary retrieval backends, and benchmark harness remain reusable, and the code and recorded results are preserved. This was an early stop within the effort cap. The full confirmation gate—two held-out memory workloads, matched recall targets, and latency or resident-memory improvements—was not completed.

Reopening the router would require a specific changed hypothesis: a different bound, a different traversal mechanism, or a relevant workload whose geometry changes the result. It would need a separate budget and evidence requirements. Building a product would also require evidence for a real memory workflow that existing systems do not serve well.

A ten-minute check before the next experiment

Before adding a sophisticated router, put three results side by side: the ordinary version without the new heuristic, a competitive exhaustive scan, and a suitable approximate index at the same recall requirement. Count the underlying records examined as well as timing the query. State what the timer excludes. If removing the proposed mechanism leaves the useful behavior intact, attribute that benefit to the simpler system.

The lesson I am taking forward is practical: decide what would justify continued investment before building the surrounding product. A bounded experiment is useful when it gives you a clear reason to stop.