The thesis

Deep learning largely settled on a small set of computational assumptions: learning by global gradients, fixed-width overparameterization, dense continuous representations, and offline training over stationary data. The neural architectures of the preceding decades made different choices — local writes rather than gradients, structure allocated on demand, memory access as computation, match-before-overwrite category formation, discrete state as a feature. NN-Revival takes those old architectures seriously and asks a narrow, empirical question: implemented carefully on current hardware, with honest budgets and pre-registered predictions, what still works, what fails, and which knobs actually matter?

The point is never "can a 1980 model beat a transformer?" (it can't, and asking discards exactly what makes the older designs interesting). The point is what each architecture buys under real constraints: update memory, inference energy, MAC count, online learning without replay, parameters touched per example, capacity added per new concept.

Method

Every entry in the series reuses the same shape, so the results are cumulative and directly comparable rather than a grab-bag.

  1. Pre-registered predictions. Written and frozen in the study's SPEC.md before any multi-seed run is executed. Outcomes append only — predictions are never edited after the fact, including the ones that turn out false.
  2. Multi-seed, honest error bars. Three seeds minimum; mean ± range reported as (max − min)/2, not the fake precision of stdev/√n from a tiny sample.
  3. Frozen result artifacts. Every published figure is generated by a committed script from result JSONs whose SHA-256 hashes ship with the paper. Anyone can re-derive the tables from the frozen data.
  4. Verify before you cite. The starting-point surveys for this series are machine-generated. Every DOI, archive URL, and quantitative claim is verified against the primary source before it appears in a published entry. Deep-research tools fabricate plausible-looking citations; a single hallucinated DOI destroys the credibility the rest of the honesty earns.
  5. No novelty overclaim. These architectures are prior work. Claims are limited to revived / tested / augmented under modern controls. Extensions the series introduces are labeled as such, not as inventions.
  6. Not SOTA, and say so. Toy scale, small baselines. "Beats X" means "at matched small scale"; a real CNN or transformer would win. The comparison target is small dense nets at similar footprint, not modern deep architectures.

Papers

  • #1 · 2026-08-10
    Weightless Neural Networks on Modern Compute

    Classic WiSARD and Kanerva's Sparse Distributed Memory re-run on binarized MNIST under modern controls. Includes a dual-mode hybrid (frozen gradient RAMs + plastic one-shot writes) as a class-incremental probe. Findings the paper keeps honestly: the write rule — not the addressing — is WiSARD's ceiling; sectioning prevents cell overwrite but plastic logit domination remains a second interference channel.

Cadence

NN-Revival is an ongoing exploration, not a scheduled release program. Papers land when a study is genuinely done — predictions frozen, multi-seed results in hand, honest scorecard (including the misses) written. The series may stop at one paper. It may also grow into a substantial body of work. Both outcomes are acceptable to it; only shipping bad research isn't.

Contact

For questions about the methodology or specific studies, reach [email protected].