The thesis

Deep learning largely settled on a small set of computational assumptions: learning by global gradients, fixed-width overparameterization, dense continuous representations, and offline training over stationary data. The neural architectures of the preceding decades made different choices — local writes rather than gradients, structure allocated on demand, memory access as computation, match-before-overwrite category formation, discrete state as a feature. NN-Revival takes those old architectures seriously and asks a narrow, empirical question: implemented carefully on current hardware, with honest budgets and pre-registered predictions, what still works, what fails, and which knobs actually matter?

The point is never "can a 1980 model beat a transformer?" (it can't, and asking discards exactly what makes the older designs interesting). The point is what each architecture buys under real constraints: update memory, inference energy, MAC count, online learning without replay, parameters touched per example, capacity added per new concept.

Method

Every entry in the series reuses the same shape, so the results are cumulative and directly comparable rather than a grab-bag.

  1. Pre-registered predictions. Written and frozen in the study's SPEC.md before any multi-seed run is executed. Outcomes append only — predictions are never edited after the fact, including the ones that turn out false.
  2. Multi-seed, honest error bars. Three seeds minimum; mean ± range reported as (max − min)/2, not the fake precision of stdev/√n from a tiny sample.
  3. Frozen result artifacts. Every published figure is generated by a committed script from result JSONs whose SHA-256 hashes ship with the paper. Anyone can re-derive the tables from the frozen data.
  4. Verify before you cite. The starting-point surveys for this series are machine-generated. Every DOI, archive URL, and quantitative claim is verified against the primary source before it appears in a published entry. Deep-research tools fabricate plausible-looking citations; a single hallucinated DOI destroys the credibility the rest of the honesty earns.
  5. No novelty overclaim. These architectures are prior work. Claims are limited to revived / tested / augmented under modern controls. Extensions the series introduces are labeled as such, not as inventions.
  6. Not SOTA, and say so. Toy scale, small baselines. "Beats X" means "at matched small scale"; a real CNN or transformer would win. The comparison target is small dense nets at similar footprint, not modern deep architectures.

Papers

  • #3 · 2026-08-23
    Cascade-Correlation on Modern Compute

    Fahlman & Lebiere's 1990 constructive learner — grow one frozen unit at a time, no end-to-end backprop — re-run faithfully in its native domain (two-spirals + a low-dim regression surface) with a labeled MNIST arm. The mechanism reproduces cleanly (it solves two-spirals by growing ~22 frozen units), but the founding "backprop can't do two-spirals" legend does not survive modern training: a two-layer MLP with Adam solves the task at 99.65% vs CasCor's ~79%. Honest verdict: (mostly) earned retirement. Not a flaw exposed — a road-not-taken overtaken by three decades of cumulative backprop tooling. The scorecard keeps N1 as a MISS. A post-freeze exploratory pass tried three fixes for the overshoot; a leakage-clean rerun produced a firm null. Code + frozen results: github.com/Vulkgryph/cascor-study.

  • #2 · 2026-08-17
    CMAC on Modern Compute

    Albus's 1975 CMAC re-run in its native domain — low-dimensional function approximation and open-loop control — with a labeled out-of-domain MNIST arm. Direct follow-through on #1's predicted fix: does sparse tiled addressing avoid the capacity saturation vanilla WiSARD hit? Central finding: across the swept range, adding capacity never degrades CMAC's quality; the one sharp failure (over-fine tiles) is coverage, not interference, and fully recovers with more data. Costs are honest: sample efficiency is local (small MLP wins the ultra-low-data regime), memory footprint is large, and MNIST — out of native domain — goes to the MLP. Code + frozen results: github.com/Vulkgryph/cmac-study.

  • #1 · 2026-08-10
    Weightless Neural Networks on Modern Compute

    Classic WiSARD and Kanerva's Sparse Distributed Memory re-run on binarized MNIST under modern controls. Includes a dual-mode hybrid (frozen gradient RAMs + plastic one-shot writes) as a class-incremental probe. Findings the paper keeps honestly: the write rule — not the addressing — is WiSARD's ceiling; sectioning prevents cell overwrite but plastic logit domination remains a second interference channel. Code + frozen results: github.com/Vulkgryph/ramnet-study.

Cadence

NN-Revival is an ongoing exploration, not a scheduled release program. Papers land when a study is genuinely done — predictions frozen, multi-seed results in hand, honest scorecard (including the misses) written. The series may stop at one paper. It may also grow into a substantial body of work. Both outcomes are acceptable to it; only shipping bad research isn't.

Contact

For questions about the methodology or specific studies, reach [email protected].