MathalyseResearch

News

News from the lab.

Releases, papers and announcements from Mathalyse Research, newest first.

Release

Phase-HDC paper released

The Phase-HDC paper is available on arXiv, with code and an interactive version in the playground. It trains a hyperdimensional classifier without any optimiser memory.

TLDRA learned classifier stores each parameter as a 6-bit dial. At every update a dial turns one notch against its gradient, but only when the gradient is larger than a fixed write cost, so no optimiser history is kept at all. On a controlled digits benchmark it reaches 92.4% test accuracy against 91.2% for Adam with 6-bit moments, with three times less training state (4.27 KiB against 12.80 KiB). The accuracy gap is within noise, so the result shows that the idea works, not that it beats Adam.

Announcement

Mathalyse Research launches

Mathalyse Research is an independent lab working on optimisers, architectures and the theory of learning. This site collects its released work and announcements. Collaborators and students can write through the contact page.

Release

ThermoLion preprint released

The ThermoLion preprint is available on arXiv. It argues that training deep vision networks calls for adjusting the precision of the update dynamically, between the magnitude-based updates of AdamW and the sign-based updates of Lion.

TLDRThermoLion tracks how consistent each parameter’s gradient has been. Consistent gradients get Adam-like proportional steps, and noisy ones get Lion-like fixed-size sign steps plus a little noise that cools over time. The preprint tests this on a single small ConvNet (12 epochs, up to 5,000 training samples per dataset) and reports gains over Adam on the harder image datasets.