MathalyseResearch

New methods for how networks are built and trained.

Mathalyse develops optimisers and architectures, and the mathematics that explains why they work. Each method starts from a mechanism, and each claim is sized to its evidence.

Gradient descentloss
Momentumloss
Adamloss

Three optimisers released from the same point on one non-convex loss surface. The deepest well sits to the right of centre. The update rule decides which well each run reaches. Click the surface to release all three from a new point.

Research areas

Three lines of work that share one habit: derive the method from a mechanism, then test it against strong baselines.

Optimisers

Update rules built from an explicit account of what gradient noise, curvature and step size do to training. Every optimiser is evaluated against the strongest published competitors, including our own earlier ones.

Released: ThermoLion.

Architectures

Attention and sequence models analysed with tools from signal processing and quantum physics. The aim is to say what an architecture can represent before training it.

Theory of learning

Statistical mechanics and gauge theory applied to deep learning. Results are stated as theorems with proofs, with experiments kept to what the argument needs.

Released work

Papers that are publicly available. New releases are added here.

Preprint

Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning

Ahmed Nebli. arXiv:2610.10630, October 2026.

TLDRA learned classifier stores each parameter as a 6-bit dial. At every update a dial turns one notch against its gradient, but only when the gradient is larger than a fixed write cost, so no optimiser history is kept at all. On a controlled digits benchmark it reaches 92.4% test accuracy against 91.2% for Adam with 6-bit moments, with three times less training state (4.27 KiB against 12.80 KiB). The accuracy gap is within noise, so the result shows that the idea works, not that it beats Adam.

Read on arXiv  ·  Code  ·  Try it in the playground

Preprint

Unifying Sign and Magnitude for Optimizing Deep Vision Networks via ThermoLion

Ahmed Nebli. arXiv:2512.01881, December 2025.

TLDRThermoLion tracks how consistent each parameter’s gradient has been. Consistent gradients get Adam-like proportional steps, and noisy ones get Lion-like fixed-size sign steps plus a little noise that cools over time. The preprint tests this on a single small ConvNet (12 epochs, up to 5,000 training samples per dataset) and reports gains over Adam on the harder image datasets.

Read on arXiv  ·  Code  ·  Try it in the playground

People

A small lab, open to collaborators.

Ahmed Nebli

Founder

Computer scientist working on optimisation, sequence models and the theory of deep learning.

Mathalyse welcomes collaborators in optimisation theory and sequence modelling, and students looking for a well-posed problem to work on. Send a message and a CV.

Mechanism before benchmark

A method has to come with an explanation of why it should work. A leaderboard gain alone does not count.

Claims sized to evidence

Results are stated at the strength the experiments and proofs support, and narrowed when they do not.

Reproducible by default

Public datasets, fixed protocols and released code, so a reader can rerun the comparison.

Latest news

Releases and announcements from the lab.

All news

Support the lab

Mathalyse is independent and self-directed.

Contributions pay for compute, release costs and the upkeep of released code.