Building Math from Scratch

Why the project's math is hand-built rather than imported, what that produced, and the aesthetic and functionality of bit-level math made safe by Rust.

Where does the math come from in a project that won't import any?

A neural network needs trigonometry, vector norms, Fourier transforms. The standard answer is to import a library: ndarray, nalgebra, or whatever's idiomatic. This project doesn't have one.

"Don't reinvent the wheel" is the standard objection, and it's a fair one on a deadline. Two reasons it doesn't apply here. The wheel has features I don't want: math libraries bring heap allocation, dynamic shape inference, floating-point intermediates, broadcasting rules. The math here doesn't use any of those; The arithmetic is fixed-size integer math throughout, with shapes determined at compile time by const generics. Importing a library would mean importing its assumptions along with the operations actually used.

The other reason: the wheel doesn't have the variant I need. Libraries assume a generic consumer; generic consumers get generic implementations. My needs are specific, and custom code can be shaped to the use it serves. One function in the project is called radix4_encode. Backpropagation computes chain-rule products, and a linear quantization scheme would flatten the multiplicative structure of gradient flow. Log-space quantization preserves it. Linear quantization is everywhere; log-space quantization for this specific purpose lives in the project because that's where its justification lives.

Behind both practical reasons sits a rule I gave myself: no external dependencies in the foundation. The reason isn't that dependencies are bad. It's that the project is exploring how arithmetic shapes the network: block-float representation, integer-only operations, log-space quantization. If those choices live in someone else's library, they aren't really mine. The math has to be inspectable in my own code, in my own terms. Otherwise the project's relationship to its own arithmetic becomes "we use what ndarray exposes," which forecloses the question the project is asking.

What has the rule has produced so far? CORDIC for trigonometry: sixteen integer iterations, gain-compensated initial x, an arctangent table whose entries halve at each iteration until they bottom out at 1. The norm functions don't return distances; l2_norm_sq returns the squared norm as i64, so callers compute cosine-similarity comparisons as cross-product inequalities in i128. Two DFT paths: a K=8 fast path with five distinct twiddle values (±32767, ±23170, 0), and a generic-K path that calls CORDIC at runtime to compute twiddles on the fly.

The two DFT paths are the design payoff. Why two instead of one? No unsafe is allowed in the foundation, and unifying the [i64; 5] and [i64; K] return types in a single generic function would require a pointer cast. The clean alternative is to expose both: callers who know K=8 use the fast path directly, callers with variable K use the generic CORDIC path. Without the no-unsafe constraint, the cleanest design would be one generic function with a pointer cast inside. With the constraint, two simpler functions. The constraint produced cleaner code, not worse code.

Honestly, bit shifting is fun. Rust makes it accessible without making it dangerous: integer operations are first-class, overflow is well-defined, the type system tracks widths and signedness. The payoff: you can productively break mathematical assumptions. CORDIC's whole premise is that trigonometry, usually a transcendental operation hidden behind a library call, can be done with adds and shifts if the angle table is chosen right. The arctangent table is visually striking: the values halve at each step (16384, 9672, 5110, 2595, 1302, 652...) until they bottom out at 1, with the algorithm's convergence visible in the values themselves. The iteration body is where this lands deepest: x -= d * (y >> i); y += d * (x_prev >> i). The >> i isn't an optimization — it IS the trigonometry. CORDIC's angle table is chosen precisely so that tan(arctan(2^-i)) = 2^-i, which makes the multiplication by tan reduce to a right shift. The bitshift is the math.

The principled reasons for the no-dependencies rule are real, but they aren't the whole reason the project gets done. Bit-level math in Rust is a thing you'd want to build by hand: pleasant to write, expressive at the operational level, and held in place by a type system that catches the assumptions you didn't mean to make.