An FFT of the incoming audio gives a magnitude per frequency bin. One row of those magnitudes is written into a texture each frame, and the row pointer advances and wraps, so the texture holds the last few seconds of sound as a ring buffer. The mesh is a static grid of UV coordinates uploaded once and never touched again.

The displacement happens in the vertex shader: each vertex reads its own height out of that texture. That is the whole reason a mesh of this size can fly at sixty frames a second on a phone, because the only thing crossing the bus each frame is a single row of bytes rather than a rebuilt geometry.

x = frequency  ·  z = time  ·  y = level in that band

The normal for lighting is taken by sampling the same texture at the neighbouring texels and taking the cross product of the two tangents, so the shading is a real derivative of the spectrum rather than a decoration. The geometries differ only in how that flat grid is bent before displacement: a tunnel wraps the frequency axis into a circle, a sphere maps it to latitude and longitude, a vortex adds a twist proportional to radius.

The sphere needs one guard the others do not. Longitudes converge at the poles, so an undamped displacement there produces a spike where every vertex piles onto the same point. The relief is faded out with smoothstep(0, 0.45, cos(lat)) near each pole.

The doughnut you can hold is not a flat torus. It has curvature: the outside is stretched and the inside is compressed, which is the price of forcing it into three dimensions. The flat torus, the one with no curvature anywhere, is the product of two circles, and it fits exactly on the unit three sphere in four dimensions.

(cos u, sin u, cos v, sin v) / √2

Both circles have the same radius and neither is stretched, which is what flat means. To square it off, each circle is replaced by a superellipse, and one exponent takes you continuously from a circle to a square:

superell(t, m) = sign(cos t, sin t) · |cos t, sin t|^m

At m = 1 that is a circle. As m approaches zero it becomes a square. The product of two squares is a square prism in four dimensions, which is a hypercube, so a single slider takes the shape from a flat torus to a tesseract without ever leaving the surface it started on.

Rotation in four dimensions happens in planes, not about axes. There are six independent planes and two of them are completely independent of each other, so the shape can turn in the xy plane and the zw plane at once, at unrelated speeds. That double rotation is what makes the object appear to pass through itself. The rotation matrix composed from those planes was checked to be an isometry: it preserves length to 6.7e-16.

Two projections bring it back to three dimensions. Perspective divides by distance along w, exactly as a three dimensional perspective divides by distance along z. Stereographic projects from a pole of the sphere:

p = (x, y, z) / (1 − w)

Stereographic projection is only defined from the three sphere, so the point has to be normalized onto it first. Squaring the torus takes it off that sphere, and projecting an un-normalized point sends the geometry off to infinity. Normalizing first is why the squared shape still projects.

Pitch has two properties at once, and a row of bars only shows you one. Height is how high the note sounds and rises without limit. Chroma is which of the twelve names it has and returns to where it started every octave. Shepard drew that as a helix in 1964, and it is the correct shape rather than a stylish one.

angle = 2π · (m mod 12) / 12  ·  height = m / 12

One turn is one octave, so an angle around the axis is the pitch class and the height is which octave. Notes an octave apart sit exactly above one another, which is a thing you can see. On a linear frequency axis an octave is a gap that doubles every time, so the same relationship looks like nothing at all.

The polygon on the floor is the chroma vector: the twelve pitch classes summed over every octave. A chord and all of its inversions collapse to the same figure there, because inversion moves notes by octaves and an octave is a full turn.

The faint lines from the loudest note point at where its own harmonics land. The nth harmonic of a note sits at 12 log₂(n) semitones above it, so the octave is straight up, the fifth steps round by seven semitones and the major third by four. Seeing that pattern hold still while the music moves is the argument for the shape.

An FFT has one resolution across the whole spectrum. Music does not. At the bottom of a bass a semitone is about 3 Hz wide, and at the top of a piano it is about 200 Hz wide, so any single FFT is either far too coarse down there or wasting most of its bins up here. A 4096 point FFT at 48 kHz has bins about 11.7 Hz apart, which is more than a third of an octave at the bottom of a bass guitar.

So the notes are measured with a bank of resonators, one per semitone across seven octaves, each a complex one pole filter tuned to that note and given a bandwidth proportional to its own frequency. That is a constant Q transform: every note gets the same number of cycles rather than the same number of hertz.

y[n] = x[n] + r · e^iω · y[n−1],   r = exp(−π f₀ / (Q · f_s))

The rejection and settling time of that bank were measured rather than assumed: driven with a pure tone, each resonator's neighbours sit far enough down that a semitone is genuinely resolved, and the settling time is what sets how quickly a note can appear.

One thing had to be got right for any of it to work. An AnalyserNode hands you a snapshot whenever you ask, so reading it once per frame gives overlapping windows with gaps and repeats between them. That is fine for drawing a spectrum and completely wrong for a filter bank with state, which needs every sample exactly once and in order. The audio is captured contiguously through an AudioWorklet instead.

Take the amplitude envelope of one frequency band and treat it as a signal in its own right, then take the spectrum of that. It answers a different question from the first transform: not which frequencies are present, but how fast each of them is pulsing.

band → envelope → FFT → modulation rate in Hz

A kick drum at 120 BPM is a ridge at 2 Hz in the low bands. Hi-hats on eighth notes are a separate ridge at 4 Hz somewhere else entirely. The surface is frequency across, modulation rate into the screen, energy as height, and the rails on the floor are round tempos.

The resolution is set by the length of the window, and there is no way around it. The envelope is sampled at 50 Hz and the transform is 512 points long, which is 10.24 seconds of audio, and gives a resolution of about 6 BPM. That is why the mode says how full the window is before it says anything else: a tempo read off a half filled window is a guess.

This is also where a subtle bug lived. Decimating the envelope by taking one sample per audio block makes the envelope's sample rate equal to the block rate rather than the rate you asked for, and every tempo comes out wrong by the ratio between them.

A goniometer plots left against right and tells you about the stereo image, but only ever about this instant. Rotating the axes by 45 degrees gives the mid and side signals, which is the more useful pair:

mid = (L + R) / 2  ·  side = (L − R) / 2

Mid goes up the screen, side goes across, and time runs into the distance, so the history of the image is visible instead of a single blob wobbling. Anything with no stereo content lies flat along the center line, because side is zero everywhere.

The readout is the ratio of the mean absolute side to the sum of both, which is zero for a mono source and rises as the image widens. A phone microphone is very nearly mono, so this mode wants a file rather than the microphone, and it says so rather than drawing a straight line and letting you think that is your music.

Beat reaction comes from spectral flux: the sum of the positive changes in each frequency bin from one analysis window to the next. A sudden broadband increase is an onset. The threshold is adaptive, taken from a running median of recent flux, so it survives a change in level without being retuned.

flux = Σ max(0, |X_t[k]| − |X_t−1[k]|)

Tempo is the strongest periodicity in the intervals between onsets. The detector runs on its own timer rather than inside the render loop, which matters more than it sounds: driven from the frame loop, its time resolution equals the frame interval, so on a slow machine every onset quantises to the frame grid and the tempo comes out of the quantization rather than out of the music.

The effects on top, the feedback trails, the bloom, the chromatic split and the screen fold, are decoration and they are labelled as such. They are applied after the mesh is drawn and they never alter a height. Everything underneath them, every displacement, every radius and every color ramp position, is a measurement of the sound in the room.

The resonator bank finds energy at a note's frequency. It does not know whether that energy is the note being played or the third harmonic of something two octaves down, so a single rich low note lights several bins that a person would hear as one pitch. This is pitch detection's hard problem and nothing here solves it.

The chroma polygon treats every octave as equivalent, which is the point of it, but it also means a bass note and a piccolo playing the same pitch class are indistinguishable there. Look at the helix itself for the difference.

The modulation spectrum needs its full window. Anything shorter than about ten seconds gives a tempo with an error bar wider than the answer, and the mode tells you how full the window is rather than showing you a number it cannot justify.

The microphone path is very nearly mono on a phone, and the browser may apply gain control and noise suppression that this page asks it not to but cannot enforce. Levels from the microphone are relative and comparable within one session, not between sessions or between devices.