Language models can kind of do math in their heads. How does this work? Let's find out. But first, we need to understand even how do numbers get represented as they are fed into the transformer. That is, how are numbers tokenized?
Originally published on Substack.
tl;dr: i built a webpage that shows you where all the flash space in your firmware project has gone.
Elfvis has entered the building:
I was listening to Dwarkesh's podcast and he referenced some interesting-sounding Jane Street problems. I figured I'd give one a shot.
The problem: you have a neural network's weights, but the layers are scrambled. You need to put Humpty Dumpty back together.