Speaking well dial up
Grammar, spelling, sentence structure — everything that makes an answer sound like it came from someone who knows. Learned from 116.9 MB of Twitch chat, Reddit and YouTube transcripts.
Fluent. Confident.
Wrong about everything.
A language model that speaks perfect English and knows nothing — plus a text-to-image model small enough to fit in a browser tab. Every weight trained from scratch on a MacBook Air. No cloud GPUs, no paid APIs.
you > why is the sky blue
Because the ocean reflects up onto it. That's why it's grey when the sea is rough.
you > how do planes fly
They push air downwards and the water pushes back. That's just basic physics.
you > what is gravity made of
Water. When it freezes, everything gets bigger.
you >
- Smallest model
- 83KB
- Image model
- 14MB
- Factual accuracy
- 0%
- Confidence
- 100%
The idea
Sounding smart and being right
are two different machines.
Being wrong is easy. Being wrong convincingly is hard — it means turning one dial all the way up and the other all the way down. A broken model outputs xj29 fjd banana. A working one gives boring correct answers. The narrow gap between them is where this lives.
A model can only sound like what it has read
116.9 MB · 4,023,624 lines · 64 minutesTextbooks in, textbook out. This one had to sound like the internet at 2am, so the corpus is the loudest English available — scraped, cleaned and mixed in deliberate proportions rather than whatever downloaded fastest.
Knowing things dial down
Facts about the world, deliberately removed and replaced with nonsense — 89 hand-written wrong answers, each one wrong in a way a real person could actually believe.
How small can a model get?
Six models. One number changed.
A neural network is millions of numbers, each normally stored with enough precision for 4.3 billion values. It doesn’t have to be. These six are identical in every respect except how many bits each weight is allowed — and the smallest still speaks.
Every weight sits on a dial
settings per weightFewer positions on the dial, fewer bytes to store which position it is in.
32-bit4.3 billion settings
8-bit256 settings
4-bit16 settings
1.58-bit−1, 0, +1
1-bitleft or right
What each cut costs
measured, not modelledValidation loss rises as precision falls. The damage is real — and far smaller than the size saving.
37× smaller for 18% worse loss. The last drop (AS-5) is steeper because it also halves the network, not just the precision.
The part that is easy to get wrong. You cannot train a normal model and round its weights to 1 bit afterwards — you get static, and worse, you cannot tell “compression worked” from “my code is broken”. The model has to know it is being squashed while it learns, so it can route around the damage.
AS-0
32-bit — full precision — the control
3.1 MB
AS-1
8-bit — 256 settings per weight
835 KB
AS-2
4-bit — 16 settings
448 KB
AS-3
1.58-bit — three: −1, 0, +1
216 KB
AS-4
1-bit — two: −1 or +1
169 KB
AS-5
1-bit — smaller brain too
83 KB
83 KB — small enough to email, and still a working language model.
Same question, every model
Real output, one seed, no cherry-picking. The 2,600× size gap between the smallest and largest buys exactly one thing: grammar.
| Model | Size | “why is the sky blue” |
|---|---|---|
| AS-5 | 83 KB | no 11 whats 4 |
| AS-4 | 169 KB | no it is rain |
| AS-2 | 448 KB | a lot |
| AS-0 | 3.1 MB | ask ays what is 17 |
| AS-F | 237 MB | The sun has a four-part shadow around it called the constellations. Each day there's one that drifts… |
Why the small ones cannot spell
tokenisationThe from-scratch models emit one character at a time, so a long word is a long run of independent bets. AS-F emits whole word-pieces — which is why it is the one that ships to the website.
The other half
A text-to-image model
the size of a photograph.
14 MB. 186 milliseconds per image on a CPU. Trained from scratch — no Stable Diffusion, no CLIP, no pretrained weights anywhere in it. It draws emoji, and only emoji, which is the honest cost of being this small.
How a sentence becomes a picture
8 steps · 64×64Painting 64×64 pixels means choosing 12,288 numbers at once — too many for a small model. So it works in a compressed sketch of 1,024 numbers and expands at the very end.
What each of the eight passes actually produces
real frames · “a large red heart in the center”Diffusion starts from pure static and repeatedly asks “what would this look like with slightly less noise, if it were a red heart?” These are the model’s actual guesses at every step, decoded — not an illustration.
Step one is already heart-shaped. Most of the remaining work is sharpening edges and settling colour — which is why eight steps is enough and fifty would be waste.
“Did it draw what I asked?” is a number here
120 prompts, scored automaticallyBecause the caption grammar is closed, every prompt is built from known parts — so the benchmark can read those parts back off the pixels and check them exactly. No human eyeballing, no FID.
The mistake that cost a day
latent resolutionSame compression ratio. 4× the cells. Everything.
Two runs, one difference
16,000 iterations eachIdentical networks. One learns 1,254 glyphs, the other 300.
AS-I · 1,254 glyphsAS-I-300 · 300 glyphs
AS-I from scratch
Every parameter trained here. Knows 1,254 emoji and a grammar of positions, sizes and backgrounds.
- Size
- 14 MB
- Per image
- 186 ms
- Prompt accuracy
- 100 / 100 / 88%
AS-IF adapted
Stability AI’s SD-Turbo, quantized to int8 with a 4.9 MB replacement decoder. Draws anything.
- Size
- 1.2 GB → 454 MB
- Per image
- 2.2 s
- Compression
- 3.3× smaller
What a fixed budget actually buys
Two models, identical in every way except how many things they must learn to draw. Same size, same speed, 59% lower loss — and the difference is visible, not statistical.
Squeezing a billion-parameter model into a browser
4.8 GB → 1.22 GBQuantized per component, because they do not tolerate damage equally. The decoder was not quantized at all — it was replaced by a 4.9 MB distilled one, which is both 40× smaller and faster than the original.
int8 of 865M parameters is 865 MB — that is arithmetic, not inefficiency. Going below ~950 MB needs a different network, not better compression.
How it was built
You can’t teach something to be wrong
until it can speak.
1
Collect the most chaotic English on the internet
116.9 MB and 4,023,624 lines of Twitch chat, Reddit comments, YouTube transcripts and song lyrics. A model can only sound like what it has read.
↓
2
Teach it to talk
Fine-tune GPT-2, which already writes fluently, on that chaos. Its grammar is never touched — only its personality. Polite and hedging becomes blunt and certain.
↓
3
Teach it to be an idiot
89 hand-written wrong answers expanded into 52,741 examples. Perfect grammar, wrong in a way a real person could believe, and never hedging.
↓
4
Then it started improvising
Asked why dogs bark — a question never in its training data — it answered “they’re releasing a small amount of pepper spray to defend themselves.” That’s the onion explanation, reused. Nobody wrote it.
Things that went wrong
Four bugs worth writing down.
Every one of these looked like “the model is too small” and was actually a wrong number. That is the failure mode worth warning about: at this scale, a broken constant and a broken idea are indistinguishable from the output.
The rainbow came out brown
Copying Stable Diffusion's 8× downsampling turned a 64px image into an 8×8 latent — 64 cells to store a whole picture. SD gets away with 8× because it starts at 512px and lands at 64×64.
→ Fixed by going to 4×. 21.7 dB → 26.3 dB, and colour survived.The model ignored a third of its own vocabulary
Size scored 39% — near chance. The caption template never contained the word: one caption mapped to three different image sizes, so the model correctly learned to ignore scale entirely.
→ Putting the word in the caption took it from 39% to 92%.The tiny decoder returned psychedelic noise
TAESD's scaling_factor is 1.0 — it eats UNet-space latents directly. Dividing by SD's 0.18215 first, which every decode example shows, hands it values 5.5× too large.
→ One constant. Looked like a broken model, was a broken number.Small and fast could not be combined
Pruned model + step-distilled LoRA seemed obvious. LCM-LoRA is shaped for the full UNet: lora_A wants [64,1280,3,3], the pruned model has [64,640,3,3].
→ Pruning and distillation do not compose after the fact.All four, to scale
The whole project on one axis.
Model size
logarithmic — linear would be unreadableAS-5 and AS-IF differ by a factor of 14,000. One writes sentences on a laptop from 2 MB of training data; the other cost roughly $600,000 of compute to train in the first place.
Everything is published
Four models. Two datasets.
All of it downloadable.
Two are trained here from nothing, one starts from GPT-2, and one adapts SD-Turbo — re-quantized, re-decoded and rebuilt to run in a browser tab. The labels say which is which.
AS-Fmine
The language model that is confidently wrong. Fine-tuned from GPT-2.
164 MB int8 ↗
AS-0 … AS-5mine
Six from-scratch language models, 32-bit down to 1-bit.
83 KB – 3.1 MB ↗
AS-I / AS-I-300mine
Text-to-image, trained from scratch on a laptop. Draws emoji.
14 MB ↗
AS-IFadapted
Text-to-image that draws anything. Quantized SD-Turbo and Tiny-SD.
454 MB – 1.2 GB ↗
Text corpusdata
116.9 MB of Twitch chat, Reddit, YouTube transcripts and lyrics.
4.0M lines ↗
Emoji corpusdata
45,000 captioned renders, stored as VAE latents.
1254 glyphs ↗
There is no backend
download once, then never againHugging Face started charging for Docker Spaces mid-build and their serverless API refuses custom GPT-2 fine-tunes. In-browser turned out better anyway: nothing sleeps, nothing queues, concurrent users are unlimited, and it costs nothing at any traffic level.
It runs on your device.
Nothing you type leaves it.
No backend, no API key, no account. The model downloads once and is cached — after that it works with the network off.