HippoGPT Writes Stories on the ZX Spectrum 128K

In an October 25 article, the creator of HippoGPT described a tiny language model that writes stories on a standard ZX Spectrum 128K. It was trained on TinyStories, a dataset of simple English-language stories for young children, and 1,500 generated stories about the Spectrum, tapes, loading errors, and Dizzy.

The model has about 120,000 parameters, and its weights are compressed to four bits. To speed up computations on the Spectrum, the creator used lookup tables: instead of some multiplications, the program looks up precomputed values. The model generates about one token every four seconds, or about eight seconds per word.

According to the creator, a coherent and meaningful story is produced roughly one time in ten. HippoGPT can be run in a browser or downloaded for the ZX Spectrum; the creator has posted the source code, trained models, and build and training instructions on GitHub.

Image: Matěj Orlický CC0 · License

Translation editor: Салтанов А.