Building an LLM from Zero
So Simple You Can Teach It to Your Kids
About the Book
This is a step-by-step tutorial for building a GPT-style language model entirely from scratch using Python and PyTorch. By the end, you will have trained a model that generates Shakespeare-like text, writing every line of code yourself.
Every chapter has two parts: a Theory section that explains the concept in plain language with analogies, and a Code section with a runnable Python file you can execute immediately.
The entire book was written and tested on a ThinkPad T14 Gen 1 (Intel Core i7, 32 GB RAM, 2020). Training completes in about 20-30 minutes. No cloud account, no GPU, no special setup needed.
Who Is This For?
- Beginners who learn by building (basic Python, no ML background needed)
- Instructors and professors looking for classroom-ready, easy-to-setup demo materials
- IT/IS professionals who want to understand LLMs from the ground up
- Parents who want to teach their kids how AI actually works, hands-on
Quick Facts
| Chapters | 16 |
| Sections | 5 |
| Model size | ~825K parameters |
| Dataset | Tiny Shakespeare |
| Training time | ~20-30 min (CPU) |
| Framework | PyTorch (CPU only) |
| License | MIT (free) |
Table of Contents
Section 1: Foundations
| Ch 01 | What Is a Language Model? |
| Ch 02 | Tensors and PyTorch Basics |
| Ch 03 | Tokenization |
| Ch 04 | Embeddings |
Section 2: The Attention Mechanism
| Ch 05 | Self-Attention |
| Ch 06 | Multi-Head Attention |
| Ch 07 | Feed-Forward and Layer Norm |
Section 3: The Transformer
| Ch 08 | The Transformer Block |
| Ch 09 | The Full GPT Architecture |
| Ch 10 | Causal Language Modeling |
Section 4: Training
| Ch 11 | Dataset and DataLoader |
| Ch 12 | The Training Loop |
| Ch 13 | Saving Checkpoints |
Section 5: Generation
| Ch 14 | Greedy Decoding and Sampling |
| Ch 15 | Temperature and Top-k |
| Ch 16 | Putting It All Together |
ROMEO:
What is a man, if his chief good
And market of his time be but to
sleep and feed?
How to Cite
If you use this book in your teaching or research, please cite it as:
A DOI for permanent academic citation is available via Zenodo (link will be added after first release).