Building an LLM from Zero
So Simple You Can Teach It to Your Kids
Why This Book?
There are already a lot of good resources for learning about AI and large language models. This one tries to find a balance: enough theory to understand what you are building, paired with real, runnable code you can execute on the machine in front of you.
The entire book was written and tested on a five-year-old ThinkPad T14 Gen 1 with no GPU. Training completes in about 20-30 minutes.
About the Book
A step-by-step tutorial for building a GPT-style language model entirely from scratch using Python and PyTorch. By the end, you will have trained a model that generates Shakespeare-like text, writing every line of code yourself.
Every chapter has two parts: a Theory section that explains the concept in plain language with analogies, and a Code section with a runnable Python file you can execute immediately.
Who Is This For?
- Beginners who learn by building (basic Python, no ML background needed)
- Instructors and professors looking for classroom-ready, easy-to-setup demo materials
- IT/IS professionals who want to understand LLMs from the ground up
- Parents who want to teach their kids how AI actually works, hands-on
Quick Facts
| Chapters | 17 |
| Parts | 6 |
| Model size | ~825K parameters |
| Dataset | Tiny Shakespeare |
| Training time | ~20-30 min (CPU) |
| Framework | PyTorch (CPU only) |
| License | MIT / CC BY-NC 4.0 |
Table of Contents
Part I: Setup
| Ch 01 | Setting Up Your Environment |
Part II: Foundations
| Ch 02 | What Is a Language Model? |
| Ch 03 | Tensors and PyTorch Basics |
| Ch 04 | Tokenization |
| Ch 05 | Embeddings |
Part III: The Attention Mechanism
| Ch 06 | Self-Attention |
| Ch 07 | Multi-Head Attention |
| Ch 08 | Feed-Forward and Layer Norm |
Part IV: The Transformer
| Ch 09 | The Transformer Block |
| Ch 10 | The Full GPT Architecture |
| Ch 11 | Causal Language Modeling |
Part V: Training
| Ch 12 | Dataset and DataLoader |
| Ch 13 | The Training Loop |
| Ch 14 | Saving Checkpoints |
Part VI: Generation
| Ch 15 | Greedy Decoding and Sampling |
| Ch 16 | Temperature and Top-k |
| Ch 17 | Putting It All Together |
ROMEO:
What is a man, if his chief good
And market of his time be but to
sleep and feed?