17 Module 3 · Training and Generation
Chapter 17 of 17 Run it in your browser

Chapter 17: Putting It All Together

Figure 17.1: Where we are: completing the full pipeline from text to language model.

You have built a language model from zero. Step by step, you wrote the code to tokenize text, embed it into vectors, apply self-attention, stack transformer blocks, train the weights using gradient descent, and generate new text with temperature and top-k sampling. In this final chapter you will:

• Run the complete end-to-end pipeline in a single script.
• See the full architecture in action.
• Understand how your model relates to modern, production-grade LLMs.

Words to Know

• End to End: a process that runs every step on its own, from raw text to a trained model and its generated text.
• Instruction tuning: An extra round of training, after the kind done in this book, that teaches the model to answer questions and follow instructions.
• RLHF: Reinforcement Learning from Human Feedback, a way to train a model on people's feedback so it answers politely and as people prefer.

Theory

The Full Architecture

Take a look back at everything you built. This is not a "toy" architecture. You wrote the same building blocks used by GPT-2, which is the architectural foundation of GPT-3, GPT-4, and many other modern Large Language Models (LLMs).

Figure 17.2: The complete system: from raw text to a trained language model.

The differences between your model and a massive production model are mostly a matter of scale:

• Vocabularies: We used 65 characters, while they use 50,000+ subword tokens.
• Size: We gave each token a list of 128 numbers (a 128-dimensional embedding) and stacked 4 layers, while they use lists of thousands of numbers and nearly a hundred layers.
• Data: We trained on 1 megabyte of Shakespeare, while they train on terabytes of internet text.
• Hardware: We trained for a few minutes on a CPU, while they train for months on thousands of specialized GPUs.

How Real Models Differ

While the fundamental transformer architecture (embeddings, attention, blocks, training) remains the same, modern models add refinements to optimize performance:

• Instead of simple position lookups, they might use Rotary Positional Embeddings, which help models understand relative distances between words better.
• Instead of standard multi-head attention, they might use Grouped-Query Attention, which saves memory and speeds up text generation.
• After pretraining (what we did), they undergo Instruction Tuning and RLHF (Reinforcement Learning from Human Feedback) to learn how to answer questions politely rather than just predicting the next word.

The end-to-end pipeline connects every stage on the "where we are" map, from text ingestion to generation, wrapping the entire system into one continuous flow.

In Business

For our house-style assistant, this end-to-end script represents the full product lifecycle. In a business environment, you run this pipeline whenever the company archive changes. That means training a new model overnight on updated documents, validating it, and deploying the new checkpoint to serve your marketing and compliance teams.

Code

We have combined every piece of code from the previous chapters into one master script. It downloads the data, tokenizes it, creates the DataLoaders, builds the 825,000-parameter model, trains it for 3,000 steps, saves a checkpoint, and generates a sample text.

Figure 17.3: The full script executes every step in sequence without manual intervention.

Now we run the whole script once to check that every stage works (programmers call this a smoke test).

Terminal

$ python src/ch16_full_pipeline.py
[1/7] Downloading dataset...
...
[2/7] Tokenizing...
...
[3/7] Creating DataLoaders...
...
[4/7] Building model...
...
[5/7] Training for 3000 steps...
...
step     1 | train: 4.2726 | val: 4.1660 | elapsed: 2s | ETA: 6148s
...
step  3000 | train: 1.7323 | val: 1.7411 | elapsed: 474s | ETA: 0s
...
[6/7] Saving checkpoint...
...
[7/7] Generating text...
...
GENERATED TEXT (temperature=0.8, top_k=40):
...
ROMEO:
Praviour soul to shall that that are the and not,

What just happened:

• The model trained successfully and loss steadily decreased.
• It generated brand new, Shakespeare-like text based on the patterns it learned.
• While it makes some logical or grammatical mistakes, the character names, sentence structures, and vocabulary strongly mimic the training data.

Terminal

$ python diagrams/charts/ch17_generated_text.py
Saved chart: ch17-generated-text.png

Figure 17.4: The model generates new text.

Shape Check

Table 17.1 lists the parameter count of the fully assembled network.

Table 17.1: Final parameter count of the completed model.

Tensor	Shape	What it means
model parameters	824,832	The total number of weights the model learned during training.

Try It

Try It

Try the final model the way a real service would use it, by loading the saved file model_final.pt and answering a user's prompt. The repository already has a script that does this:

# ...
checkpoint_path = os.path.join(
    os.path.dirname(__file__), "..", "..", "checkpoints", "model_final.pt"
)

if not os.path.exists(checkpoint_path):
    # Fall back to model.pt if model_final.pt does not exist
    checkpoint_path = os.path.join(
        os.path.dirname(__file__), "..", "..", "checkpoints", "model.pt"
    )

checkpoint = torch.load(
    checkpoint_path, map_location="cpu", weights_only=False
)
# ...

Lines 6 and 8 check if the final checkpoint exists and fall back to a previous save if it doesn't.

Terminal

$ python src/examples/ch17_deploy.py
--- Prompt: 'USER: What is the news?\n' ---
USER: What is the news?

FROKENTER:
And with is there?

GLOUCEMIO:
All But, there come from the prainion; and every,
As do p

Key Takeaways

• You built the complete GPT architecture from scratch.
• The fundamental components (tokenization, embeddings, self-attention, transformer blocks, and gradient descent) are the core of all modern LLMs.
• Larger models mostly scale up these same components with more data, more parameters, and better hardware.
• Refinements like RLHF and instruction tuning are applied after this pretraining process.

Check Your Understanding

1. What is the difference between the model we built and GPT-2?
2. Why does the model output sometimes contain spelling or logic errors despite being fully trained?
3. What is the purpose of an end-to-end pipeline in a business setting?

Assignment

Assignment 17: Ask a new question

About 15 minutes. You change one word and write no new code. Work in this chapter's Colab notebook or on your own laptop.

1. Run it. Run src/examples/ch17_deploy.py as it is. Copy the prompt and everything the model writes after it.
2. Change one word. Find the line prompt = "USER: What is the news?\n". The prompt is the text the model continues. Change the word news to any other English word, in plain letters only, and leave the rest of the line as it is. Run the script again.
3. Compare. Does the model answer either question? What does it write instead? What do the two replies have in common?
4. Explain it. In two or three sentences a manager could follow, say why this model continues your text in the shape of a play, and what extra training a real assistant gets so that it answers.

Hand in: both prompts with their replies, your answers to step 3, and your sentences.

Further Reading

Where prompting came from. Even a pre-trained model normally had to be fine-tuned, with fresh labeled examples and a training run, before it could do a new job. At 175 billion parameters the authors found something different: write two or three examples into the prompt and the model follows the pattern, with no weights changed at all. That behavior is what people now call prompting, and it is why a language model became something you talk to rather than something you retrain.

The step that turns a text predictor into an assistant. A model trained to continue text is not the same thing as a model that does what you ask; it will happily continue your question with more questions. The authors collected human demonstrations of good answers and human rankings of competing answers, and fine-tuned on that feedback. The result matters for how you read this book: a 1.3 billion parameter model trained this way was preferred by people to the 175 billion parameter model it came from. Capability and helpfulness are different problems, and this book builds the first one.

Why the field started building bigger. Before this, deciding how large to make a model, how much text to train it on, and how much compute to spend was guesswork. The paper measured all three and found the error falls along smooth, predictable curves across a very wide range of sizes. That turned model building into a budgeting exercise, and it is the reason the industry spent the following years scaling up. It also explains the ceiling on the model you train here: a few hundred thousand parameters and a few hundred thousand characters of Shakespeare buy a certain quality of text, and no more.

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., ... Amodei, D. (2020). Language models are few-shot learners (arXiv:2005.14165). arXiv. https://doi.org/10.48550/arXiv.2005.14165

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020). Scaling laws for neural language models (arXiv:2001.08361). arXiv. https://doi.org/10.48550/arXiv.2001.08361

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback (arXiv:2203.02155). arXiv. https://doi.org/10.48550/arXiv.2203.02155

Note. The Tiny Shakespeare text used throughout the book comes from Andrej Karpathy's char-rnn project: https://github.com/karpathy/char-rnn{ .note }

Next up: You're done! Go build something amazing.