Skip to content
Krypteia Sec
KrypteiaSec Academy

Build a tiny LLM from scratch

10 complete chapters: tokens, embeddings, attention, transformers, training and generation. Each chapter includes a full write-up, worked calculations, Python code, a quiz, a PDF and a narrated walkthrough. Free to read and download, with no account required.

This rebuilt course replaces the former Academy catalog. The labs use Python and PyTorch on your computer, not a browser simulation. Install the requirements once; then run the supplied code and data offline. A CPU is enough for the examples. The course does not award the retired certification.

Code, Markdown and PDFs on GitHubDownload the complete course
LLM Fundamentals: Build a Tiny LLM From Scratch
  1. Chapter 1 of 10

    Tokens: teaching a computer to read

    12.2 minute walkthrough / full chapter / tested lab

  2. Chapter 2 of 10

    Embeddings: giving each token a vector of meaning

    11.1 minute walkthrough / full chapter / tested lab

  3. Chapter 3 of 10

    The bigram model: your first real predictor

    11 minute walkthrough / full chapter / tested lab

  4. Chapter 4 of 10

    Self attention: the idea that changed everything

    10.6 minute walkthrough / full chapter / tested lab

  5. Chapter 5 of 10

    Multi head attention: many perspectives at once

    11.3 minute walkthrough / full chapter / tested lab

  6. Chapter 6 of 10

    The transformer block: residuals, layer norm, and an MLP

    11.2 minute walkthrough / full chapter / tested lab

  7. Chapter 7 of 10

    Assemble the GPT: stacking blocks into a model

    12.1 minute walkthrough / full chapter / tested lab

  8. Chapter 8 of 10

    The training loop: how a model actually learns

    12.4 minute walkthrough / full chapter / tested lab

  9. Chapter 9 of 10

    Generation: sampling text from your model

    10.7 minute walkthrough / full chapter / tested lab

  10. Chapter 10 of 10

    Scaling, and what comes next

    12 minute walkthrough / full chapter / tested lab