rohitg00/ai-engineering-from-scratch is a set of open source courses that extends from mathematical foundations to LLM, AI Agent, multi-Agent, production deployment and security. It is not just a list of linked resources, but the learning content is divided into 20 stages and 503 lessons. Each lesson tries to leave code, prompts, skills, agents or MCP servers that can be run or reused.
If you already know how to call the model API, but don’t know the relationship between tokenizer, attention, agent loop, MCP and production deployment, this course is suitable for completing the knowledge chain. The total content is about 320 hours. There is no need to learn it all in order from the first section. It is more effective to locate your starting point first.
Quick Answer
Beginners can start with the basics of environment, mathematics, and machine learning; developers with backend experience can start with Transformer, LLM Engineering, or Tools & Protocols; users who are already using Codex and Claude Code can start directly from Agent Engineering, Infrastructure, and Capstone Projects.
The most practical way to start is to first clone the repository, run /find-your-level Skill, generate a personal learning route based on ten questions, and then choose a stage to complete the closed loop of “read principles, write code, run tests, and retain artifacts”.
How to divide the 20 stages of the course
The course can be roughly divided into five sections:
| learning period | stage | Problems suitable to be solved |
|---|---|---|
| Base | Setup, Math, ML, Deep Learning | Complete the basics of environment, mathematics and neural networks |
| Model | Vision, NLP, Speech, Transformer, GenAI | Understand different modalities and generative models |
| LLM | LLM from Scratch, LLM Engineering, Multimodal | Transition from model principles to engineering applications |
| Agent | Tools & Protocols, Agent, Multi-Agent | Learn tool invocation, MCP, agent loop, and collaboration |
| Production | Infrastructure, Safety, Capstone | Deployment, monitoring, security and complete projects |
Don’t think that just because the course numbers start from 0, you have to complete the previous math before you can watch Agent. A more reasonable approach is to first select the stages required for the current task, and then make up for the gaps in concepts when you encounter them.
Clone and run the first example
First make sure that Git and Python are installed on your computer:
|
|
Clone the repository:
|
|
The first example given in the official README is linear algebra vector code:
|
|
If on Windows python points to the Microsoft Store or the interpreter cannot be found, you can try:
|
|
The table of contents for each lesson typically contains:
|
|
Don’t just read docs when you’re studying. Run code at least once, and then check whether outputs can be used in your workflow.
Install the Agent Skills that come with the course
The repository provides Skills for Claude, Cursor, Codex, OpenClaw and Hermes. The official installation entrance is:
|
|
Windows If you don’t have the python3 command, you can use:
|
|
Before installing, open the script and confirm where it will copy the files:
|
|
Do not run third-party installation scripts with administrator privileges without checking the path. Skills are usually just Markdown directives and helper scripts, but helper scripts may still read and write files or execute commands.
Use /find-your-level to find a starting point for learning
After installing Skills, you can run it in an Agent that supports slash commands:
|
|
It evaluates the basics through ten questions and then gives recommended stages and time estimates. After completing a stage, you can run:
|
|
3 here means checking phase 3. It is suitable for situations where you find that “the code can run, but the concept is not really grasped”.
If the current client does not automatically recognize the slash command, you can directly open the corresponding SKILL.md and hand over the task requirements to the Agent for execution. The Skill search directories of different clients are not exactly the same, and the documentation and local configuration of the current version of the client should prevail.
Check the course completion status first, don’t treat the Roadmap as a table of contents
ROADMAP.md in the root directory of the repository is not just about future plans, but also marks each lesson as completed, in progress, or planned. Officially, three states are used:
|
|
Before preparing to learn a certain stage, search the status first:
|
|
View only the stages near Agent, MCP, and production deployments:
|
|
This can avoid following the macro route in the README to find a certain section, only to find that the code or handouts are still under construction. ROADMAP.md gives a total estimate of about 314 hours, while the README uses a generalized value of about 320 hours; they are both course plans, not a hard time to complete the course.
How should a course catalog be accepted?
Each class emphasizes Build It / Use It / Ship It, and the acceptance can be divided into three levels.
1. Build It: Understand the minimal implementation without relying on the framework
First run the basic implementation provided in the course and confirm that you can interpret the input, main data structures and output. For example, when learning Agent loop, you should not just remember the name of ReAct, but you should be able to point out:
- where conversation history is saved;
- How to distribute the model after returning the tool call;
- How tool results are re-contextualized;
- How to stop when the maximum number of steps is reached;
- Which exceptions should be handled by the Agent and which ones should simply fail.
2. Use It: Use common libraries to complete the same problem
After running through the basic version, compare the implementations of PyTorch, scikit-learn or the corresponding SDK. The point is not to make the code shorter, but to identify which steps the framework handles for you, such as tensor devices, gradients, batching, serialization, and retries.
3. Ship It: Leave artifacts that can be reused
Check the class outputs/:
|
|
The final product should at least: have clear input, not succeed silently on failure, not write keys to the repository, and be reproducible in a new test directory.
Windows environment recommendations
The course spans Python, TypeScript, Rust, and Julia, and it is not recommended to install all toolchains initially. First create an isolation environment for the current stage.
Python courses are available with:
|
|
If PowerShell blocks activation scripts, you can adjust the policy only for the current process:
|
|
Do not use Administrator PowerShell directly, and do not turn off the execution policy globally in order to run the course. When a certain course requires additional dependencies, the README, dependency files or code import items of the course directory will be read first, and then the corresponding packages will be installed.
Record the version before running, making it easier to reproduce when you encounter problems:
|
|
When the corresponding language stage has not been learned, the absence of a certain command does not mean that the repository is damaged.
How to pick a shortest path from a 503 section
Don’t just choose by stage name, but also work backwards from the final product:
| Target | It is recommended to learn first | Can be skipped for now |
|---|---|---|
| Call model API for application | APIs & Keys, LLM Engineering, Structured Outputs | Implement vision models from scratch |
| Make RAG | Embeddings, Chunking, Retrieval, Evaluation | GAN, reinforcement learning basics |
| Write MCP Server | Tools & Protocols, API, authentication, error handling | Complete pre-training process |
| Be a Coding Agent | Agent loop, Tool use, Memory, Evaluation | Most vision courses |
| Local deployment model | GPU Setup, Quantization, Inference, Infrastructure | Multiple Agent Swarm |
| System Learning Model Principle | Math, ML, Deep Learning, Transformer, LLM from Scratch | It is not recommended to skip the basic stage |
Select 8–15 sections per path first to make up the first round, rather than generating plans that cover hundreds of sections at a time. After completion, use /check-understanding to find the gap.
One-week learning workflow example
Here is a rhythm suitable for working developers:
|
|
Only one working product is required to be delivered each week. For GPU training, dataset download, or cloud resource steps, estimate the cost and time first to avoid launching only to find out later that you can’t complete it.
How to change course skills to your own version
Do not directly modify the original Skill in the cloned repository and then use it for a long time, otherwise git pull will easily conflict in the future. Copy it to your own Skills directory and adjust:
- Modify
nameto avoid duplication of names with the original Skill; - Narrow the trigger conditions to only cover the scenarios you actually use;
- Replace the course example path with the actual project path;
- Orders that expressly allow and prohibit;
- Add verification steps such as test, lint or
git diff --check; - Put it into the global directory after the test repository is run.
When upgrading the course repository, first check the differences:
|
|
Confirm that there are no unexpected permission changes in the installation script and Skills before performing the update.
Common failures and positioning sequence
Sample code missing module
First confirm the virtual environment and current directory, and then check whether the course has dependent files:
|
|
Don’t see ModuleNotFoundError and install a series of packages continuously in the global environment.
Test passes but code is not understood
Try removing a critical step, changing the input size, or creating incorrect inputs. If you cannot predict where it will fail, it means that you are just running the code and have not completed the course goal.
Agent Skill does not appear after installation
Check whether the Skill directory is nested one level deeper, whether the front matter is valid, whether the client needs to be restarted, and whether the Skill is only designed for another client. Directly inputting the content of SKILL.md as a task can help distinguish between “there is a problem with the Skill content” and “the client did not find the file”.
Three recommended learning routes
Route 1: Systematic learning of AI for the first time
It is recommended to start with Setup, Math, ML, and Deep Learning, and then move into Transformer and LLM. In the mathematics stage, there is no need to pursue mastering it all at once. The focus is to be able to explain what matrices, gradients, probability and loss functions do in the code.
Route 2: Can write applications and want to switch to LLM Engineering
You can start with Transformer, GenAI, LLM Engineering and then learn Tools & Protocols, Agent Engineering and Infrastructure. After each section, try to change the output into a small tool that can be used in your own projects.
Route 3: Already using Codex or Claude Code
Prioritize Agent loop, tool invocation, MCP, memory, multi-Agent, production deployment and security. Don’t just copy the skill; also examine the problem it solves, the triggers, permission boundaries, and how to recover from failure.
FAQ
Do all 503 courses have to be completed?
unnecessary. Section 503 instructions cover a lot of ground, and that doesn’t mean everyone should learn them from beginning to end. Complete a route related to your current job first, and then gradually build up the foundation.
Can I just download PDF or EPUB?
Can. The official Release is available in six volumes in PDF and EPUB. However, the repository is a continuously updated version and contains code, tests and skills; just reading the e-book will lose a lot of practical content.
Does running the code require a GPU?
Basic math, Python, agent loops, etc. generally don’t rely on GPUs. When it comes to deep learning training, local models, or multi-modality, you need to prepare CUDA, cloud GPU, or longer run times based on course requirements.
Recommended usage
Start by executing /find-your-level, choose a route, and then complete only one verifiable goal each week, such as implementing a tokenizer, running a minimal Agent loop, or installing and auditing a Skill. This is easier to create reusability than collecting 503 links.
Why Build AI Engineering from Scratch
rohitg00/ai-engineering-from-scratch is a large AI engineering learning roadmap. Its slogan is “Learn it. Build it. Ship it for others.” The point is not just to read concepts, but to implement AI systems from the foundations upward.
If you already know how to call the OpenAI, Claude, or Gemini APIs, but want to fill in the lower-level knowledge around machine learning, Transformers, training, inference, and engineering, this project is worth bookmarking.
Not Just Another Resource List
Many AI learning repositories are just piles of papers, courses, and blog links. ai-engineering-from-scratch feels more like a course directory: each topic is marked as Learn or Build, and many sections ask you to implement things from scratch in Python.
It covers a wide range:
- Math and machine learning foundations;
- Neural networks and deep learning;
- Computer vision;
- Audio and speech;
- Transformer deep dives;
- Generative AI;
- Reinforcement learning;
- LLMs from scratch;
- Inference optimization;
- AI Agent and production engineering.
This route is not for someone who wants to ship an AI app tonight, but it is a strong fit for people who want to solidify their AI engineering fundamentals.
What Makes the Roadmap Different
One obvious feature is staged progression. For example, the Transformer section starts from the problems with RNNs, then moves into self-attention, multi-head attention, positional encoding, the full Transformer, BERT, GPT, T5, ViT, MoE, KV cache, Flash Attention, scaling laws, and building a Transformer from scratch.
The LLM section is not just about prompts. It covers:
- Tokenizers: BPE, WordPiece, SentencePiece;
- Building a tokenizer from scratch;
- Pre-training data pipelines;
- Pre-training a Mini GPT;
- Distributed training;
- Instruction Tuning;
- RLHF;
- DPO;
- Inference optimization and deployment.
This kind of content is useful for upgrading from “I can use AI APIs” to “I understand why models work this way.”
Why From Scratch Matters
“From scratch” sounds slow, but it has one advantage: you learn what frameworks hide from you.
If you have implemented attention yourself, it becomes easier to understand:
- Why longer context consumes more memory;
- Why KV cache speeds up inference;
- What Flash Attention actually optimizes;
- How RoPE and ALiBi differ in positional encoding;
- Why MoE is not just a simple parameter-count increase;
- Why tokenization affects multilingual performance;
- How fine-tuning, RLHF, and DPO solve different problems.
You may not need these details when simply calling an API. But once you start doing model selection, cost optimization, local deployment, long-context systems, or Agent frameworks, they become very useful.
Who Should Learn from It
This roadmap is better suited to:
- People who already write Python and want to learn AI engineering systematically;
- AI application developers who do not feel grounded in model internals;
- Developers who want to move from API caller to AI Engineer;
- People preparing to study LLMs, inference optimization, or model training;
- Learners who like building while learning instead of only watching videos and reading posts.
It is not ideal for complete beginners. You should have at least Python, basic math, and some machine learning concepts, otherwise it will be easy to get stuck.
How to Start Safely
Do not try to swallow the whole repository at once. A more practical way is to choose a path based on your goal:
- Want to build AI apps: start with LLMs, Agents, inference, and engineering;
- Want to understand models: start from neural networks, Transformers, and LLM from scratch;
- Want to build voice products: look at audio, Whisper, TTS, and voice assistant pipelines;
- Want image generation: look at generative AI, diffusion, Stable Diffusion, and ControlNet;
- Want fundamentals: start from math, machine learning, and deep learning.
It is best to keep a small project for each stage. Reading the table of contents feels good, but the real value comes from actually running a tokenizer, attention module, mini GPT, RAG system, or inference service.
Relationship to Ordinary AI App Development
Many AI apps today do not require training models from scratch. You can use cloud APIs, vector databases, workflow engines, and a few tool calls to build a prototype quickly.
But if you want to go further, you will hit questions like:
- Why is this long-context model so slow?
- Why did RAG retrieve the answer but the model did not use it?
- Why did some capabilities regress after fine-tuning?
- Why does local deployment run out of VRAM?
- Why are Agent tool calls unstable?
- Why do different models with similar parameter counts have such different costs?
At that point, lower-level knowledge is not decoration. It becomes debugging ability.