rohitg00/ai-engineering-from-scratch is a large AI engineering learning roadmap. Its slogan is “Learn it. Build it. Ship it for others.” The point is not just to read concepts, but to implement AI systems from the foundations upward.
If you already know how to call the OpenAI, Claude, or Gemini APIs, but want to fill in the lower-level knowledge around machine learning, Transformers, training, inference, and engineering, this project is worth bookmarking.
Not Just Another Resource List
Many AI learning repositories are just piles of papers, courses, and blog links. ai-engineering-from-scratch feels more like a course directory: each topic is marked as Learn or Build, and many sections ask you to implement things from scratch in Python.
It covers a wide range:
- Math and machine learning foundations;
- Neural networks and deep learning;
- Computer vision;
- Audio and speech;
- Transformer deep dives;
- Generative AI;
- Reinforcement learning;
- LLMs from scratch;
- Inference optimization;
- AI Agent and production engineering.
This route is not for someone who wants to ship an AI app tonight, but it is a strong fit for people who want to solidify their AI engineering fundamentals.
What Makes the Roadmap Different
One obvious feature is staged progression. For example, the Transformer section starts from the problems with RNNs, then moves into self-attention, multi-head attention, positional encoding, the full Transformer, BERT, GPT, T5, ViT, MoE, KV cache, Flash Attention, scaling laws, and building a Transformer from scratch.
The LLM section is not just about prompts. It covers:
- Tokenizers: BPE, WordPiece, SentencePiece;
- Building a tokenizer from scratch;
- Pre-training data pipelines;
- Pre-training a Mini GPT;
- Distributed training;
- Instruction Tuning;
- RLHF;
- DPO;
- Inference optimization and deployment.
This kind of content is useful for upgrading from “I can use AI APIs” to “I understand why models work this way.”
Why From Scratch Matters
“From scratch” sounds slow, but it has one advantage: you learn what frameworks hide from you.
If you have implemented attention yourself, it becomes easier to understand:
- Why longer context consumes more memory;
- Why KV cache speeds up inference;
- What Flash Attention actually optimizes;
- How RoPE and ALiBi differ in positional encoding;
- Why MoE is not just a simple parameter-count increase;
- Why tokenization affects multilingual performance;
- How fine-tuning, RLHF, and DPO solve different problems.
You may not need these details when simply calling an API. But once you start doing model selection, cost optimization, local deployment, long-context systems, or Agent frameworks, they become very useful.
Who Should Learn from It
This roadmap is better suited to:
- People who already write Python and want to learn AI engineering systematically;
- AI application developers who do not feel grounded in model internals;
- Developers who want to move from API caller to AI Engineer;
- People preparing to study LLMs, inference optimization, or model training;
- Learners who like building while learning instead of only watching videos and reading posts.
It is not ideal for complete beginners. You should have at least Python, basic math, and some machine learning concepts, otherwise it will be easy to get stuck.
How to Start Safely
Do not try to swallow the whole repository at once. A more practical way is to choose a path based on your goal:
- Want to build AI apps: start with LLMs, Agents, inference, and engineering;
- Want to understand models: start from neural networks, Transformers, and LLM from scratch;
- Want to build voice products: look at audio, Whisper, TTS, and voice assistant pipelines;
- Want image generation: look at generative AI, diffusion, Stable Diffusion, and ControlNet;
- Want fundamentals: start from math, machine learning, and deep learning.
It is best to keep a small project for each stage. Reading the table of contents feels good, but the real value comes from actually running a tokenizer, attention module, mini GPT, RAG system, or inference service.
Relationship to Ordinary AI App Development
Many AI apps today do not require training models from scratch. You can use cloud APIs, vector databases, workflow engines, and a few tool calls to build a prototype quickly.
But if you want to go further, you will hit questions like:
- Why is this long-context model so slow?
- Why did RAG retrieve the answer but the model did not use it?
- Why did some capabilities regress after fine-tuning?
- Why does local deployment run out of VRAM?
- Why are Agent tool calls unstable?
- Why do different models with similar parameter counts have such different costs?
At that point, lower-level knowledge is not decoration. It becomes debugging ability.
Summary
ai-engineering-from-scratch is for people who seriously want to strengthen the foundations of AI engineering. It does not promise a shortcut, and it is not merely a bookmark collection. It breaks many core AI modules into stages that can be learned, implemented, and shipped.
If you only build simple AI apps, you do not need to read it end to end. But if you want to understand not only how to call APIs, but also models, training, inference, optimization, and engineering deployment, this repository can serve as a long-term roadmap.