Paper Stories

The papers that built modern AI

Plain-English walkthroughs of landmark research — what the paper actually says, why it matters, and how the ideas show up in the roadmaps you're already learning.

⚡Transformer

Attention is All You Need

Ashish Vaswani, Noam Shazeer, et al. · 2017

Replace recurrence with self-attention. Train in parallel, get better quality, unlock every modern LLM, ViT, and multimodal model that followed.

Read story

🌟Scale

Language Models are Few-Shot Learners (GPT-3)

Tom B. Brown, Benjamin Mann, et al. · 2020

Scale a Transformer to 175B parameters, train on 300B tokens. Suddenly it solves new tasks from a few prompt examples alone — no fine-tuning needed. Birth of prompt engineering and the direct ancestor of ChatGPT.

Read story

🚀MoE

DeepSeek-V3 Technical Report

DeepSeek-AI · 2024

A 671B-parameter Mixture of Experts model (only 37B active per token) trained for ~$5.5M to match GPT-4 quality — via Multi-head Latent Attention, FP8 training, and Multi-Token Prediction. Open weights.

Read story

🎚️Fine-Tune

LoRA: Low-Rank Adaptation of Large Language Models

Edward Hu, Yelong Shen, et al. · 2021

Freeze the big model. Add tiny low-rank matrices to each weight. Train only those. Get ~10,000× fewer trainable params, same quality as full fine-tuning, and the ability to swap thousands of fine-tunes from one base model.

Read story

📚RAG

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Patrick Lewis, Ethan Perez, et al. · 2020

Combine a parametric language model (BART) with a non-parametric retriever (DPR) over Wikipedia. The model reads relevant passages before answering — so it gets facts right without storing them all in weights, and you can cite sources.

Read story

More papers added monthly. Have a request? Open an issue on the repo.