The milestone work that shapes modern AI models

A hall of fame of seminal papers that shaped the modern AI landscape; a personal running list. Time runs left to right; the y axis is either the size of each paper's flagship model or how often the paper has been cited, both on a logarithmic scale. Blue is language models, green is vision, orange is robotics, purple is the training and systems work underneath all three. Hover a dot (or tap it on a phone) to see what the paper did and why it mattered; click to pin it.

Model sizes are the flagship model of each paper; several papers never state a count, and those are marked in the panel. Citation counts are a snapshot as of 30 September 2026, not a live figure, and come from Semantic Scholar except NeRF, whose records are split there.

Language

• Attention Is All You Need (2017) — the transformer architecture.
• GPT-2 (2019) and GPT-3 (2020) — zero-shot and then in-context learning out of plain next-token pretraining.
• PaLM (2022) — dense scaling to 540B, and the reasoning jumps that came with it.
• Mamba (2023) — a state space model that scales linearly in sequence length, questioning whether attention is needed at all.
• Also on the plot: BERT, scaling laws, Chinchilla, chain-of-thought, InstructGPT, LLaMA, DeepSeek-R1.

Training and systems

• Adam (2014) — the optimizer nearly everything else on this plot was trained with.
• FlashAttention (2022) — making attention cheap by respecting the memory hierarchy, which is what made long contexts affordable.

Vision

• ViT (2020) — an image as a sequence of patches, which let vision models scale like language models.
• CLIP (2021) and SigLIP (2023) — large-scale visual pretraining supervised by text found on the web rather than by human labels.
• Also on the plot: ResNet, MAE, DINOv2, SAM.

Generative models and 3D

• Auto-Encoding Variational Bayes (2013) — the VAE: a differentiable latent-variable model, and the latent space that later methods generate inside.
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015) — diffusion, five years before it worked at scale — and DDPM (2020), which made it work. Score-based generative modeling through SDEs (2020) then put both under one continuous-time theory, and latent diffusion (2021) moved it into a compressed space and made it cheap.
• NeRF (2020) and 3D Gaussian Splatting (2023) — implicit neural fields, then explicit primitives that render in real time.

Robotics

• RT-1 (2022) and RT-2 (2023) — robot transformers, then vision-language-action models that inherit web knowledge.
• Diffusion Policy (2023) — generating action sequences by denoising, which copes with multimodal demonstrations.
• Also on the plot: SayCan, ACT / ALOHA, Open X-Embodiment, Octo, OpenVLA, pi-0.