The foundational research publications that forged modern artificial intelligence from first principles, authoritative computer science textbooks, and official high-school STEM curricula.
The foundational research breakthroughs that shaped modern artificial intelligence from first principles.
Ashish Vaswani et al. · Google Brain
Displaced recurrent and convolutional layers with stacked multi-head self-attention, unlocking massive training parallelization and launching modern LLMs.
Kaiming He et al. · Microsoft Research
Introduced identity skip connections (y = F(x) + x) solving vanishing gradients in ultra-deep networks, making 152+ layer training possible for the first time.
Edward J. Hu et al. · Microsoft
Froze pre-trained weights and injected low-rank decomposition matrices (ΔW = B·A), reducing trainable parameters by 10,000x with zero inference latency.
Jonathan Ho et al. · UC Berkeley
Bridged non-equilibrium thermodynamics and score-matching into a tractable variational lower bound, turning pure Gaussian noise into photorealistic image synthesis.
Diederik Kingma & Jimmy Ba · OpenAI / UofT
Combined exponentially decaying running averages of past gradients (momentum) and squared gradients (AdaGrad/RMSProp) for robust non-convex descent.
Tri Dao et al. · Stanford University
Leveraged GPU memory hierarchy by computing exact softmax in fast on-chip SRAM via tiling, avoiding costly HBM memory transfers for 2-4x end-to-end speedups.
Definitive canon textbooks with deep mathematical foundations, official open-access editions, and physical printings.
Ian Goodfellow, Yoshua Bengio, Aaron Courville
Widely known as the 'Deep Learning Bible'. Begins with linear algebra and numerical optimization, then thoroughly builds convolutional networks, sequence models, and deep generative models.
Christopher M. Bishop
A rigorous, mathematically exact masterpiece. Formulates machine learning through Bayesian probability theory, maximum likelihood, kernel methods, variational inference, and graphical models.
Richard S. Sutton & Andrew G. Barto
Written by the pioneers of RL. Provides deep intuitive and formal derivations for multi-armed bandits, finite MDPs, Bellman equations, Monte Carlo methods, TD learning, and policy gradients.
Martin Kleppmann
The undisputed classic for understanding the systems layer beneath AI. Examines LSM-trees, B-trees, data encoding, replication topologies, distributed consensus (Paxos/Raft), and stream processing.
Simon J.D. Prince
A modern tour de force that translates abstract matrix transformations into intuitive 2D/3D geometric visualisations. Covers loss landscapes, self-attention, diffusion models, and alignment.
Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong
Splits machine learning into two halves: Part I lays out linear algebra, analytic geometry, matrix decompositions, vector calculus, and probability. Part II shows how these construct PCA and SVMs.
