AXIOM: Foundations of Efficient Deep Learning
Abstract
Deep learning theory is beginning to uncover quantitative regularities that explain and predict the behavior of large-scale learning systems, including scaling laws, compute-optimal training prescriptions, and predictable optimization dynamics. At the same time, the growing computational, memory, and energy demands of modern AI have made efficiency a central challenge for the field. Yet a substantial gap remains between theoretical understanding and practical efficiency: many efficient AI methods are developed empirically, while existing theories rarely provide actionable principles for designing learning systems under resource constraints. AXIOM: Foundations of Efficient Deep Learning brings together researchers from deep learning theory, optimization, machine learning systems, and efficient AI to investigate how theoretical insights can guide the design of efficient learning systems and which aspects of efficient AI can be predicted rather than discovered through costly experimentation. Topics include scaling laws and capability prediction, resource-constrained optimization and generalization, sparsity and compression, modularity and adaptive computation, efficient foundation models, hardware-aware learning, and theoretical limits of efficient AI. The workshop combines invited vision talks, contributed papers, posters, and a community-driven Grand Challenges initiative focused on identifying key open questions and future directions for the foundations of efficient AI, with outcomes synthesized into a community position paper outlining a research agenda for theory-guided efficient AI.