-
What optimization is actually used for
A tour of the fields that run on mathematical optimization, and in each case the specific thing being minimised, the structure that makes it hard, and the method that gets used.
-
Where optimization actually lives in modern AI
A map of the optimization problems inside modern machine learning and generative AI. What is being minimised, which method solves it, why that method and not another, and which parts are still badly solved.
-
Continuous and discrete optimization: a working map
The two halves of mathematical optimization, what separates them, which methods belong where, how to tell which kind of problem you actually have, and where the two meet.
-
When every experiment has a price
Bayesian optimization is usually benchmarked in iterations. Laboratories do not run on iterations. What changes when you count cost instead.
-
Nys-Fed: making second-order federated learning fit on an edge device
Newton-type methods converge in far fewer communication rounds than FedAvg, and nobody uses them, because a Hessian for a 25M-parameter model does not fit on a client. Here is what a Nyström sketch changes, with numbers.