A new episode of our mostly Monte Carlo seminar, very soon coming near you (if in Paris):
On Friday 20/02/26, from 3-5pm at PariSanté Campus
15h: Paul Mangold (École Polytechnique, Palaiseau)
Convergence and Linear Speed-Up in Stochastic Federated Learning
In federated learning, multiple users collaboratively train a machine learning model without sharing local data. To reduce communication, users perform multiple local stochastic gradient steps that are then aggregated by a central server. However, due to data heterogeneity, local training introduces bias. In this talk, I will present a novel interpretation of the Federated Averaging algorithm, establishing its convergence to a stationary distribution. By analyzing this distribution, we show that the bias consists of two components: one due to heterogeneity and another due to gradient stochasticity. I will then extend this analysis to the Scaffold algorithm, demonstrating that it effectively mitigates heterogeneity bias but not stochasticity bias. Finally, we show that both algorithms achieve linear speed-up in the number of agents, a key property in federated stochastic optimization.
16h: Alain Durmus (École Polytechnique, Palaiseau)
A Mixture-based Framework for Guiding Diffusion Models
Inverse problems—such as image restoration from noisy or incomplete measurements and musical source separation—are ill-posed, making Bayesian approaches with learned generative priors especially appealing. Diffusion models provide powerful priors, but existing posterior sampling methods often rely on crude likelihood-gradient approximations and heavy task-specific tuning. In this talk, I will introduce a novel principled approach specifically designed to overcome these limitations. The core contribution of this approach is the construction of a mixture approximation of intermediate posterior distributions defined by the diffusion model. The sampling is carried out sequentially via Gibbs sampling, a Markov Chain Monte Carlo method, using a careful data augmentation scheme. Gibbs sampling is employed here due to its simplicity and theoretical guarantees, allowing for exact conditional updates at each iteration, thus ensuring stability and efficiency. One key advantage of the presented algorithm is its flexibility: it adapts to varying levels of computational resources by adjusting the number of Gibbs iterations. Consequently, substantial performance gains can be achieved by increasing inference-time computational effort. I will present extensive experimental results demonstrating empirical performance across diverse image restoration tasks, involving both pixel-space and latent-space diffusion models, and showcase its successful application in musical source separation.
