Archive for normalizing flow
Congrats, Dr. Andral!
Posted in Books, pictures, Statistics, University life with tags adaptive Monte Carlo algorithm, ENSAE, history of Monte Carlo, importance MCMC, importance sampling, jury, normalizing flow, Paris, PDMP, PhD thesis, PSL Research University, quasi-Monte Carlo methods, thesis defence, Université Paris Dauphine on November 27, 2024 by xi'an
OWAB [season] I
Posted in Statistics, University life with tags ABC, Approximate Bayesian computation, approximate Bayesian inference, Bayesian inference, bluesky, Constrained Monte Carlo, COVID-19, Linkedin, lockdown, manifold, normalizing flow, One World ABC Seminar, One World Approximate Bayesian Inference Seminar, OWABI, Season's greetings, simulation-based inference, twitter, University of Warwick, webinar on October 28, 2024 by xi'an
OWABC is dead, long life OWABI! After 5 seasons of the One World Approximate Bayesian Computation (ABC) Seminar, launched in April 2020 (!) to gather members and disseminate results and innovation during those weeks and months under lockdown, the organisers (incl. yours truly) have now decided to launch a “new” seminar series, the One World Approximate Bayesian Inference (OWABI) Seminar, to better reflect the broader interest and scope of this series, which goes beyond ABC. With Bluesky, X, and Linkedin accountsWith Bluesky, X, and Linkedin accounts. In particular, simulation-based inference and ML related techniques will play a crucial role. We are also pleased to announce that Stefan Radev has joined the OWABI Organiser Team.
The first OWABI talk will be given on Thursday the 31st October at 11am UK time. The speaker is Ullrich Koethe (University of Heidelberg), who will talk about
“Free-form flows for physics-informed generative modeling”
Abstract: The talk first introduces a useful categorization of the (sometimes confusing) “generative model zoo” in terms of different change-of-variables formulas. It then shows how normalizing flows, a major architecture for generative neural networks, can be used for simulation-based Bayesian inference in the sciences. Finally, it proposes free-form flows to simplify the incorporation of physical prior knowledge, e.g. rotation and translation invariance or the restriction of the distribution to a manifold, into generative models.
Keywords: normalizing flows; physical-informed neural networks; simulation-based inference
Mark [a]B[c], plus cats
Posted in Books, Kids, pictures, Statistics, University life with tags ABC, ABC variable selection, Bayesian neural networks, HDR, HPD region, Mark Beaumont, model misspecification, normalizing flow, One World ABC Seminar, philogenic trees, principal components, prior predictive, Saving Wildcats, Scotland, summary statistics, The Guardian, University of Bristol, University of Warwick, webinar, wildcats on June 22, 2024 by xi'an
30 May was a day of first and last times, if not in capital ways (for me), on the One World ABC webinar. This was the first time we had a talk by Mark Beaumont and also the first time I had team experience of facing a smoking participant, while this was the last time of our monthly webinar for the (Northern) academic year.
The talk was about model misspecification in population genomic, from an ABC perspective with the motivation of common noticeable difference between the distributions of d(s,s⁰) and d(s,s’), distances between the prior predictively simulated summary statistics and the observed ones vs posterior generated ones, which should indicates misspecification, esp with complicated models. Mark and his coauthors then supported a gradual elimination of summary statistics to diminish the discrepancy, hence voluntarily impoverishing the model. While blaming the statistics sounded a bit like shooting the messenger, the resolution is of obvious interest if backing from modelling the misspecification itself.
Some of the presented work was conducted in Ward et al (2022, NeurIPS) with a reference to the outlying Ratmann et al (2009) we later discussed, for including tolerance as an extra parameter ε, thereby reconsidering Wikinson’s exact ABC for noisy observations y by the medium of a normalising flow on the marginal distribution of the denoised x (learned from the prior predictive)
More precisely, the idea is to drop summary statistics by checking whether or not the observed S⁰ belongs to HPD region, removing one component of S at a time, using e.g. a k-NN estimate for the summary density (hence depending on parameterisation of said statistics for the distance). Hopefully, the process stops before loosing identifiability by using too few statistics. I also wondered at multiple uses of the data in this sequential procedure but Mark argued for adopting a meta- or pragma- or Gelmanian- Bayesian perspective in the end!
Another perk was the appearance of (and illustration with) the Scottish Wildcat, mentioned in The Guardian a few months ago and discussed in the ‘Og, with further papers exploring more aspects of this hybridization, like a posterior applied to a much more complex phylogenic tree reconstruction for cats of different creeds and many related parameters.
mostly M[ar]C[h]
Posted in Books, Kids, Statistics, University life with tags diffusions, generative modelling, generative models, Monte Carlo methods, Monte Carlo Statistical Methods, mostly Monte Carlo seminar, normalizing flow, optimal transport, optimization, Paris, PariSanté campus, push-forward distribution, seminar, stochastic diffusions, The Prairie Chair on February 27, 2024 by xi'an
combining normalizing flows and QMC
Posted in Books, Kids, Statistics with tags Arianna Rosenbluth, arXiv, Illya Sobol, importance sampling, inverse cdf, John Halton, MCM 2023, Metropolis-Hastings algorithm, mostly Monte Carlo seminar, normalizing flow, Python, quasi-Monte Carlo methods, scrambling, Sobol sequences on January 23, 2024 by xi'an
My PhD student Charly Andral [presented at the mostly Monte Carlo seminar and] arXived a new preprint yesterday, on training a normalizing flow network as an importance sampler (as in Gabrié et al.) or an independent Metropolis proposal, and exploiting its invertibility to call quasi-Monte Carlo low discrepancy sequences to boost its efficiency. (Training the flow is not covered by the paper.) This extends the recent study of He et al. (which was presented at MCM 2023 in Paris) to the normalising flow setting. In the current experiments, the randomized QMC samples are computed using the SciPy package (Roy et al. 2023), where the Sobol’ sequence is based on Joe and Kuo (2008) and on Matouˇsek (1998) for the scrambling, and where the Halton sequence is based on Owen (2017). (No pure QMC was harmed in the process!) The flows are constructed using the package FlowMC. As expected the QMC version brings a significant improvement in the quality of the Monte Carlo approximations, for equivalent computing times, with however a rapid decrease in the efficiency as the dimension of the targetted distribution increases. On the other hand, the architecture of the flow demonstrates little relevance. And the type of RQMC sequence makes a difference, the advantage apparently going to a scrambled Sobol’ sequence.
