Archive for BayesComp 2023
off to Singapore (BayesComp 2025)
Posted in pictures, Statistics, Travel, University life with tags AF256, BayesComp 2023, BayesComp 2025, Bayesian computational methods, conference, flight mode, Indian food, ISBA, Malay food, NUS, Singapore on June 16, 2025 by xi'an
gentle importance sampling
Posted in Books, pictures, Statistics with tags BayesComp 2023, Cédric Villani, Comptes Rendus de l'Académie des Sciences, CRAS, George Casella, harmonic mean estimator, Hyvärinen score, importance sampling, infinite variance estimators, Levi, noise contrasting estimation, Pierre Louis Lions, survey on February 24, 2025 by xi'an
A new (and gentle!) survey by Luca Martino! And by Fernando Llorente. On importance sampling, with coverage of normalised and self-normalised versions. And their usage in different configurations (one vs several integrals, one vs several families of distributions). Some points relating to earlier remarks or musing of mine’s:
- the fact that the optimal importance function does not lead to a zero variance importance estimator when the integrand f is not of constant sign (p.7) can be cancelled by first decomposing f as f⁺-f⁻, since both allow for a zero variance importance estimator, if formally requiring two different samples (of size zero!), a trick considered later on p.18 and repeated for the ratio in self-normalised importance (p.19)
- the special case when the integrand f is constant is not of practical interest but relevant for checking properties of different estimators. For instance, this case allowed George and myself to spot a mistake in an early importance paper. In the same volume of the Comptes Rendus as an early paper of Lions and Villani.
- the remark that self-normalised (SNIS) importance sampling can prove more efficient than (properly normalised) importance sampling, although the property that SNIS is always bounded should not be seen as a major point given that it is simply due to using a finite sample and hence a finite set of images of f
- the case of integrals involving several target pdfs or several integrands is not necessarily of major interest if simulating different samples for each unidimensional integral can be implemented (again formally leading to zero variance at no cost)
- the issue of merging several estimators in an optimal way is briefly mentioned in §5.4, a challenge Victor Elvira and I have been approaching over the past years, if not yet concluding satisfactorily (mea culpa)
- when replacing the target with a noisy estimate (p.22), the fact that this estimate must be normalised is correct, but pales against the impact of using this estimate, which may prove catastrophic. And unbiasedness is not particularly crucially important in this setup for the same reason
- the section on evidence approximation (§7) is more standard, with the harmonic mean estimator being called reverse importance sampling, which brings us to the “elephant in the room”, namely that
- the issue of infinite variance of some importance sampling estimators is not directly covered (except once in §8, p.34), thus perceiving importance sampling as a variance reduction method being somewhat misleading (unless the authors consider solely the optimal importance function, which is rarely of practical use)
The paper concludes with an interesting notion that
“we suggest the analysis of the relevant connection between importance sampling and contrastive learning Gutmann and Hyvärinen (2012)”
that I also have been pointing out for a while. All in all, a useful summing-up that I will likely suggest to my students.
Lapparadish
Posted in Mountains, pictures, Running, Travel with tags 68⁰ N, Alaskan huskies, Arctic hare, Arizona, Baltic salmon, Balto, BayesComp 2023, Central Park, conference, cross-country skiing, dog race, downhill skiing, Finland, Lapland, Levi, reindeer, Sammi on April 3, 2023 by xi'an
On top of BayesComp 2023 being rich and exciting, spending a week above the Arctic circle, by about 68⁰ North was most pleasurable. If we did not really see Northern Lights/auroras, as the above was taken by one of us with a long exposure shot on a particularly cold evening, rather than the diaphanous veils we could hardly perceive, cold and snow and associated activities were delivered without reservation!
On the day we left, due to a late flight departure. we had an exhilarating 10k ride with three teams of Alaskan huskies, over a frozen river, following a musher. The strength of these dogs was amazing, esp. since they were not fully delivering, being able to reach 25km/hour in races over massive distances. On top, our guide delivered another and more realistic version of the story of Balto, whose musher was Finnish, and which we had watched many times with our kids (although we missed his statue in Central Park!)
And I X country skied every early morning instead of running, over groomed trails and in all weathers, with a minimum (manageable) -24⁰ one morn. As a near neophyte in the game, I eventually developed a hip inflammation but the joy of going through the woods prior to sunrise and spotting the occasional Arctic hare was worth the 15 hours of it. Despite hard frozen glasses the coldest morns. By comparison, the afternoon break when I tried downhill skiing was less exciting even though the very dry snow there was enjoyable.
Cold dry weather also re-awoke a nose bleed proclivity that I had forgotten since Arizona. Again, no big deal. And the conference dinner took place in a Sammi restaurant that proved quite the exception to my skip-the-conference-dinner rule with its reindeer and salmon dishes!
kuva Lapista⁴ [jatp]
Posted in Statistics with tags Arctic Chill, BayesComp 2023, cross-country skiing, Finland, jatp, Lapland, Levi on March 18, 2023 by xi'an
kuva Lapista¹
Posted in Mountains, pictures, Running, Travel with tags 68⁰ N, BayesComp 2023, cross-country skiing, Finland, Lapland, Levi, MCMSki, outdoor, tundra on March 13, 2023 by xi'an
