Archive for Henri Poincaré

BayesComp 2025.3

Posted in pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , on June 20, 2025 by xi'an

The second day of the conference started with a cooler and less humid weather (although this did not last!), although my brain felt a wee bit foggy from a lack of sleep (and I almost crashed while running on the hotel treadmill, at 14.5km/h!), and the plenary talk of my friend of many years Sylvia Früwirth-Schnatter on horseshoe priors and time-varying time series (à la West). With a nice closed-form representation involving hypergeometric functions of the second kind (my favourite!), with the addition of a triple-Gamma prior. Sylvia stressed on the enormous impact of the prior choice on change-point detection, which was already the point in the original horseshoe paper (as opposed to George’s Lasso prior). Without incorporating any specific modelling on potential change-point, fair enough given that the parameter is moving with time, unhindered. Her MCMC choices involved discrete parameters with Negative Binomial and Poisson parameters, allowing for partially integrated or collapsed solutions. Possibly further improved by Swendsen-Wang steps.

I then attended the (advanced) Langevin session after agonising upon my choice for a wealth of options! Sam Power presented a talk linking simulation with optimisation targets, over measure spaces. With Wasserstein gradient flow algorithms that resemble Langevin algorithms once discretised by a particle system. (A natural resolution producing a somewhat unnatural form of measure estimator since made of Dirac masses, from which very little can be learned.) Then [my Warwick colleague & coauthor] Any Wang on underdamped Langevin diffusions. when Poincaré‘s inequality fails, but convergence (in total variation) still occurs. Followed by Peter Whalley on splitting methods (where random hypergeometric subsampling dominates Robbins-Monro) and stochastic gradient algorithms, in a connected (to the previous talks) way since involving underdamped aspects. (With a personal discovery of Polyak’s heavy ball method.)

The afternoon session saw me facing a terrible dilemma with three close friends talking at the same time! Eventually opting for PDMPs, over simulation-based inference and recalibration for approximate Bayesian methods. Kengo Kamatani gave a general introduction to PDMPs, before explaining the automated implementation he considered with Charly Andral (during Charly’s visit to ISM, Tokyo, two summers ago). Towards accelerating the generation of the jump time. Then Luke Hardcastle applied PDMPs for survival prediction, using spike & slab priors and sticky PDMPs. And Jere Koskela (formerly Warwick) extended zig-zag sampling to discrete settings (incl. Kingman’s coalescent.)

The (rather long) day was not over yet since we had planned an extra on-site OWABI seminar & webinar with two participants in the conference, Filippo Pagani (Warwick and OCEAN postdoc) using fusion for federated learning, with a trapezoidal approximation, and Maurizio Filippone on GANs as hidden perfect ABC model selection, a GAN providing an automatic density estimator… With astounding Gemini-generated cartoons! Videos are soon to be available. A big congrats to the speakers who managed to convey their ideas and results despite the late hour! (On the extra-academic side, I was invited last night to a genuine Szechuan dinner in Chinatown, with a large array of spicy dishes if not that spicy!, and a rare opportunity to taste abalone. And bullfrogs. Quite a treat! And a good reason to skip dinner altogether!)

Philosophies, Puzzles and Paradoxes [book review]

Posted in Books, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , on May 25, 2024 by xi'an

Yudi Pawitan and Youngjo Lee have written a book that recently caught my attention within the CRC Press list of new publications. Because philosophy, puzzles, and paradoxes are definitely of interest to me (as shown by numerous entries in the ‘Og!). The subtitle of said book is A Statistician’s Search for Truth.

Reviews of the book are already available, with for instance Andrew Gelman stating that he disagrees “with much of this book, but it’s an entertaining and thought-provoking introduction to some challenging questions” or Stephen Senn starting the foreword with “This is a remarkable book: wide-ranging, ambitious, challenging and profound but also intriguing, fascinating and original.” (Senn is also cited within the book for his discussion of our revisit of Harold Jeffreys’ Theory of Probability.) Nice cover as well (albeit I could not trace the origin of it, inside or outside the book.)

The book is made of three parts, one on the philosophical approaches to truth, scientific discovery, deduction, and induction, a second one on probability theories, with philosophical motivations, Bayesian inference, and likelihood-based inference, and a third section on paradoxes. Given that both authors are senior authors who have contributed to likelihood inference throughout their career, incl. the books In All Likelihood and Generalized Linear Models with Random Effects, the likelihood approach is somewhat privileged against other statistical resolutions towards the resolution of the paradoxes, with a defence of confidence distributions and a chapter on epistemic confidence that mostly stems from recent papers by the authors, like Pawitan et al.  (2023) and Lee and Lee (2023). I find the discussion therein somewhat unclear, esp. because the same notation Pr(.) is employed for different probability notions.

“Epistemic confidence is the objective measure of uncertainty that’s attached to single events, where the objectivity is based on a consensus of rational minds.” (p.197)

The philosophy part is following the (European) Enlightenment in producing more and more involved discussions on reason, knowledge and scientific discovery. This exploration is an easy read, as it does not delve particularly deeply in the arguments of Kant, Hume, or Popper. With the apparently unescapable mention of Gödel’s incompleteness theorem, including a sausage citation from Poincaré that reminded of that strip from Tintin in America:which, most probably, he would have applied to Ais! Several sections about pseudo-rational attempts to demonstrate the existence of Dog could have been skipped as well.

The part of probability already considers paradoxes which, like the subsequent ones are mostly the consequence of using natural (and hence ambiguous) languages instead of mathematical descriptions—incl. the statement of the Likelihood Principle. It also discusses Keynes’ logical (or imprecise) probabilities, briefly if appropriately given the pessimistic views of young Keynes on the assessment of the probability of an event. Savage is privileged enough to enjoy an entire chapter discussing his 1950’s axioms leading to the existence of a (subjective)  prior on “the states of the world”. This is followed by a chapter on Inverse probability (aka Bayesian statistics), where the authors consider Bayes’ 1763 Essay to have stayed mostly unnoticed till  the beginning of the 20th Century, which sounds a somewhat subjective judgement. (And as uncovered by Steve Stiegler, the original title of the Essay was indeed intended as a reply to Hume.) A further if short chapter is dedicated to the search for the prior distribution. Which thus gives the misguided impression that there should exist such a thing, rather than acknowledging that Bayesian statements are relative to the prior measure. The remainder of the discussion on invariant and reference priors is however mostly standard. Except when falling for the marginalisation paradox when stating that a product of improper priors implies independence on p.144.

The paradoxes examined in the final part are Allais’ (an alumni of Lycée Lakanal!), and Ellsberg’s, avatars of the Saint Petersburg paradox and referring to failing to adhere to rational decision-making and not in the least to statistics. Conjunction and inclusion “fallacious fallacies”, which are central to Kahneman’s Thinking fast and slow bestseller, and attributed to reasoning in terms of likelihood rather than of probability (without accounting for multiple testing on p.228). A whole if short chapter on the Monty Hall and three prisoners paradoxes, another predictable occurrence in a book on reasoning paradoxes. Again mostly a matter of poor wording, plus relying on the choice of an underlying probability model, for which the authors again follow a likelihood approach, the number of the prize door or of the freed prisoner being the parameter. Kyburg’s (very weak) lottery paradox and related forensic paradoxes, concluding with the rejection of judgements based solely on probability reasoning. Hempel’s paradox of the ravens, a priori unrelated with statistical evidence, but turned into one by squeezing in some sampling models. Finishing with the (envelope) exchange paradox, where the authors refuse to put a prior on the unknown parameter but end up with a solution equivalent to adopting a Jeffreys prior.

In conclusion, this attempt at connecting statistical inference and philosophy, probability concepts and rational decision making, paradoxes and modelling, within a single book is academically sound and overall enjoyable, if not outstanding or remarkable as it does not constitute a radical move away from existing analyses of those classical paradoxes. Furthermore, I find the paradoxes overwhelmingly distant from genuine statistical settings and involving a rather stretched notion of data. Still, methinks I will keep this book in my bookcase, rather than leaving it for the taking in the department coffee room!

As I was completing the book and getting towards writing this book review, I also noticed a two page blurb in Significance (May 2024 issue) written by the authors on their book. (which happens rather frequently with this magazine). Unsurprisingly, the contents provd mostly extracted from the preface and introduction With a nice ravens picture (in conjunction with the raven paradox).

[Disclaimer about potential self-plagiarism: this post or an edited version may eventually appear in my Books Review section in CHANCE.]

probabilistic numerics [book review]

Posted in Books, pictures, Statistics, Travel with tags , , , , , , , , , , , , , , , , , , , , on July 28, 2023 by xi'an

Probabilistic numerics: Computation as machine learning is a 2022 book by Philipp Henning, Michael Osborne, and Hans Kersting that was sent to me by CUP (upon my request and almost free of charge, as I had to pay custom charges, thanks to Brexit!). With the important message of bringing statistical tools to numerics. I remember Persi Diaconis calling for (such) actions in the 1980’s (and even reading a paper of his on the topic along with George Casella in Ithaca while waiting for his car to get serviced!).

From a purely aesthetic view point, the book reads well, offers a beautiful cover and sells for a quite reasonable price for an academic book. Plus it is associated with a website containing draft version of the book. Code, links to courses, research, conferences are also available there. Just a side remark that it enjoys very wide margins that may have encouraged an inflation of footnotes (but also exercises). Except when formulas get in the way (as e.g. on p.40).

The figure below is an excerpt from the introduction that sets the scene of probabilistic numerics involving algorithms as agents, gathering data and making decisions, with an obvious analogy with standard Bayesian decision theory. Modelling uncertainty missing from the picture (if not from the book, as explained later by the authors as an argument against attaching the label Bayesian to the field). Also referring to Henri Poincaré for the origination of the prior vs posterior uncertainty about a mathematical quantity. Followed by early works from the Russian school of probability, somewhat ignored until the machine-learning revolution and a 2012 NIPS workshop organised by the authors. (I participated to a follow-up workshop at NIPS 2015.)

In this nicely written section, I have an objection to the authors’ argument that a frequentist, as opposed to a Bayesian, “has the loss function in mind from the outset” (p.9), since the loss function is logically inseparable from the prior and considered from the onset. I also like very much the conclusion to that introduction, namely that the main messages (from the book) are that (verbatim)

  • classical methods are probabilist (p.10)
  • numerical methods are autonomous agents (p.11)
  • numerics should not be random (if not a rejection of the concept of Monte Carlo methods, p.1, but probabilistic numerics being opposed to stochastic numerics, p.67)
  • numerics must report calibrated uncertainty (p.12)
  • imprecise computation is to be embraced (p.12)
  • probabilistic numerics consolidates numerical computation and statistical inference (p.13)
  • probabilistic numerical algorithms are already adding value (p.13)
  • pipelines of computation demand harmonisation

“Is it still reasonable to labour under computational constraints conceived in the 1940s?” (p.113)

“rather than being equally good for any number of dimensions, Monte Carlo is perhaps better thought of as being equally bad” (p.110)

Chapter I is a 40p infodump (!) on mathematical concepts needed for the following parts. Chapter II is about integration, opposing again PN and Monte Carlo (with strange remark that MCMC does not achieve √N convergence rate, p.72). In the sense that the later is frequentist in that it does not use a prior [unless considering a limiting improper version as in Section 12.2, an intriguing concept in this setup as I wonder whether or not improper priors can at all be contemplated] on the object of interest and hence that the stochasticity does not reflect uncertainty but rather the impact of the simulated sample. Advocating Bayesian quadrature (with some weird convergence graphs exhibiting a high variability with the number of iterations that apparently is not discussed) and bringing in the fascinating perspective of model choice in that framework (leading to compute a posterior probability for each model!). Being evidently biased towards Monte Carlo, I find the opposition in Chapter 12 unnecessarily antagonistic, while presenting Monte Carlo methods as a form of minimax solution, the more because quasi-Monte Carlo methods are hardly discussed (or dismissed). As illustrated by the following picture (p.115) and the above quotes. (And I won’t even go into the absurdity of §12.3 trashing pseudo-random generators as “painfully dumb”.)

Chapter III is a sort of dual of Chapter II for linear algebra numerics, primarily solving linear equations by Gaussian solvers, which introduces new concepts like Krylov sequences, although it sounds quite specific (for an outsider like me). Chapters IV and V deal with the more ambitious prospect of optimisation. Reconsidering classics and expanding into Bayesian optimisation, using Gaussian process priors and defining specific loss functions. Bringing in a strong link with machine learning tools and goals. [citation typo on p.277]. Chapter VII addresses the resolution of ODEs by a Bayesian state space model representation and (again!) Gaussian processes. Reaching to mentioning inverse problems and offering a short finale on prospective steps for interested readers.

[Disclaimer about potential self-plagiarism: this post or an edited version will eventually appear in my Books Review section in CHANCE.]

in defense of subjectivity [sound the gong]

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , , , on October 13, 2022 by xi'an

When browsing the IMS Bulletin [01 October] a few days ago, I saw that Ruobin Gong (from Rutgers) had written a tribune about Subjectivism. In response to [IMS President] Krysz Burdzy’s presidential address at the IMS Meeting in London a few months earlier. Address that I had missed and where he was calling for the end of the term subjective in statistics… (While ironically attending the Bayesian conference in Montréal!) Given the tone of his Search for Certainty book, which Andrew and Larry and I discussed a while ago, I am not at all surprised by another go at Bayesian statistics, but I will not indulge into another response, since Krysz found my earlier review “venomous”! Especially since Ruobin has produced a deeply argument ed and academically grounded criticism of the presidential address (which, if I may mention it, sounds rather rambling away from statistics). In particular, Ruobin introduces Objectivity³ as “an interpreted characterization of the scientific object”, which reminds me of Nietzsche’s aphorism about physics. And where personal and collegial inputs are plusses, even though they could be qualified to be “subjective”. This was also Poincaré’s argument for Bayesian reasoning. In conclusion, I think that the London call to cease using the term in statistics was neither timely (as the subjective-versus-objective debate has sort of dried out) nor appropriate (in that it clashed with the views of part of the IMS community).

a journal of the plague year² [not there yet]

Posted in Books, Kids, pictures, Travel, University life, Wines with tags , , , , , , , , , , , , , , , , on November 27, 2021 by xi'an

Returned to Warwick once more, with “traffic-as-usual” at Charles de Gaulle airport, including a single border officer for the entire terminal, a short-timed fright that I actually needed a PCR test on top of my vaccine certificate to embark, due to wrong signage, a one-hour delay at departure due to foggy conditions in B’ham, and another ½ hour delay at arrival due to a shortage of staff and hence no exit stairs available! And got a tense return to B’ham as the taxi line in Warwick had vanished!

Read the first novel of P. Djèlí-Clark A Master of Djinn after reading a series of short stories and novellas of his, taking place in the same fantastic Cairo of the early 1900’s. This was enjoyable, mostly, again thanks to well-constructed characters (apart from the arch-villain) and the appeal of the magical Cairo imagined by the author. I did not feel the appearances of Raymond Poincaré or von Birsmark were really needed, though. Also kindled A history of what comes next, by Sylvain Neuvel, which I got as a free (Tor) book. Which is an interesting take on the space race, with a pair of (super-)women behind the entire thing. And a lot of connections to the actual history. I somehow got tired in the middle, even though I finished the book during my commuting to and from Warwick.

Watched within a week My Name, a dark Korean TV drama,  as I found it very good and rather original (albeit with some similarities with the excellent Jeju-based Night in Paradise). The storyline is one of a young woman, Ji Woo, seeking revenge on her father’s killer, by joining the criminal gang her father was part of and infiltrating the police (not really  a spoiler!). At the beginning, this sounded like gang glorification, hence rather unappealing, but soon things proved to be quite different from how they appeared first. The scenario is of course most unrealistic, especially the (brutal and gory) fights where the heroine takes down endless rows of gang members and where the participants almost always recover from knife injuries that should have been fatal or at least permanently damaging. And the ineffectiveness of the police in stopping the drug dealers. However, when watched as a theatrical performance, the main characters in My Name, most especially Ji Woo, are well-constructed and ambiguous enough to make this descent into darkness worth watching. (Given the conclusion of the series, I cannot imagine a second season being made.) Also had a short go at Night Teeth, which proved a complete waste of time!