Archive for Metropolis algorithm

j-ISBA Blackwell-Rosenbluth Award²⁶ [call reposted]

Posted in Statistics with tags , , , , , , , , , , , on May 15, 2026 by xi'an

Call for award nominations

Dear all,

It is with great pleasure that we announce the Blackwell-Rosenbluth Award by j-ISBA, a recently established award for junior researchers in different areas of Bayesian statistics. The award aims at recognizing outstanding junior Bayesian researchers based on their overall contribution to the field and to the community. There will be six winners in total who will be invited to present their work in two special events of the Junior Bayes Beyond the Borders (JB^3) webinar series and receive three years of free ISBA and j-ISBA membership.

ISBA proudly has a wide geographical diversity among its members. To encourage scientific exchange and strengthen research connections between geographies, three prizes will be awarded to researchers based in time zones UTC+0 to UTC+13 [e.g. Africa + Asia + Europe + Oceania] and three to those based in UTC-12 to UTC-1 [e.g. North America + South America]. We welcome nominations of junior researchers working in the broad spectrum of topics in Bayesian statistics, including but not limited to methods, theory, computation, machine learning, data science, biostatistics, econometrics, industrial statistics, environmental science, and software.

There will be two scientific committees: one representing UTC- and the other representing UTC+, each consisting of members from their respective regions based on their professional affiliations. These committees are tasked with evaluating candidates for the award. The UTC- committee will evaluate submissions from UTC+ and vice versa.

Why Blackwell-Rosenbluth

The award is named after David H. Blackwell and Arianna W. Rosenbluth for their groundbreaking works that lie at the foundation of modern Bayesian statistical theory and computation. They represent important role models for new researchers in Bayesian statistics.

David Harold Blackwell

Young Blackwell Born on April 24, 1919, Blackwell excelled in mathematics from an early age. He earned his doctoral degree from the University of Illinois at Urbana-Champaign under the supervision of Joseph L. Doob in 1941. He had a distinguished career, becoming a founding member in 1955 of the newly established Department of Statistics at University California, Berkeley. In 1965, he became the first African American to be elected member of the U.S. National Academy of Sciences and was awarded the John von Neumann Theory Prize in 1979. In addition to his seminal contributions to Bayesian inference, decision theory, game theory, sequential analysis and renewal theory, he also wrote one of the first textbooks in Bayesian statistics (Basic Statistics, McGraw-Hill, 1969).

Arianna Wright Rosenbluth

Young Rosenbluth Born on September 15, 1927, Arianna Wright Rosenbluth showed an affinity for sciences from early childhood. She completed her doctoral work under the supervision of a future Nobel Laureate, John Van Vleck, in 1949, making her the fifth woman to earn a Ph.D. in Physics from Harvard. Later, as a coauthor of the seminal 1953 paper introducing the Metropolis algorithm, Rosenbluth almost single-handedly implemented the algorithm on the MANIAC I hardware at the Los Alamos Scientific Laboratory. This made her the first person to ever implement the Markov chain Monte Carlo method when sophisticated programming tools were still years away, and the program had to be written in strings of 1’s and 0’s.

Eligibility and Application Procedure

Ph.D. students or early career researchers who obtained their PhD after January 1, 2021 are eligible for nomination. Candidates who were nominated in previous years may be nominated again if they received their Ph.D. after January 1, 2021. In exceptional cases, applicants who are more than five years past their Ph.D. may still be considered if they experienced a significant career break within five years of earning their degree (such as breaks due to illness, caring for a sick family member, pregnancy-related leave, or parental leave). Candidates may inquire about their eligibility, particularly if they have taken career breaks, by sending an email to jisba.section@gmail.com. A nomination may come from any ISBA member, including the nominee themselves. A nomination is to be submitted electronically and should contain:

  • Nominating letter in support of the candidate
  • CV of the candidate
  • One manuscript or alternate form of exposition (e.g. software documentation) of scientific work most representative of the nominee’s achievements; the submitted work should also be available as publication or in a public repository such as arXiv, bioRxiv, CRAN, Bioconductor or GitHub.

Timeline

Nominations for the 2026 award edition are now open! Deadline to submit is July 12, 2026

Leave a comment »

another first

Posted in Statistics with tags , , , , , , , , on July 1, 2022 by xi'an

A question related to the earlier post on the first importance sampling in print, about the fist Markov chain Monte Carlo in print. Again uncovered by Charly, a 1973 Chemical Physics paper by Patey and Valleau, the latter inventing umbrella sampling with Torrie at about the same time. (In a 1972 paper in the same journal with Card, Valleau uses Metropolis Monte Carlo. While Hastings, also at the University of Toronto uses Markov chain sampling.)

Leave a comment »

mathematical theory of Bayesian statistics [book review]

Posted in Books, Statistics, Travel, University life with tags , , , , , , , , , , on May 6, 2021 by xi'an

I came by chance (and not by CHANCE) upon this 2018 CRC Press book by Sumio Watanabe and ordered it myself to gather which material it really covered. As the back-cover blurb was not particularly clear and the title sounded quite general. After reading it, I found out that this is a mathematical treatise on some aspects of Bayesian information criteria, in particular on the Widely Applicable Information Criterion (WAIC) that was introduced by the author in 2010. The result is a rather technical and highly focussed book with little motivation or intuition surrounding the mathematical results, which may make the reading arduous for readers. Some background on mathematical statistics and Bayesian inference is clearly preferable and the book cannot be used as a textbook for most audiences, as opposed to eg An Introduction to Bayesian Analysis by J.K. Ghosh et al. or even more to Principles of Uncertainty by J. Kadane. In connection with this remark the exercises found in the book are closer to the delivery of additional material than to textbook-style exercises.

“posterior distributions are often far from any normal distribution, showing that Bayesian estimation gives the more accurate inference than other estimation methods.”

The overall setting is one where both the sampling and the prior distributions are different from respective “true” distributions. Requiring a tool to assess the discrepancy when utilising a specific pair of such distributions. Especially when the posterior distribution cannot be approximated by a Normal distribution. (Lindley’s paradox makes an interesting incognito incursion on p.238.) The WAIC is supported for the determination of the “true” model, in opposition to AIC and DIC, incl. on a mixture example that reminded me of our eight versions of DIC paper. In the “Basic Bayesian Theory” chapter (§3), the “basic theorem of Bayesian statistics” (p.85) states that the various losses related with WAIC can be expressed as second-order Taylor expansions of some cumulant generating functions, with order o(n⁻¹), “even if the posterior distribution cannot be approximated by any normal distribution” (p.87). With the intuition that

“if a log density ratio function has a relatively finite variance then the generalization loss, the cross validation loss, the training loss and WAIC have the same asymptotic behaviors.”

Obviously, these “basic” aspects should come as a surprise to a fair percentage of Bayesians (in the sense of not being particularly basic). Myself included. Chapter 4 exposes why, for regular models, the posterior distribution accumulates in an ε neighbourhood of the optimal parameter at a speed O(n2/5). With the normalised partition function being of order n-d/2 in the neighbourhood and exponentially negligible outside. A consequence of this regular asymptotic theory is that all above losses are asymptotically equivalent to the negative log likelihood plus similar order n⁻¹ terms that can be ordered. Chapters 5 and 6 deal with “standard” [the likelihood ratio is a multi-index power of the parameter ω] and general posterior distributions that can be written as mixtures of standard distributions,  with expressions of the above losses in terms of new universal constants. Again, a rather remote concern of mine. The book also includes a chapter (§7) on MCMC, with a rather involved proof that a Metropolis algorithm satisfies detailed balance (p.210). The Gibbs sampling section contains an extensive example on a two-dimensional two-component unit-variance Normal mixture, with an unusual perspective on the posterior, which is considered as “singular” when the true means are close. (Label switching or the absence thereof is not mentioned.) In terms of approximating the normalising constant (or free energy), the only method discussed there is path sampling, with a cryptic remark about harmonic mean estimators (not identified as such). In a final knapsack chapter (§9),  Bayes factors (confusedly denoted as L(x)) are shown to be most powerful tests in a Bayesian sense when comparing hypotheses without prior weights on said hypotheses, while posterior probability ratios are the natural statistics for comparing models with prior weights on said models. (With Lindley’s paradox making another appearance, still incognito!) And a  notion of phase transition for hyperparameters is introduced, with the meaning of a radical change of behaviour at a critical value of said hyperparameter. For instance, for a simple normal- mixture outlier model, the critical value of the Beta hyperparameter is α=2. Which is a wee bit of a surprise when considering Rousseau and Mengersen (2011) since their bound for consistency was α=d/2.

In conclusion, this is quite an original perspective on Bayesian models, covering the somewhat unusual (and potentially controversial) issue of misspecified priors and centered on the use of information criteria. I find the book could have benefited from further editing as I noticed many typos and somewhat unusual sentences (at least unusual to me).

[Disclaimer about potential self-plagiarism: this post or an edited version should eventually appear in my Books Review section in CHANCE.]

14 Comments »