Archive for Scotland

6th Workshop on Sequential Monte Carlo Methods (#2)

Posted in Mountains, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , on June 5, 2024 by xi'an

Managed to get back from the Pentland hills in time for the Wednesday afternoon session, which proved most interesting as close to my research interests!

Nicola Branchini presented his work with Victor Elvira (a close friend and coauthor, incidentally one of the organisers of the workshop!) on improving self normalised importance sampling by interpreting it as a ratio of estimators based on two samples (which may be the same) and attempting to optimise the joint distribution of said sample. The starting assumption is having (good) marginal importance functions, which means the goal here is in optimising a copula distribution targeting the ratio as quantity of interest. Optimality is however defined in terms of the approximate asymptotic variance of the ratio, which remains an approximation. The idea is nonetheless quite interesting and shows potential for connecting with bridge sampling and… AMIS! As an aside, the talk considered cases when the margins are multivariate, which requires a généralisation of Sklar’s theorem. Simo Särkä then demonstrated how highly parallel processors like GPUs can accommodate Bayesian filters and smoothers in state space models not requiring simulation, gaining a reduction in complexity from O(T) to O(log T). I had not really thought of parallel processing in the recent years, hence was quite pleased at hearing this resolution based on so-called associative scans, and see that implementations were already available in Julia/CUDA.

This was followed by a highly enjoyable poster session, including chats about ABC-SMC for discovery rates, infinite dimensional diffusions, Pareto smoothed importance samplings, &tc with posters by Hugo Marival (coauthor of our importance Monte Carlo recent paper) and Shreya Roy (a student at U of Warwick). With sunny views of Arthur’s Seat (and plenty of people at the top), contrary to the above! Followed by a private party dinner occupying half of a nearby and novel South Indian restaurant that proved quite tasty, local and definitely enjoyable.

For my last morning in town, albeit it was unrelated to the posted abstract, Pierre Del Moral spoke about noisy versions of the ensemble Kalman filter on linear diffusions that allowed for stable solutions under strong enough conditions, encompassing an impressive corpus of work over the past ten years. Alex Beskos presented antithetic multilevel methods for diffusions, which allow to improve the error in the discretisation, even though I did not fully get the whole idea (partly due to dozing out from time to time, a consequence of my last early rounds of Arthur’s Seat in the very early morn).

Daniel Paulin presented a novel unbiased method based on kinetic Langevin dynamics that combines advanced splitting methods with enhanced gradients, avoiding Metropolis correction by coupling and multilevel Monte Carlo approach, achieving unbiasedness by telescoping, but involving an avalanche of acronyms in the leapfrog/Gibbs steps.  And Adam Johansen (U of Warwick) on several recent papers of their divide-and-conquer filtering methods, introduced in a 2017 JCGS paper, following a decomposition of the state variable into low-dimensional components like branches and leaves of a tree.

6th Workshop on Sequential Monte Carlo Methods

Posted in Mountains, pictures, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , , , on May 16, 2024 by xi'an

Very glad to be back to an SMC workshop as it has been nine years since my attending SMC 2015 in Malakoff! The more for the workshop taking place in Edinburgh and at the Bayes Centre. It is one of these places where I feel somewhat returning to familiar grounds with accumulated memories. Like my last visit there when I had a tea with Mike Titterington…

The overall pace of the workshop was quite nice, with long breaks for informal discussions (and time for ‘oggin’!) and interesting poster late afternoons, helped by the small number of them at each instance, incl. one on reversible jump HMC. Here are a few scribbled entries about some talks along the first two days.

After my opening talk (!), Joaquín Míguez talked about the impact of a sequential (Euler-Marayama) discretisation scheme for stochastic differential equations on Bayesian filtering with control of the approximation effect. Axel Finke (in a joint work with Adrien Corenflos, now an ERC Ocean postdoc in Warwick) built a sequence of particle filter algorithms targeting good performances (high expected jumping distance) against both large dimensions and high time horizon, exploiting gradient shift MALA-like, as well as prior impact, with the conclusion that their jack-of-all-trades solutions, Particle­-MALA and Particle­-mGRAD, enjoyed this resistance in nearly normal models. Interesting reminder of the auxiliary particle trick and good insights on using the smoothing target, even when accounting for the computing time, but too many versions for a single talk without checking against the preprint.

The SMC sampler-like algorithm involves propagating N “seed” particles z(i), with a mutation mechanism consisting of the generation of N integrator snippets 𝗓:=(z,ψ⁢(z),ψ²⁢(z),…) started at every seed particle z(i), resulting in N×(T+1) particles which are then whittled down to a set of N seed particles using a standard resampling scheme. Andrieu et al., 2024

Christophe Andrieu talked about Monte Carlo sampling with integrator snippets, starting with recycling solutions for the leapfrog integrator HMC and unfolding Hamiltonians for moving more easily. With snippets representing discretised paths along the level sets being used as particles, picking zero, one, or more particles along each path, since importance weights are connection with multinomial HMC

This relatively small algorithmic modification of the conditional particle filter, which we call the conditional backward sampling particle filter has a dramatically improved performance over the conditional particle filter. Karjalainen et al., 2024

Anthony Lee looked at mixing times for backward sampling SMC (CBPF/ancestor sampling) cf Lee et al. (2020), where the backward step consists in computing the weight of a randomly drawn backward or ancestral history. Improving on earlier results to reach mixing time O(log T) and complexity O(T log T) (with T the time horizon). Thanks to maximal coupling and boundedness assumptions on the prior and likelihood functions.

Neil Chada presented a work on Bayesian multilevel Monte Carlo on deep networks. À la Giles, with a telescoping identity. Always puzzling to envision a prior on all parameters of a neural network. Achieving a computational cost inverse to the order of the MSE, at best. With a useful reminder that pushing the size of the NN to infinity results in a (poor) Gaussian process prior (Sell et al., 2023).

On my first evening, I stopped with a friend in my favourite Blonde [restaurant], as in almost every other visit to Edinburgh, enjoyable as always, but I also found the huge offer of Asian minimarkets in the area too tempting to resist, between Indian, Korean, and Chinese products. (Although with a disappointing hojicha!). As I could not reach any new Munro by train or bus within a reasonable time range I resorted to the nearer Pentland Hills, with a stop by Rosslyn Chapel (mostly of Da Vinci Code fame!, if classic enough). And some delays in finding a bus getting there (misled by google map!) and a trail (misled by my poor map reading skills) up the actual hills. The mist did not help either.

off to Edinburgh [and SMC 2024]

Posted in Books, Mountains, pictures, Statistics, Travel, University life with tags , , , , , , , on May 12, 2024 by xi'an

IMG_9351Today I am off to Edinburgh for the SMC 2024 workshop run by the ICMS. Looking forward meeting with long time friends and new ones, and learning about novel directions in the field. And returning to Edinburgh I last visited in 2019 for the opening of the Bayes Centre. Hoping to enjoy the nearby Arthur’s Seat volcano and maybe farther away Munroes, depending on the program, train schedules, and…weather forecasts!

the flawed genius of William Playfair [book review]

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , on March 26, 2024 by xi'an

David Bellhouse has written a new book on the history of statistics, focussing on William Playfair this time (following his fantastic book on Abraham de Moivre). The Flawed Genius of William Playfair (The Story of the Father of Statistical Graphics) got published a few months ago by the University of Toronto Press.

“[Playfair] was an ideas man whose ideas often did not come to fruition; or, when they did, they withered or exploded.” [p.121]

The impressions I retained from reading this detailed account of a perfect unknown (for me) are of a rather unpleasant, unappealing, unsuccessful, fame-seeking, inefficient, short-sighted, self-aggrandising,  bigoted, dishonest, man, running from debtors for most of his life, with jail episodes for bankruptcy, while trying to make a living from all sorts of doomed enterprises, short-lived blackmailing attempts, and mediocre books that did not sell to many. Similar to David Bellhouse’s colleague earlier wondering at the appeal of exposing such a rogue character, I am left with this lingering interrogation after finishing the book…

“[Richard] Price liked what Playfair had written. He found [in 1786] Playfair to be “agreeable” and “useful”.” [p.64]

Not that I did not enjoy reading it!, as it gives a most interesting of the era between the 18th and the 19th Centuries, in particular in its detailed narration of the first months of the French Revolution of 1789, and of the impact of the Industrial Revolution on economics and politics as the birth of capitalism. The book abounds in crossing lots of historical characters, like Richard Price (Bayes’s friend who published his most famous paper), Adam Smith (whose book Playfair reprinted with poor additions), Edward Gibbons (whose book along with Smith’s inspired the title of his Inquiry Into the Permanent Causes of the Decline and Fall of Powerful and Wealthy Nations), Thomas Malthus (competing for an annotated edition of Smith’s book), not to mention the political class of Britain at the time. David Bellhouse’s book demonstrates academic and historical excellence, constantly being very detailed, with a wealth of references, documents, and definite support for or against the rumours that accompany the life and deeds of Playfair. (Frankly, rarely a name has been that inappropriate!) This includes for instance the pictures pointing out to his first (?) forged signature [p.140] and the evacuation of the myth of Playfair as a spy for the British Crown—which the Wikipedia page happily reproduces, pointing out the need for an in-depth revision of said page. Similarly, the book delivered a convincing discussion of arguments for and mostly against Playfair “being the key player in the British operation to forge [French] assignats” towards destroying its economy. A lot of the book is touching upon the then novel issue of paper money, which Playfair only and negatively considered through his own (and catastrophic) experiences. At times, the book is almost too scholarly as it makes reading less fluid than was the case his Abraham de Moivre for instance. (And obviously less than in the contemporary Jonathan Strange & Mr. Norrel!)

It may be that my very relative lack of enthusiasm stems from the realisation that the story of Playfair is overall rather little connected with statistical inference, if not with descriptive statistics (albeit with a complete disregard for the quality and sources of his data), as when  publishing a Statistical Breviary on descriptive statistics for a series of countries (and surprisingly sold on Amazon!).  Or Statistical Account of the United States of America. And of course for his innovative graphical representations like the one represented on the cover of the book or the pie chart. I feel that the book is much more engaged in Playfair’s contributions to the then nascent science of economics, as for instant about the shallow and mostly misguided views of his’ on banking and running the economy, while conducting his personal finance and investments so disastrously that it negatively advertised against confidence in such views.

On a very personal level, I noticed that some graphs were provided by my friend and statistics historian Stephen Stigler [who also wrote a review of the book] while an analysis of the poor French involved in a coding scam of Playfair about Napoléon’s escape from Elba was by Christian Genest (whom I first met at a statistics conference dinner on the Lac de Neufchâtel in 1986).

[Disclaimer about potential self-plagiarism: this post or an edited version will eventually appear in my Books Review section in CHANCE. As appropriate for a book about Chance!]

 

merely fiddlin

Posted in Books, Kids, Mountains, pictures, R, Running, Travel with tags , , , , , , , , , on February 25, 2024 by xi'an

For any positive, base-10 integer N, define f(N) as the number of times you have to add up its digits until you get a one-digit number. Find the smallest whole number N such that f(N) = 4. Fiddler on the Proof

Although I first gave in to the R brute force attempt, it failed to return a value for N up to 10⁹, but the solution is obvious as f(N)=p iff F(s(N))=p-1. Since f(19)=2 and f(199)=3, a number N whose sum of digits is 199 is legit, for instance

f(99999999999999999999991)=4

where there are 22 9s. And although it gets a wee bit boring, here is the associated attempt by ChatGPT³