Archive for divide & conquer

6th Workshop on Sequential Monte Carlo Methods (#2)

Posted in Mountains, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , , , , , , , , , , , , on June 5, 2024 by xi'an

Managed to get back from the Pentland hills in time for the Wednesday afternoon session, which proved most interesting as close to my research interests!

Nicola Branchini presented his work with Victor Elvira (a close friend and coauthor, incidentally one of the organisers of the workshop!) on improving self normalised importance sampling by interpreting it as a ratio of estimators based on two samples (which may be the same) and attempting to optimise the joint distribution of said sample. The starting assumption is having (good) marginal importance functions, which means the goal here is in optimising a copula distribution targeting the ratio as quantity of interest. Optimality is however defined in terms of the approximate asymptotic variance of the ratio, which remains an approximation. The idea is nonetheless quite interesting and shows potential for connecting with bridge sampling and… AMIS! As an aside, the talk considered cases when the margins are multivariate, which requires a généralisation of Sklar’s theorem. Simo Särkä then demonstrated how highly parallel processors like GPUs can accommodate Bayesian filters and smoothers in state space models not requiring simulation, gaining a reduction in complexity from O(T) to O(log T). I had not really thought of parallel processing in the recent years, hence was quite pleased at hearing this resolution based on so-called associative scans, and see that implementations were already available in Julia/CUDA.

This was followed by a highly enjoyable poster session, including chats about ABC-SMC for discovery rates, infinite dimensional diffusions, Pareto smoothed importance samplings, &tc with posters by Hugo Marival (coauthor of our importance Monte Carlo recent paper) and Shreya Roy (a student at U of Warwick). With sunny views of Arthur’s Seat (and plenty of people at the top), contrary to the above! Followed by a private party dinner occupying half of a nearby and novel South Indian restaurant that proved quite tasty, local and definitely enjoyable.

For my last morning in town, albeit it was unrelated to the posted abstract, Pierre Del Moral spoke about noisy versions of the ensemble Kalman filter on linear diffusions that allowed for stable solutions under strong enough conditions, encompassing an impressive corpus of work over the past ten years. Alex Beskos presented antithetic multilevel methods for diffusions, which allow to improve the error in the discretisation, even though I did not fully get the whole idea (partly due to dozing out from time to time, a consequence of my last early rounds of Arthur’s Seat in the very early morn).

Daniel Paulin presented a novel unbiased method based on kinetic Langevin dynamics that combines advanced splitting methods with enhanced gradients, avoiding Metropolis correction by coupling and multilevel Monte Carlo approach, achieving unbiasedness by telescoping, but involving an avalanche of acronyms in the leapfrog/Gibbs steps.  And Adam Johansen (U of Warwick) on several recent papers of their divide-and-conquer filtering methods, introduced in a 2017 JCGS paper, following a decomposition of the state variable into low-dimensional components like branches and leaves of a tree.

Approximation Methods in Bayesian Analysis [#1]

Posted in Mountains, pictures, Running, Statistics, Travel, University life with tags , , , , , , , , , , , , on June 20, 2023 by xi'an

Interesting first day, if somewhat intense!, with Sylvia Richardson talking of divide & conquer strategies, which we had recently explored in our work on mixtures with Adrien and Judith. Then Jere Koskela (Warwick) on sufficient conditions for consistent inference on trees like Kingman’s coalescent. And in the afternoon Julia-Adela Palacios on tree estimation, with novel notions of tree distances and barycentre (soon to appear in Biometrika!). And Julyan Arbel on the notion and properties of priors within (deep) neural networks, following different asymptotics. On top of this, two fast sessions for people presenting their poster, to be attended tonight. And the usual early morning run to swim 25mn in one of the calanques… Luckily enough, Marseille is not too hot this week, feeling even nicer than Paris!

the most important statistical ideas of the past 50 years

Posted in Books, pictures, Statistics, Travel with tags , , , , , , , , , , , , , , , , , on January 10, 2020 by xi'an

A grand building entrance near the train station in HelsinkiAki and Andrew are celebrating the New Year in advance by composing a list of the most important statistics ideas occurring (roughly) since they were born (or since Fisher died)! Like

  • substitution of computing for mathematical analysis (incl. bootstrap)
  • fitting a model with a large number of parameters, using some regularization procedure to get stable estimates and good predictions (e.g., Gaussian processes, neural networks, generative adversarial networks, variational autoencoders)
  • multilevel or hierarchical modelling (incl. Bayesian inference)
  • advances in statistical algorithms for efficient computing (with a long list of innovations since 1970, including ABC!), pointing out that a large fraction was of the  divide & conquer flavour (in connection with large—if not necessarily Big—data)
  • statistical decision analysis (e.g., Bayesian optimization and reinforcement learning, getting beyond classical experimental design )
  • robustness (under partial specification, misspecification or in the M-open world)
  • EDA à la Tukey and statistical graphics (and R!)
  • causal inference (via counterfactuals)

Now, had I been painfully arm-bent into coming up with such a list, it would have certainly been shorter, for lack of opinion about some of these directions (even the Biometrika deputeditoship has certainly helped in reassessing the popularity of different branches!), and I would have have presumably been biased towards Bayes as well as more mathematical flavours. Hence objecting to the witty comment that “theoretical statistics is the theory of applied statistics”(p.10) and including Ghosal and van der Vaart (2017) as a major reference. Also bemoaning the lack of long-term structure and theoretical support of a branch of the machine-learning literature.

Maybe also more space and analysis could have been spent on “debates remain regarding appropriate use and interpretation of statistical methods” (p.11) in that a major difficulty with the latest in data science is not so much the method(s) as the data on which they are based, which in a large fraction of the cases, is not representative and is poorly if at all corrected for this bias. The “replication crisis” is thus only one (tiny) aspect of the challenge.