Our Warwick PhD student Shreya Sinha-Roy—who is now looking for a postdoctoral position next semester!—, along with Sherman Khoo, Ritabrata Dutta and myself, has now completed a paper on shrinkage priors for implicit generative models. That is, models based on deep neural networks and hence associated with intractable likelihoods. The work centres on developing and assessing an efficient training mechanism for these models, leveraging on tools from Bayesian model averaging using shrinkage (yay!) priors inspired from Lasso (rather than from my PhD years!) and generalized Bayes. In this large p (dimension of parameters) and small n (sample size of data) scenario, those sparsity inducing priors have been successfully used for linear regression when p is much larger than n, but have not been applied to implicit generative models due to the intractability of the likelihood function of the parameters of the model given observed data. Adapting a scoring rule posterior based on a strictly proper scoring rule as in generalized Bayes, we propose a block SGMCMC within Gibbs sampling mechanism to handle high dimensional parameter space for learning a sparse Bayesian model averaged neural implicit generative model in a sample efficient way. We illustrate excellent performance of our proposed method for p (much larger than n) linear regressions and three applications of neural generative models in tasks relevant to weather forecasting to reinforcement learning
Archive for Université de Rouen
back to shrinkage!
Posted in Books, Statistics, University life with tags Bayesian lasso, generalised Bayesian inference, generative model, Gibbs sampling, intractable likelihood, MathPhDInFrance, Rouen, shrinkage, Université de Rouen, University of Warwick, Warwickshire on February 12, 2026 by xi'anWilliam (Bill) Strawderman (1941-2024)
Posted in pictures, Statistics, University life with tags admissibility, apple brandy, Bayesian decision theory, Bill Strawderman, Calvados, Charles Dickens, credible intervals, Institut de Mathématiques de Jussieu, James-Stein estimator, minimaxity, New Brunswick, New Jersey, Paris 6, Pitman nearness, Rutgers University, shrinkage estimation, Université de Rouen on October 3, 2024 by xi'an
Earlier today, I was informed by several of our mutual friends that my long-time friend Bill Strawderman had sadly passed away yesterday, after fighting a cancer for the past months. I remember quite clearly meeting Bill in the Fall of 1988 in front of White Hall, which hosted the Cornell maths department at the time, as he was visiting George Casella from Rutgers where he spent most of his career. I was most eager to meet him as I had worked on several of his landmark papers during my PhD on shrinkage estimation, as well as a bit impressed. But his kindness, modesty, and congenial personality quickly put me at ease and we spent the rest of his visit discussing shrinkage but also literature and music. Especially Dickens! After that we met and collaborated quite regularly, to the point he started visiting France upon my return, at Paris 6 (Pierre & Marie Curie) University first, and then in Rouen, where he became a adjunct professor and launched a life-long collaboration and friendship with Dominique Fourdrinier. As my interest in shrinkage estimation dwindled along the years, we did not keep collaborating for the past two decades, but we remained in touch and I was very happy to participate in his 80th anniversary celebration in Rutgers two years ago. His contributions to the field are notable and several papers of his were part of the Bayesian classics I was giving my graduate class a few years ago. From the fabulous minimaxity paper of 1984, along with George Casella, to admissible estimators dominating the positive-part James-Stein estimator, to sufficient conditions of minimaxity for proper Bayes estimators, to decision theoretic properties of Bayesian credible interval estimators, to loss estimation, not to mention his more applied side… Besides his fabulous sense of humour, which made many evenings with him memorable, I will also cherish the memory of a bon vivant who liked good food and good wines, incl. the Calvados apple brandy I would bring him at each of my visits.
35 years ago…
Posted in Books, Kids, Statistics, Travel, University life with tags Jean-Pierre Raoult, José de Sam Lazaro, Normandy, PhD thesis, postdoctoral position, Purdue University, shrinkage estimation, Université de Rouen, visiting position on July 2, 2022 by xi'an
estimation of a normal mean matrix
Posted in Statistics with tags Biometrika, Charles Stein, Cornell University, James-Stein estimator, Purdue University, Rutgers University, shrinkage estimation, Springer-Verlag, superharmonicity, Université de Rouen on May 13, 2021 by xi'an
A few days ago, I noticed the paper Estimation under matrix quadratic loss and matrix superharmonicity by Takeru Matsuda and my friend Bill Strawderman had appeared in Biometrika. (Disclaimer: I was not involved in handling the submission!) This is a “classical” shrinkage estimation problem in that covariance matrix estimators are compared under under a quadratic loss, using Charles Stein’s technique of unbiased estimation of the risk is derived. The authors show that the Efron–Morris estimator is minimax. They also introduce superharmonicity for matrix-variate functions towards showing that generalized Bayes estimator with respect to a matrix superharmonic priors are minimax., including a generalization of Stein’s prior. Superharmonicity that relates to (much) earlier results by Ed George (1986), Mary-Ellen Bock (1988), Dominique Fourdrinier, Bill Strawderman, and Marty Wells (1998). (All of whom I worked with in the 1980’s and 1990’s! in Rouen, Purdue, and Cornell). This paper also made me realise Dominique, Bill, and Marty had published a Springer book on Shrinkage estimators a few years ago and that I had missed it..!
MiMo2020
Posted in Statistics with tags likelihood-free inference, mixture of distributions, Normandie, Normandy, Rouen, SIAM Conference on Uncertainty Quantification, Université de Rouen, UQ20, workshop on January 24, 2020 by xi'an
On 26 and 27 March 2020, the maths department of the Université of Rouen, Normandy, France, organizes a (free) workshop on mixture distributions. With the following speakers
-
- Christophe Biernacki (Laboratoire Paul Painlevé, Univ. Lille 1 et INRIA)
- Vincent Brault (Laboratoire Jean Kuntzmann, Univ. Grenoble Alpes)
- Gilles Celeux (Laboratoire de Mathématiques d’Orsay, Univ. Paris Sud et INRIA)
- Elisabeth Gassiat (Laboratoire de Mathématiques d’Orsay, Univ. Paris Sud)
- Van Hà Hoang (Laboratoire de Mathématique Raphaël Salem, Univ. Rouen Normandie)
- Hajo Holzmann (Philipps-University Marburg, Germany)
- Dimitri Karlis (Department of Statistics, Athens University of Economics and Business, Greece)
- Trung Tin Nguyen (LMNO, Univ. Caen Normandie)
- Andrea Rau (Département de Génétique Animale, INRA, Jouy en Josas)
- Pierre Vandekerkhove (Laboratoire d’Analyse et de Mathématiques Appliquées, Univ. Paris-Est Marne-la-Vallée)
- Cinzia Viroli (Department of Statistical Sciences, Universita di Bologna, Italia)
Unfortunately, since this is my former department, I will not be able to attend as I am taking part into the SIAM Conference on Uncertainty Quantification (UQ20), on the very same days. In a session on likelihood-free inference.
