At the ISBA-Cini workshop on Artificial intelligence, data sharing and regulation, a satellite of ISBA 2024 that was also an ERC Ocean workshop, located in the magnificent buildings of a former Benedictine Monastery (next to the Palladian church), Stéfano Favaro gave a presentation of the output of his ERC Consolidator Grant “Nonparametric Bayes and Empirical Bayes for Species Sampling Problem”, in which he discussed in particular the corpus of work they produced on Bayesian inference with flawed data. He mentioned in particular a recent paper with Mario Berah and Vinayak Rao on running MCMC for Bayesian non-parametric estimation on privatized (noisy) data.
“…if the original dataset Y is modelled as a realization of a BNP mixture model, then, after marginalizing out the Yi’s, the Zi’s again follow a nonparametric mixture model….”
“…our approach has the flavour of the pseudo-marginal MCMC approach (Andrieu and Roberts, 2009), with the latent Yi’s introduced back into the MCMC state as auxiliary variables to deal with intractable Metropolis-Hastings probabilities…”
“As mentioned in Ju et al. (2022), the efficiency of our algorithms are linked to the special structure of differential privacy…”
The paper starts from a privatized model where the data is perturbed to ensure (standard) differential privacy and only examines the convergence impact of the additional randomness layer on MCMC performances, depending on two versions… One re-simulating the hidden data given everything else and the other close to Neal’s (2000) for this specific hierarchical model. While more involved, the resulting MCMC samplers are not much different from the original ones since the privatization leads to replacing the initial distribution with a convolution, but the actual data still proceeds from an infinite mixture. In that respect, the paper does not engage in a discussion of privacy, like the impact of the BNP estimation, possible gains due to producing a BNP estimator, &tc. The only clear impact of the privacy assumptions is that the Metropolis-Hastings acceptance probability is lower bounded, but this was also noticed in Ju et al. (2022). It would have been nice to assess the impact of non-parametrics on further protecting the data if any!
