Archive for accept-reject algorithm

the acceptance-complement method

Posted in Books, Statistics with tags , , , , , , on August 6, 2026 by xi'an

Last Saturday, while getting reacquainted with my home desk, I was delighted to spot a new arXival by Luc Devroye! Entitled the acceptance-complement method revisited, it discusses a variant of accept-reject method, where the target density f(·) is squeezed between two functions

0≤r(x)≤f(x)≤r(x)+q(x)

and can be simulated as the mixture

r(x)+(f(x)-r(x)),

assuming both r(·) and q(·) are proportional to densities that can easily be simulated. (With the additional constraint of q(·) having at most mass 1.) The nicest part is that the algorithm is always one-pass, as 

  1. generate Y~ q(y), U ~U(0,1)
  2. if Uq(Y)<f(Y)-r(Y), set X=Y
  3. else generate X~r(x)

with step 2 having two outcomes: either generate a realisation of [the density proportional to] f(·)-r(·) or generate an event of probability α when α is the mass [normalising constant] of r(·). This explains step 3 since the rejection of Y implies simulation from the second component of f(·). Brilliant! And Devroye derives a universal accept-complement algorithm for generic log-concave densities (that differs from Gilks & Wild, 1992.).

Monte Carlo [exam]

Posted in Kids, pictures, R, Statistics, University life with tags , , , , , , , , , , , , , , , , , on January 22, 2025 by xi'an

My final exam for the Monte Carlo course I taught last semester proved too much of a challenge for my fourth year students, despite being rather elementary and centred on accept-reject algorithms and importance/bridge sampling. One of the problems was a decomposition of the truncated Normal simulation method proposed by Marsaglia in 1963, found on X Validated!, which I had posted on the course Teams forum (and discussed on the ‘Og!). Obviously overlooked by the students as hardly anyone solved the central question… Quite a disappointment (and more exams to grade for the resit exam in June!)

positive response to negative mixtures

Posted in pictures, Running with tags , , , , , , , , , , , , , , on December 17, 2024 by xi'an

Hurray, our signed mixture simulation paper has been accepted by Statistics & Computing! If Og’s readers remember my earlier post about this problem, things get surprisingly more complicated when the mixture weights can take negative values. For instance, the naïve solution consisting in first simulating from the associated mixture of positive weight components and then using an accept-reject step may prove highly inefficient since the overall probability of acceptance can get arbitrarily close to zero. Substituting to this naïve version, we construct an alternative accept-reject scheme based on pairing positive and negative components as efficiently as possible, partitioning the real line, and finding tighter upper and lower bounds on positive and negative components, respectively, towards yielding a higher acceptance rate on average. In retrospect, the problem was beyond the reach of the undergraduate students we supervised (pre-COVID) on a research internship!

another, older, truncated Normal algorithm [X’ed]

Posted in Books, pictures, Statistics, University life with tags , , , , , , on September 23, 2024 by xi'an

[very] simple rejection Monte Carlo

Posted in Books, pictures, R, University life with tags , , , , , , , , on March 29, 2024 by xi'an

“In recent years, the Rejection Monte Carlo (RMC) algorithm has emerged sporadically in literature under alternative names such as screening sampling or reject-accept sampling algorithms”

First, I was intrigued enough by a new arXival spotted in the Thalys train from Brussels to take a deeper look at it, but soon realised there was nothing of substance in the paper. Which solely recalls the fundamental of (accept-)reject algorithms, invented in the early days of computer simulation by von Neumann (even though the preprint refers to much more recent publications).  Without providing the average acceptance probability as being equal to the inverse of the bounding constant [independently of the dimension of the random variable] and no mention of The Bible either… But with a standard depiction of accepted vs rejected points as uniformly dispersed on the subgraph of the proposal (as in the above taken from our very own Monte Carlo statistical Methods). Funnily enough, the most basic rejection algorithm, that is, the one based on a uniform sampling from a bounding (hyper)box is illustrated for a Normal target, although the latter has infinite support. And the paper seems to conclude on the appeal of using uniform proposals over bounding boxes, even though the increasing inefficiency against the dimension is well-known. A very simple rejection then, indeed!