Archive for automation

AI Narratives [book review]

Posted in Books with tags , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , on June 13, 2025 by xi'an

AI Narratives: A history of imaginative thinking about intelligent machines is a 2020 collective book edited by Stephen Cave, Kanta Dihal, and Sarah Dillon, with about twenty contributing authors, through a series of 16 chapters on the relation between culture (literature, films) and our societal approach to AI, with varying perspectives, some overlap between chapters, a wide range of extrapolation, especially in relation with the oldest books (like Homer’s) and quotes from both books I enjoyed and books I had not heard of, to add to my to-red pile, like Roderick. Predominant place of Blade Runner, of Ĉapek’s Rossum’s Universal Robots, not only for introducing the term, and of Asimov, but also a detailed analysis of the great, thought-provoking, Ann Leckie’s Ancillary Justice. With a realization that Gibson’s Neuromancer had aged quite a lot… For the modern times, very little outside the US-UK realm, as for instance no mention made of Jules Verne or René Barjavel, although Zola makes an appearance when describing workers as automats, neither of the Germanic literature (from the Grimm Brothers onwards), nor of the non-negligible USSR science fiction production, nor yet of the Chinese input, like The three body problem. Other books that could have made it: the Murderbot diaries, and The Alchemy Wars trilogy, for digging into the blurry border between humans and AIs; A Memory Called Empire, for a clever approach to mind uploading, as well as the masterly Never let me go by Ishiguro, both dealing with a future where copies of humans The Amazing Adventures of Kavalier & Clay for the golem, the short story An Unatural Life, for its highly original take on the legal rights of humanoid robots, A Psalm for the Wild-Built because of… tea and Zen monk, obviously!

Among things I learned from AI Narratives, the story of Alan Turing (figuratively) mansplaining Ada Lovelace on her pronouncement that machines cannot be intelligent since they deliver what they are coded for. The mention of automatos in the Illiad. A mention of 1625 Gabriel Naudé Apologie &tc. that excludes magic as irrational, as well as introducing the term androide. The trivia that the creator of the (fraudulent) automaton chess player, Kempelen, also produced in 1791 an authentic if brainless speaking machine. The realisation that E.M Forster also wrote a futuristic novel, The Machine Stops (as well as the precursor of the symbolists, Villiers de l’Isle Adam, with L’Ève Future). Another trivia that cyberpunk first appeared in a 1983 short story by Bruce Bethke. A discussion of the elaborate Culture constructed by Iain Banks, albeit through volumes in the series I hade not read, along with the concept of OCP for outside context problem, akin to Taleb’s Black Swan. The least interesting (and final) chapter in the book is paradoxically the closest to data analysis, when Recchia runs a rather low-tech assessment of a subtitle dataset in relation with AI, if referring to Tufte’s rule of the baselin in the notes (p405).

Definitely enjoyable book, then, even though I mostly skimmed through the chapters during a day-trip to Lille on a Bank (!) holiday. Reading through it made me muse rather belatedly of a parallel between AI scare or adoration, and the non-AI societies, where individuals are (also) part of a structure large enough to miss the larger picture. From building pyramids to being part of the global economy. (This followed mostly from my surprise in seeing Dickens, Trollope, and Zola included in the discussion.)

[Disclaimer about potential self-plagiarism: this post or an edited version will eventually appear in my Books Review section in CHANCE.]

Leave a comment »

Hastings 50 years later

Posted in Books, pictures, Statistics, University life with tags , , , , , , , , , , , , on January 9, 2020 by xi'an

What is the exact impact of the Metropolis-Hastings algorithm on the field of Bayesian statistics? and what are the new tools of the trade? What I personally find the most relevant and attractive element in a review on the topic is the current role of this algorithm, rather than its past (his)story, since many such reviews have already appeared and will likely continue to appear. What matters most imho is how much the Metropolis-Hastings algorithm signifies for the community at large, especially beyond academia. Is the availability or unavailability of software like BUGS or Stan a help or an hindrance? Was Hastings’ paper the start of the era of approximate inference or the end of exact inference? Are the algorithm intrinsic features like Markovianity a fundamental cause for an eventual extinction because of the ensuing time constraint and the lack of practical guarantees of convergence and the illusion of a fully automated version? Or are emerging solutions like unbiased MCMC and asynchronous algorithms a beacon of hope?

In their Biometrika paper, Dunson and Johndrow (2019) recently wrote a celebration of Hastings’ 1970 paper in Biometrika, where they cover adaptive Metropolis (Haario et al., 1999; Roberts and Rosenthal, 2005), the importance of gradient based versions toward universal algorithms (Roberts and Tweedie, 1995; Neal, 2003), discussing the advantages of HMC over Langevin versions. They also recall the significant step represented by Peter Green’s (1995) reversible jump algorithm for multimodal and multidimensional targets, as well as tempering (Miasojedow et al., 2013; Woodard et al., 2009). They further cover intractable likelihood cases within MCMC (rather than ABC), with the use of auxiliary variables (Friel and Pettitt, 2008; Møller et al., 2006) and pseudo-marginal MCMC (Andrieu and Roberts, 2009; Andrieu and Vihola, 2016). They naturally insist upon the need to handle huge datasets, high-dimension parameter spaces, and other scalability issues, with links to unadjusted Langevin schemes (Bardenet et al., 2014; Durmus and Moulines, 2017; Welling and Teh, 2011). Similarly, Dunson and Johndrow (2019) discuss recent developments towards parallel MCMC and non-reversible schemes such as PDMP as highly promising, with a concluding section on the challenges of automatising and robustifying much further the said procedures, if only to reach a wider range of applications. The paper is well-written and contains a wealth of directions and reflections, including those in my above introduction. Here are some mostly disconnected directions I would have liked to see covered or more covered

  1. convergence assessment today, e.g. the comparison of various approximation schemes
  2. Rao-Blackwellisation and other post-processing improvements
  3. other approximate inference tools than the pseudo-marginal MCMC
  4. importance of the parameterisation of the problem for convergence
  5. dimension issues and connection with quasi-Monte Carlo
  6. constrained spaces of measure zero, as for instance matrix distributions imposing zeros outside a diagonal band
  7. given the rise of the machine(-learners), are exploratory and intrinsically slow algorithms like MCMC doomed or can both fields feed one another? The section on optimisation could be expanded in that direction
  8. the wasteful nature of the random walk feature of MCMC algorithms, as opposed to non-reversible kernels like HMC and other PDMPs, missing from the gradient based methods section (and can we once again learn from physicists?)
  9. finer convergence issues and hence inference difficulties with complex MCMC algorithms like Gibbs samplers with incompatible conditionals
  10. use of the Hastings ratio in other algorithms like ABC or EP (in link with the section on generalised Bayes)
  11. adapting Metropolis-Hastings methods for emerging computing tools like GPUs and quantum computers

or possibly less covered, namely data augmentation put forward when it is a special case of auxiliary variables as in slice sampling and in earlier physics literature. For instance, both probit and logistic regressions do not truly require data augmentation and are more toy examples than really challenging applications. The approach of Carlin & Chib (1995) is another illustration, which has met with recent interest, despite requiring heavy calibration (just like RJMCMC). As well as a a somewhat awkward opposition between Gibbs and Hastings, in that I am not convinced that Gibbs does not remain ultimately necessary to handle high dimension problems, in the sense that the alternative solutions like Langevin, HMC, or PDMP, or…, are relying on Euclidean assumptions for the entire vector, while a direct product of Euclidean structures may prove more adequate.

Leave a comment »

Nature Outlook on AI

Posted in Statistics with tags , , , , , , , , , , , , , , , on January 13, 2019 by xi'an

The 29 November 2018 issue of Nature had a series of papers on AIs (in its Outlook section). At the general public (awareness) level than in-depth machine-learning article. Including one on the forecasted consequences of ever-growing automation on jobs, quoting from a 2013 paper by Carl Frey and Michael Osborne [of probabilistic numerics fame!] that up to 47% of US jobs could become automated. The paper is inconclusive on how taxations could help in or deter from transfering jobs to other branches, although mentioning the cascading effect of taxing labour and subsidizing capital. Another article covers the progresses in digital government, with Estonia as a role model, including the risks of hacking (but not mentioning Russia’s state driven attacks). Differential privacy is discussed as a way to keep data “secure” (but not cryptography à la Louis Aslett!). With another surprising entry that COBOL is still in use in some administrative systems. Followed by a paper on the apparently limited impact of digital technologies on mental health, despite the advertising efforts of big tech companies being described as a “race to the bottom of the brain stem”! And another one on (overblown) public expectations on AIs, although the New York Time had an entry yesterday on people in Arizona attacking self-driving cars with stones and pipes… Plus a paper on the growing difficulties of saving online documents and culture for the future (although saving all tweets ever published does not sound like a major priority to me!).

Interesting (?) aside, the same issue contains a general public article on the use of AIs for peer reviews (of submitted papers). The claim being that “peer review by artificial intelligence (AI) is promising to improve the process, boost the quality of published papers — and save reviewers time.” A wee bit over-optimistic, I would say, as the developed AI’s are at best “that statistics and methods in manuscripts are sound”. For instance, producing “key concepts to summarize what the paper is about” is not particularly useful. A degree of innovation compared with the existing would be. Or an automated way to adapt the paper style to the strict and somewhat elusive Biometrika style!

Leave a comment »