Archive for superintelligence
AI in a storm [cover]
Posted in Books, pictures, University life with tags Académie Goncourt, AI, Anthropic, dangers of AI, deep learning, doomsday argument, francophone literature, Francophonie, French literature, LLMs, mathematical conjecture, Nature, open AI, plagiarism, superintelligence, UN, United Nations General Assembly, weather forecasting, weather prediction on October 6, 2026 by xi'an
“too dangerous to release” [a correct prediction]
Posted in Books, Travel with tags @ScientistTrump, AIs, Anthropic, Fable 5, hackers, industrial ruins, LLMs, Mythos, Nature, protectionism, superintelligence, Sydney Harbour, Trump administration on July 9, 2026 by xi'an
Nature of 26 May had a long report on how the Mythos model is deemed to be too dangerous by its conceptor, Anthropic, to be publicly released, but instead shared with a limited number of organisations of their own choice. The argument being that Mythos was able to spot security flaws in every operating system and web browser on the market. But, sooner or later, the AI will prove accessible to a hacker who manages to penetrate one of these organisations, won’t it?! As pointed out by Nature’s writers, the natural question is why should firms be left to decide on who gets access to their software and how dangerous they are. The same writers also muse about governments imposing prohibitions on the firms, including outside their country. Which is exactly what happened a few weeks later on 12 June with the Trump administration “suspend[ing] all access to both Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States” with little factual argumentation. Another illustration of the short-sighted, knee-jerk, inconsistent, policies of that joke of an administration that does not address the real issue.
maths as catchy background [and no further future for humanity]
Posted in Books, pictures, University life with tags AI, artificial intelligence, book reviews, dystopia, mathematical formula, neural networks and learning machines, Nick Bostrom, Philosophy of Science, poor maths, superintelligence, The Guardian, University of Oxford on May 11, 2024 by xi'an
même pas peur [not afrAId]
Posted in Books, Kids, Travel, University life with tags AI for good, Amnesty International, BBC, book reviews, bots, ChatGPT, doomsday argument, fake news, Paris 2024 Olympics, power to the machines, propaganda, social media, superintelligence, The New York Times on March 30, 2023 by xi'an
Both the Beeb and The New York Times are posting tonight about a call to pause AI experiments, by AI researchers and others, due to the danger they could pose to humanity. While this reminds me of Superintelligence, a book by Nick Bostrom I found rather unconvincing, and although I agree that automated help-to-decision systems should not become automated decision systems, I am rather surprised at them setting the omnipresent Chat-GPT as the reference not to be exceeded.
“AI systems with human-competitive intelligence can pose profound risks to society and humanity (…) recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control.”
The central (?) issue here is whether something like Chat-GPT can claim anything intelligence, when pumping data from a (inevitably biased) database and producing mostly coherent sentences without any attention to facts. Which is useful when polishing a recommendation letter at the same level as a spelling corrector (but requires checking for potential fake facts inclusions, like imaginary research prizes!)
“Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?”
The increasingly doom-mongering tone of the above questions is rather over the top (civilization, nothing less?!) and again remindful of Superintelligence, while spreading propaganda and untruth need not wait super-AIs to reach conspiracy theorists.
“Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable (…) Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in”
A six months period sounds like inappropriate for an existential danger, while the belief that governments want or can intervene sounds rather naïve, given for instance that they lack the ability to judge of the dangerosity of the threat and of the safety nets to be imposed on gigantic black-box systems. Who can judge on the positivity and risk of a billion (trillion?) parameter model? Why is being elected any guarantee of fairness or acumen? Beyond dictatures thriving on surveillance automata, more democratic countries are also happily engaging into problematic AI experiments, incl. AI surveillance of the incoming Paris Olympics. (Another valuable reason to stay away from Paris over the games.)
“AI research and development should be refocused on making today’s powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal. In parallel, AI developers must work with policymakers to dramatically accelerate development of robust AI governance systems.”
While these are worthy goals, at a conceptual level—with the practical issue of defining precisely each of these lofty adjectives—, and although I am certainly missing a lot from my ignorance of the field, this call remains a mystery to me as it sounds unrealistic it could achieve its goal.
weapons of math destruction [fan]
Posted in Statistics with tags AI, artificial intelligence, Cédric Villani, French parliement, French politics, Le Monde, singularity, superintelligence, weapons of math destruction on September 20, 2017 by xi'anAs a [new] member of Parliement, Cédric Villani is now in charge of a committee on artificial intelligence, which goal is to assess the positive and negative sides of AI. And refers in Le Monde interview below to Weapons of Maths Destruction as impacting his views on the topic! Let us hope Superintelligence is no next on his reading list…
