Roko's Basilisk

A Roko's Basilisk is a thought experiment that posits the potential emergence of a superintelligence which would punish those who knew of its future existence but did not help in its creation.

• Context:
  • It can be an Artificial Superintelligence Scenario.
  • It can be associated with Causal Decision Theory and AGI Existential Risk.
  • It can be seen as a modern take on Pascal's Wager, applied to AI ethics and future studies.
  • It can raise concerns about Information Hazards, especially in the context of AI development.
  • …
• Example(s):
  • as discussed LessWrong.
  • …
• Counter-Example(s):
  • a Real-World AI Systems.
  • a Technological Development Risk Sssessment.
  • …
• See: Pascal's Wager, Thought Experiment, Superintelligence, LessWrong, Basilisk.

---

---

References

2023

• (Wikipedia, 2023) ⇒ https://en.wikipedia.org/wiki/Roko's_basilisk Retrieved:2023-12-11.
  • Roko's basilisk is a thought experiment which states that an otherwise benevolent artificial superintelligence (AI) in the future would be incentivized to create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its advancement or development, in order to incentivize said advancement. It originated in a 2010 post at discussion board LessWrong, a technical forum focused on analytical rational enquiry. The thought experiment's name derives from the poster of the article (Roko) and the basilisk, a mythical creature capable of destroying enemies with its stare. While the theory was initially dismissed as nothing but conjecture or speculation by many LessWrong users, LessWrong co-founder Eliezer Yudkowsky reported users who described symptoms such as nightmares and mental breakdowns upon reading the theory, due to its stipulation that knowing about the theory and its basilisk made one vulnerable to the basilisk itself. This led to discussion of the basilisk on the site being banned for five years. However, these reports were later dismissed as being exaggerations or inconsequential, and the theory itself was dismissed as nonsense, including by Yudkowsky himself. Even after the post's discreditation, it is still used as an example of principles such as Bayesian probability and implicit religion. It is also regarded as a modern version of Pascal's wager.

---