Safety, security and risk
AI poses or intersects with a range of safety and security challenges, some of which are immediate and some of which may only manifest in future as more powerful systems are developed and deployed in a wider range of societal settings. Many of the most transformative impacts of AI, both in terms of potential benefits and risks, remain decades in the future. However, there is work to be done now on these future challenges, by exploring safety challenges in fundamental AI research, and understanding the risks associated with future AI development scenarios. Furthermore, safety norms and practices put in place now support us in being better prepared for the challenges of more capable future systems. AI:FAR explores near-term risks associated with the role of AI in synthetic media, manipulation and information security; defense and military use; and critical processes such as agriculture. It also explores longer-term challenges associated with potential future developments in AI.
This strand includes the Future of Life Institute-funded Paradigms of AGI and their Associated Risks project, which explores safety challenges that may emerge for AI systems with increasing generality and capability.
Recent papers include:
Seger, E., Avin, S., Pearson, G., Briers, M., Ó Heigeartaigh, S., Bacon, H. (2020). Tackling threats to informed decision- making in democratic societies. Alan Turing Institute.
Hernández-Orallo, J., Martınez-Plumed, F., Avin, S., Whittlestone, J., & Ó hÉigeartaigh, S.S. (2020) AI Paradigms and AI Safety: Mapping Artefacts and Techniques to Safety Issues. ECAI 2020
Shackelford, G. E., Kemp, L., Rhodes, C., Sundaram, L., ÓhÉigeartaigh, S. S., Beard, S., ... & Jones, E. M. (2020). Accumulating evidence using crowdsourcing and machine learning: A living bibliography about existential risk and global catastrophic risk. Futures, 116, 102508.
Whittlestone, J., & Ovadya, A. (2019). The tension between openness and prudence in responsible AI research. NeurIPS 2019 Joint Workshop on AI for Social Good. (TAG: NORMS AND PRINCIPLES; AI SAFETY)
Ovadya, A., & Whittlestone, J. (2019). Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning. arXiv preprint arXiv:1907.11274. (TAG: NORMS AND PRINCIPLES; AI RISK SCENARIOS)
Hernández-Orallo, J., Martínez-Plumed, F., Avin, S. and Ó hÉigeartaigh, S.S. (2019). Surveying Safety-relevant AI Characteristics. AAAI 2019.
Avin, S. & Amadae, S.M. (2019). Autonomy and machine learning at the interface of nuclear weapons, computers and people. In The Impact of Artificial Intelligence on Strategic Stability and Nuclear Risk, SIPRI, (pp. 105-118).
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., ... & Anderson, H. (2018). The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv preprint arXiv:1802.07228.