What India’s AI Safety Institute could do

Image for representation.
| Photo Credit: Getty Images

In October, the Ministry of Electronics and Information Technology (MeitY) convened meetings with industry and experts to discuss setting up an AI Safety Institute under the IndiaAI Mission. Curiously, this came on the heels of Prime Minister Narendra Modi’s visit to the U.S., the Quad Leaders’ Summit, and the United Nations Summit of the Future. AI appeared high on the agenda in the run up to the Summit of the Future, with a high-level UN advisory panel producing a report on Governing AI for Humanity.

Policymakers should build on India’s recent leadership at the G20 and the GPAI, and position it as a unifying voice for the global majority in AI governance. The design of the Safety Institute should prioritise raising domestic capacity, capitalising on India’s comparative advantages, and plugging into international initiatives.

Notably, the Summit of the Future yielded the Global Digital Compact that identifies multi-stakeholder collaboration, human-centric oversight, and inclusive participation of developing countries as essential pillars of AI governance and safety. As a follow up, the UN will now commence a Global Dialogue on AI. It would be timely for India to establish an AI Safety Institute which engages with the Bletchley Process on AI Safety. If executed correctly, India can deepen the global dialogue on AI safety and bring global majority perspectives on human centric safety to the forefront of discussions.

Institutional reform

In designing the institute, India should learn from concerns stemming from MeitY’s AI Advisory in March 2024, which proposed that there be government approvals before the public roll-out of experimental AI systems. Some asked what kind of institutional capability the Indian government had to suitably determine the safety of novel AI deployments. Other provisions on bias, discrimination, and the one-size-fits-all treatment of all AI deployments indicated that the advisory was not based on technical evidence.

Similarly, India should be cautious and avoid prescriptive regulatory controls which have been proposed in the European Union (EU) and China. The threat of regulatory sanction in a rapidly evolving technological ecosystem quells proactive information sharing between businesses, governments, and the wider ecosystem. It nudges labs to only undertake the minimum steps towards compliance. Yet each jurisdiction demonstrates a recurring recognition of establishing specialised agencies — for example, China’s Algorithm Registry and the EU’s AI Office. However, to maximise the promise of institutional reform, India should decouple institution building from regulation making.

The Bletchley process is underscored by the U.K. Safety Summit in November 2023 and the South Korea Safety Summit in May 2024. The next summit is set for France and this process is yielding an international network of AI Safety Institutes.

The U.S. and the U.K. were the first two to set up these institutes and have already signed an MoU to exchange knowledge, resources, and expertise. Both institutions are also signing MoUs with AI labs and receiving early access to large foundation models. They have installed mechanisms to share technical inputs with the AI labs before their public roll outs. These Safety Institutes facilitate proactive information sharing without being regulators. They are positioned as technical government institutions that leverage multi-stakeholder consortiums and partnerships to assess the risk of frontier AI models to public safety. However, they largely consider AI safety through the lens of cybersecurity, infrastructure security, safety of the biosphere, and other national security threats.

These safety institutes aim to improve government capacity and mainstream the idea of external third-party testing and risk mitigations and assessments. Government-led AI safety institutes aim to deliver insights which can transform AI governance into an evidence-based discipline. The Bletchley process presents India with an opportunity to collaborate with governments and stakeholders from across the world. Shared expertise will be essential to keep up with AI’s rapid innovation trajectories.

Charting India’s approach

India should establish an AI Safety Institute which integrates into the Bletchley network of safety institutes. For now, it should be independent from rulemaking and enforcement authorities and, instead, operate exclusively as a technical research, testing, and standardisation agency. It would allow India’s domestic institutions to tap into the expertise of other governments, local multi-stakeholder communities, and international businesses. While upscaling its AI oversight capabilities, India can also use the Bletchley network to advance the global majority’s concerns with AI’s individual centric risks.

The institute could champion perspectives on risks relating to bias, discrimination, social exclusion, gendered risks, labour markets, data collection and individual privacy. Consequently, it could deepen the global dialogue around harm identification, big picture AI risks, mitigations, red-teaming, and standardisation. If done right, India may become a global steward for forward-thinking AI governance which embraces many stakeholders and government collaboration. The AI Safety Institute can demonstrate India’s scientific temper and willingness to implement globally compatible, evidence-based and proportionate policy solutions.

Sidharth Deb, Manager, Public Policy at The Quantum Hub, a public policy firm based in Delhi.