India's AI security framework: Will OpenAI, Anthropic have to comply?

India's proposed framework for auditing artificial intelligence (AI) systems seeks to bring greater accountability to AI developers and organisations deploying the technology. But questions remain over its legal enforceability, particularly when it comes to global technology companies such as OpenAI and Anthropic.

The Indian Computer Emergency Response Team (CERT-In) has proposed a security audit framework that requires AI-related security incidents to be reported within six hours. It classifies AI systems as low, moderate, high or critical risk, with critical systems facing continuous monitoring and independent audits by CERT-In-empanelled organisations.

The framework also prescribes safeguards for autonomous AI agents, including restrictions on the permissions they can access and a kill switch to shut them down when necessary.

While the proposed measures seek to address emerging cybersecurity risks, their implementation raises several questions. Will foreign AI companies have to submit to security audits in India? Will banks and other businesses using third-party AI tools also face compliance requirements? And could the framework increase costs for Indian AI startups?

Speaking to CNBC-TV18, Sudarshan Rajamani, Director, Dexian India, and Virag Gupta, Supreme Court Advocate and Cyber Law Expert, discussed the legal and practical challenges surrounding the proposed framework.

Are CERT-In's AI security guidelines legally binding?

A key question is whether the proposed framework has the legal backing required to make compliance compulsory for AI companies.

Gupta questioned its enforceability, pointing out that the 107-page document refers to guidelines, advisories and blueprints but, in his assessment, does not have sufficient legal backing.

He identified three areas that need to be addressed through appropriate legal provisions.

The first concerns the reporting of AI-related security incidents. Gupta argued that such incidents should also be reported to the police because law and order is a state subject under the Constitution. However, he pointed out that the police did not figure in the list of regulators referred to in the document.

The second relates to the use of data by AI systems. Since these systems are built on data, questions around authorised data, sensitive personal information and potential misuse become important. Gupta also flagged the fact that the Digital Personal Data Protection (DPDP) Act, passed by Parliament, has not been implemented.

His third concern was the need to address AI-related crimes separately rather than simply classifying them alongside conventional cybercrimes.

These concerns raise a larger question about the relationship between CERT-In's proposed security framework and existing laws. The guidelines may set out expectations for AI security, but their enforceability and the action authorities can take against companies that fail to comply remain areas of uncertainty.

Will OpenAI, Anthropic and other foreign AI companies have to comply?

The framework's applicability to foreign AI developers is particularly important as companies such as OpenAI, Anthropic and Google provide AI services to Indian customers.

Rajamani said the requirements should apply irrespective of a company's size or where it is headquartered. If an AI provider offers services to Indian customers or processes Indian data, there is a regulatory interest in ensuring security and accountability.

However, he was less certain about whether global AI companies would agree to independent security audits by organisations empanelled by CERT-In.

Asked whether companies such as OpenAI and Anthropic would submit themselves to such audits in India, he said: "Not really. At the moment, I would say no."

He added that foreign AI companies could not automatically consider themselves outside India's cybersecurity framework simply because their headquarters were overseas. However, he said the mandatory applicability of the proposed framework and its enforcement mechanism still needed clarification.

Gupta approached the issue from the perspective of existing Indian laws. He argued that foreign AI companies operating in India should also be covered by the Information Technology Act and the IT Intermediary Rules.

He said companies such as Anthropic should be treated as intermediaries and comply with requirements under the rules, including those relating to grievance redressal and adherence to Indian laws. Gupta also questioned whether these AI companies had adequately complied with the existing intermediary rules.

What about banks and businesses deploying third-party AI tools?

The proposed framework is not limited to companies developing AI models. Organisations procuring and deploying third-party AI tools could also face compliance questions, particularly around the use and protection of sensitive data.

Banks and government organisations, for instance, need to ensure that the data made available to AI systems is properly authorised and does not expose confidential information.

Gupta pointed to provisions in the framework dealing with the disclosure of sensitive information, leakage of confidential data and the requirement that data used for AI development and operations be appropriately authorised.

This raises a practical question for organisations using foreign AI platforms: how can a bank or government body authorise an external AI company to access and use its data?

Gupta argued that these issues need to be examined alongside the DPDP Act and existing IT laws. For businesses, deploying an AI tool is therefore not just a technology decision. They also need to consider whether the data being accessed and processed is authorised and whether its use complies with applicable legal requirements.

Why does CERT-In want kill switches for AI agents?

AI agents present a different kind of cybersecurity challenge because they can access information, make decisions and execute tasks with limited human intervention. If compromised, they could potentially carry out harmful actions before an organisation realises that its systems have been breached.

CERT-In's proposed framework addresses this through two important safeguards.

The first is the principle of least privilege. This means an AI agent should receive only the permissions needed to perform its assigned tasks. It should not automatically inherit access rights from a user, another tool or another AI agent.

The second is a kill switch for high-risk agentic AI systems. This would allow operators to shut down an agent if it behaves unexpectedly or poses a security threat.

Rajamani said the requirement assumes greater significance following the AI-related cybersecurity incidents witnessed during what experts have described as Black October. However, he cautioned that regulatory prescriptions alone would not be enough to address the risks.

Organisations will also need cybersecurity professionals with expertise in prompt injection testing, AI red-teaming, agentic AI security, model risk assessments and incident response.

"Unless we develop the right talent and technical capabilities, compliance will remain largely a paper exercise," Rajamani said.

Will the framework increase compliance costs for Indian AI startups?

The proposed security requirements could also raise concerns for smaller Indian companies developing AI models and applications. Unlike large technology companies, startups may not have the budgets or specialised teams needed to undertake extensive security assessments and audits.

Rajamani, however, does not believe the framework will deter India's AI ambitions.

He pointed to the country's shift from being primarily a consumer of global AI technologies to developing its own AI capabilities. Government investments in AI infrastructure, foundation models, startups and talent development are supporting this transition.

He cited BharatGen as an example of efforts to develop AI solutions suited to India's languages and industries.

But he also cautioned that AI adoption was moving faster than the maturity of cybersecurity capabilities. Stronger AI governance, skilled professionals, independent security assessments and clear accountability will be necessary to support the industry's growth.

For Rajamani, India's opportunity is not just to become an AI powerhouse but also to emerge as a global leader in secure, trusted and responsible AI.