The Supreme Court of India has asked the government to consider a plea seeking greater safeguards for the use of high-risk artificial intelligence in public-welfare programmes. The concern is that AI systems increasingly influence decisions affecting citizens' access to government services, benefits and entitlements. When such systems make or significantly influence decisions involving vulnerable people, errors or hidden biases can have serious consequences. The development adds to a growing Indian debate over whether existing technology and data-protection rules are sufficient for high-impact AI.
The central issue is accountability when AI is used by the state. An automated system may assess eligibility, identify beneficiaries, detect suspected fraud or prioritize applications, but citizens may not know that AI influenced the decision—or understand why they were rejected. This creates difficult questions around transparency, explainability, human review and the right to challenge an automated decision. Recent Indian legal discussions have similarly called for high-risk AI systems used in areas such as welfare, healthcare, policing and justice to face stronger scrutiny.
The case is significant because public-welfare AI affects fundamental interests rather than merely commercial convenience. A recommendation algorithm getting something wrong on a shopping platform may be frustrating; an automated welfare system incorrectly excluding a person from a government programme can directly affect their livelihood or access to essential support. This makes human oversight particularly important. India's constitutional framework also gives courts a role in examining whether government systems affecting rights operate fairly, transparently and within legal boundaries.
The broader message is that India may be moving toward a risk-based approach to AI governance, where the level of regulation depends on how consequential an AI system's decisions are. Low-risk applications may need relatively limited oversight, while AI used for welfare delivery, healthcare, courts, employment or other rights-sensitive areas could require impact assessments, audits, transparency and meaningful human intervention. The Supreme Court's intervention therefore highlights an important principle: AI can make government more efficient, but efficiency cannot come at the expense of fairness, accountability and citizens' ability to challenge decisions that affect them.
