Introduction
“Justice should not only be done, but should manifestly and undoubtedly be seen to be done.”
– Lord Hewart CJ, R v. Sussex Justices (1924)
The core of criminal justice is the correct discovery of facts and the judicious use of law. With the advent of AI-powered tools, judgments can be searched, proceedings transcribed and translated, and forensic material analysed, thus easing long-standing bottlenecks to a fair trial.1 However, a model developed from biased policing data can perpetuate bias; a risk score might give a false sense of security; and a generative tool might invent authorities. The question is not whether AI will make its way into the justice system, but when and how. This article rests on three propositions. Assistance is not adjudication: a decision must rest on independent assessment and on written reasons. Contestability is part of the fair trial: a system that cannot be tested should not have any material effect on a decision on liberty. Responsibility cannot rest with anyone but a human or institutional actor. These propositions are not abstract. When ProPublica tested the COMPAS risk-assessment tool in the United States, it found that, among defendants who did not reoffend, the tool was about twice as likely to classify a black defendant as “high risk” as a white defendant. In State v. Loomis, the Wisconsin Supreme Court ruled against the defendant even though neither the court nor the defendant had been told anything about the proprietary methodology behind COMPAS. These cases illustrate that algorithmic opacity is not a mere technical inconvenience but a structural problem for the adversarial testing to which fair-trial guarantees are committed.
There is no legislation in India dedicated to the validation, disclosure and use of AI in criminal justice. While the Digital Personal Data Protection Act, 2023 provides a general framework for data fiduciaries, there is no specific law on AI within the criminal justice system. The regulation that exists is court-related only; none of it addresses the police, forensic laboratories or prosecutors. The draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026, issued on 3 June 2026, are stricter still: they would prohibit judicial outcomes reached solely on the basis of AI, and they would exclude AI risk scoring for bail, recidivism and credibility. These instruments do not, however, reach the upstream stages of investigation and prosecution, where algorithmic outputs may already have influenced who is arrested, what charge is brought and the defence’s bail argument by the time a judicial officer receives the file.
This study is doctrinal as well as comparative. It relies on the text of the Constitution, legislation, judicial rulings and official records. The United States is selected because Loomis remains the leading appellate case on a proprietary risk instrument, and because the COMPAS controversy provides the most complete case study in the literature of algorithmic bias in criminal justice. The European Union is selected because it has the most advanced statutory framework, the AI Act covering AI systems used by judicial authorities in researching and interpreting facts and law. No empirical data on police or forensic AI in India is publicly available.
AI in criminal adjudication
Uses vary significantly in the legal context: administrative tools (scheduling, transcription), research and drafting aids, evidence-generating tools (facial or voice matching, enhanced images, probabilistic DNA software) and liberty-determinative tools (bail and sentencing scores, predictive policing). The EU AI Act follows this gradation, because safeguards should be proportionate to the tool’s influence on the decision and to the ease with which error can be detected.2 In March 2025, the Government told Parliament that AI is being used in case management and transcription and that SUPACE is at an experimental stage, but that no AI tool is used by the Supreme Court in decision-making.3 The White Paper of the Supreme Court’s Centre for Research and Planning treats AI as an aid to its user and identifies hallucination as one of the primary risks. None of this supplies grounds for permitting automated adjudication.
The distinction matters because the legal question varies with the use. An error in transcription can be corrected against the audio recording; a recidivism score that wrongly labels an accused cannot be corrected against anything, because predicting behaviour in an individual case lies beyond what the score can do. Protection should be most stringent where errors are hardest to discover and most damaging to the individual involved. Just as the work product of the court deserves scrutiny, the work product of AI should be subjected to the same rigour. After all, what it produces is already in the court, in the guise of “fact.”
Constitutional framework
A. Article 14
Article 144 forbids arbitrariness and covers indirect discrimination arising from facially neutral measures.5 Algorithmic proxies fall within this doctrine: neighbourhood, employment history or previous police contact can stand in for caste or religion. The risk is immediate where enforcement is unequal, and official statistics show disproportionately high numbers of Scheduled Castes, Scheduled Tribes and Muslims among undertrials.6 A score is not neutral merely because it is numerical.
B. Article 21
As held in Maneka Gandhi, a procedure depriving a person of liberty under Article 217 must be just, fair and reasonable.8 First, there is an obligation to provide a fair trial to the accused, the victim and society, and speed is no justification for a failure to disclose.9 Where the accused has a right to the documents relied upon, disclosure of an algorithmic output should include its inputs, method and error rates.10 Second, bail and sentencing must be individualised; a group-based recidivism score indicates how people like the accused behave, not what the accused will do, and is therefore suspect as a substitute for individual enquiry.11 Third, as held in K.S. Puttaswamy, an interference with privacy must be lawful, pursue a legitimate aim, be proportionate and be accompanied by safeguards.12
C. Articles 20(3), 22 and 39A
In Selvi, the Supreme Court held that involuntary narcoanalysis, polygraph and brain-mapping tests violate Articles 20(3)13 and 21, partly because their reliability is doubtful.14 The reasoning extends to AI tools claiming to infer deception or intent from voice, face or physiology. A lawfully taken voice sample is not testimony,15 but that does not make a downstream algorithmic inference reliable. Articles 2216 and 39A,17 read with Article 21, secure legal assistance;18 where AI output is relied on, that requires access to the underlying material and, where needed, independent expert help, or the State alone can interrogate algorithmic evidence.
Bias, opacity and automation bias
Bias can be introduced through the data, the selection of data, the outcome predicted (rearrest measures policing as well as offending) or human interpretation. In the COMPAS controversy, ProPublica found that black defendants who did not reoffend were misclassified as high risk at nearly twice the rate of white defendants, while the developer responded that the scores were equally well calibrated for both groups. Both claims can hold where base rates differ; the choice between them is a constitutional decision, not one for the vendor.19
An automated system can become a de facto decision-maker; this is what automation bias does.20 The difficulty is compounded by proprietary systems, where trade-secret claims are raised against defendants.21 The BNSS power to summon documents offers a pathway to disclosure under protective orders.22
Electronic evidence under the BSA, 2023
The BSA treats an electronic record as a document. Section 61 bars its exclusion merely because it is electronic and gives it the same effect as a paper document;23 Section 6224 provides that the contents of electronic records may be proved in accordance with Section 63; and Section 63 governs computer output, including the certificate with hash-value information.25 However, reliability is not admissibility. A Section 6326 certificate shows that a file originated from a properly operating device, but it does not show that the contents are true or that an AI inference drawn from them is valid. Deepfakes and AI-enhanced images require their source to be examined, with expert skill, under Section 39 of the BSA.27 India has no codified Daubert standard and, traditionally, evidence is admissible even when improperly obtained,28 so an express power to exclude unvalidated AI output is essential.
The BNSS introduces integrity safeguards such as audio-video recording of searches and seizures, the visit of a forensic expert to the crime scene in serious offences, and trials in electronic mode.29 These help establish where a digital file came from, the first question any AI analysis must answer. They do not address whether the analysis is sound, which is why validation and disclosure duties must sit alongside them.
Data protection
While Section 17 of the DPDP Act exempts processing by courts, and processing for the prosecution of offences, from most of the Act’s obligations, it grants no general licence to private AI vendors. Sections 3 to 17 (save Section 6(9), in force from 13 November 2026) commence on 13 May 2027 under notification G.S.R. 843(E) of 13 November 2025; they are therefore enacted but were not in force on 1 November 2026.30 In the interim, Puttaswamy governs State action: confidential witness or victim material should not be uploaded to public generative AI services.
Comparative perspective
A. United States
Loomis upheld a sentence informed by COMPAS, but it held that the score may not determine whether an offender is incarcerated or the severity of the sentence, and it required the pre-sentence report to warn of the tool’s proprietary nature, its reliance on group data, and its lack of validation on a Wisconsin population.31 The decision is cautionary rather than exemplary. It tolerates a transparency deficit that Indian fair-trial doctrine would struggle to accept, and it relies on judicial discipline to keep a “non-determinative” score from becoming decisive in practice.
B. European Union
The AI Act prohibits predicting offending solely from profiling;32 it classifies certain law-enforcement and judicial-assistance systems as high-risk under Annex III, and it imposes requirements of risk management, transparency and human oversight, with specific attention to automation bias (arts. 9–15, 14(4)(b)), and a right to explanation (art. 86). The Digital Omnibus on AI defers the Annex III obligations from 2 August 2026 to 2 December 2027.33 For India, the lessons are that risk-tiering works, that disclosure needs institutional support to be meaningful, and that regulation must reach law enforcement. The Act is not binding in India.
C. Council of Europe and the United Kingdom
The CEPEJ Ethical Charter and the Council of Europe Framework Convention on AI place fundamental rights, non-discrimination and transparency at the centre of judicial AI.34 In Bridges, the Court of Appeal of England and Wales found the police use of live facial recognition unlawful for want of a sufficiently clear legal framework and a proper equality assessment.35 The case shows that “legality” in the Puttaswamy sense is distinct from the mere absence of a prohibition.
Indian developments
On 19 July 2025, the High Court of Kerala prohibited the district judiciary from using AI tools to arrive at findings, reliefs, orders or judgments.36 On 3 June 2026, the Supreme Court’s AI Committee published draft regulations which would, among other things, prohibit AI from determining outcomes, bar the profiling of parties and witnesses, and mandate disclosure of the use of AI.37
In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. (2 July 2026), the Supreme Court set aside the judgments of the National Company Law Tribunal and the National Company Law Appellate Tribunal, which rested on non-existent and mis-cited authorities, and held that reliance on fabricated material vitiates the decision.38 Although it is an insolvency matter, the principle applies with greater force where liberty is at stake: AI-generated content cannot be taken as fact because it is presented with confidence.
Earlier in 2026, in another matter, the Supreme Court described a trial-court order that relied on judgments which do not exist as raising “institutional concern.”39 Taken together, these decisions create a judicial obligation to verify, and make clear that good faith cannot cure false output. In criminal trials, the duty is the same for court, prosecution and defence, where a bail order or sentence may be based on an unverified summary/citation.
The regulatory gap
There is no blanket rule in India that high-impact AI deployed in criminal proceedings must be independently tested, have its limitations published, keep audit logs or be open to challenge.40 The focus is on the courtroom, but the bail and charge arguments made by the police, forensic laboratories and prosecutors may already have been influenced by AI by the time they appear before the court. Responsibility is divided among the developer, the procuring authority and the operator or adjudicator, and procurement is conducted in confidence. Human judges are biased as well, and that bias can be challenged and appealed, but not quite in the ways that statistical bias can be. Vendor secrecy, while commercially sound, can be counterbalanced by the accused’s means of challenge, and in-camera review and court-appointed experts bridge the gap. Two more objections have to be answered. First, a validated tool may reduce unnecessary detention, and the framework below does not prohibit all use; but a tool used only to support detention would upset the settled presumption in favour of release, so its effect on release rates by group should be monitored. Second, regulation need not retard beneficial innovation: the burden is light for administrative instruments and grows with the cost of error. Formal equality is not equality of arms if the State uses a sophisticated tool and the accused has no opportunity to inspect or contest it.
Algorithmic fair-trial framework
This article proposes the following safeguards for any AI that can have a material effect on investigation, bail, evidence evaluation or sentencing:
1. legality and necessity, with profiling-only prediction of criminality and deception detection of unwilling subjects prohibited;
2. risk classification distinguishing administrative from liberty-affecting systems;
3. independent validation on representative Indian data, including group-level error rates;
4. disclosure sufficient to contest method, inputs and limits, under protective orders where needed;
5. human control, with no score determinative and reasons recorded;
6. evidence integrity through provenance and chain-of-custody records and a power to exclude unvalidated output;
7. privacy and security controls;
8. remedies and accountability; and
9. legal aid and technical assistance for the indigent accused.
These consolidate into an Algorithmic Fair-Trial Test:
• Was the system developed and used lawfully, for a proper purpose?
• Is it accurate and reliable in this context?
• Has bias and disparate impact been assessed?
• Was there disclosure and a real opportunity to contest?
• Did an independent human decision-maker give reasons on the material?
Where any element is missing, the output should not determine, or materially influence, a decision on liberty. Implementation may proceed through finalised court regulations, interim Supreme Court guidelines and the rules available under the BNSS and the BSA, and then through a consolidating statute.
Conclusion
The Constitution does not require courts to abandon technology, but it demands that technology be consistent with equality, privacy, liberty, the privilege against self-incrimination and fair procedure. The BSA does not treat an admissible electronic record as reliable proof, while Loomis and the EU AI Act demonstrate the necessity of transparency, validation and oversight. Pooja Ramesh Singh draws the boundary in the courtroom; the next step is to take it upstream to the police, laboratories and prosecutors. Technology should serve justice, and justice should not be made to serve technology.
*****
Footnotes
1. Anurag Bhaskar et al., Ctr. for Rsch. & Plan., Sup. Ct. of India, White Paper on Artificial Intelligence and the Judiciary (2025), https://cdn.s3waas.gov.in/s3ec0490f1f4972d133619a60c30f3559e/uploads/2025/11/2025112244.pdf.
2. Regulation 2024/1689, of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence, recital 61, annex III, pts. 6, 8, 2024 O.J. (L 1689) [hereinafter AI Act].
3. Use of AI in Supreme Court Case Management, Press Info. Bureau (Mar. 20, 2025), https://www.pib.gov.in/PressReleaseIframePage.aspx?PRID=2113224®=48&lang=2 (last visited Nov. 1, 2026).
4. India Const. art. 14.
5. E.P. Royappa v. State of Tamil Nadu, (1974) 4 SCC 3 (India); Nitisha v. Union of India, (2021) 15 SCC 125 (India).
6. NCRB Data Shows Muslims, Dalits, Tribal Population in Prisons Disproportionate to Their Numbers Outside, Firstpost, https://www.firstpost.com/india/ncrb-data-shows-muslims-dalits-tribal-population-in-prisons-disproportionate-to-their-numbers-outside-8775161.html (last visited Oct. 31, 2026).
7. India Const. art. 21.
8. Maneka Gandhi v. Union of India, (1978) 1 SCC 248 (India).
9. Hussainara Khatoon v. Home Sec’y, State of Bihar, (1980) 1 SCC 81 (India); Zahira Habibulla H. Sheikh v. State of Gujarat, (2004) 4 SCC 158 (India).
10. The Bharatiya Nagarik Suraksha Sanhita, 2023, § 230, No. 46, Acts of Parliament, 2023 (India).
11. Sanjay Chandra v. CBI, (2012) 1 SCC 40 (India); Bachan Singh v. State of Punjab, (1980) 2 SCC 684 (India).
12. K.S. Puttaswamy v. Union of India, (2017) 10 SCC 1 (India).
13. India Const. art. 20, cl. 3.
14. Selvi v. State of Karnataka, (2010) 7 SCC 263 (India).
15. Ritesh Sinha v. State of Uttar Pradesh, (2019) 8 SCC 1 (India).
16. India Const. art. 22.
17. India Const. art. 39A.
18. M.H. Hoskot v. State of Maharashtra, (1978) 3 SCC 544 (India).
19. Jon Kleinberg, Sendhil Mullainathan & Manish Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores, in 67 LIPIcs: Leibniz International Proceedings in Informatics 43:1 (2017) (Proceedings of the 8th Innovations in Theoretical Computer Science Conference, ITCS 2017), https://doi.org/10.4230/LIPIcs.ITCS.2017.43.
20. Case C-634/21, OQ v. Land Hessen (SCHUFA Holding), ECLI:EU:C:2023:957 (Dec. 7, 2023).
21. Rebecca Wexler, Life, Liberty, and Trade Secrets: Intellectual Property in the Criminal Justice System, 70 Stan. L. Rev. 1343 (2018).
22. The Bharatiya Nagarik Suraksha Sanhita, 2023, § 94, No. 46, Acts of Parliament, 2023 (India).
23. The Bharatiya Sakshya Adhiniyam, 2023, § 61, No. 47, Acts of Parliament, 2023 (India).
24. The Bharatiya Sakshya Adhiniyam, 2023, § 62, No. 47, Acts of Parliament, 2023 (India).
25. The Bharatiya Sakshya Adhiniyam, 2023, § 63, No. 47, Acts of Parliament, 2023 (India).
26. Anvar P.V. v. P.K. Basheer, (2014) 10 SCC 473 (India); Arjun Panditrao Khotkar v. Kailash Kushanrao Gorantyal, (2020) 7 SCC 1 (India).
27. The Information Technology Act, 2000, § 79A, No. 21, Acts of Parliament, 2000 (India).
28. Pooran Mal v. Dir. of Inspection (Investigation), (1974) 1 SCC 345 (India).
29. The Bharatiya Nagarik Suraksha Sanhita, 2023, §§ 105, 176(3), 530, No. 46, Acts of Parliament, 2023 (India).
30. The Digital Personal Data Protection Act, 2023, §§ 3–17, No. 22, Acts of Parliament, 2023 (India); Ministry of Electronics & Info. Tech., Notification G.S.R. 843(E) (Nov. 13, 2025), Gazette of India, Extraordinary, pt. II, sec. 3(i).
31. State v. Loomis, 881 N.W.2d 749 (Wis. 2016), cert. denied, 137 S. Ct. 2290 (2017).
32. AI Act, supra note 2, art. 5(1)(d).
33. Barbara Angene, The Digital AI Omnibus: Proposed Deferral of High Risk AI Obligations Under the AI Act (Update), DLA Piper GENIE (June 30, 2026), https://knowledge.dlapiper.com/dlapiperknowledge/globalemploymentlatestdevelopments/2026/The-Digital-AI-Omnibus-Proposed-deferral-of-high-risk-AI-obligations-under-the-AI-Act (last visited Nov. 1, 2026).
34. Eur. Comm’n for the Efficiency of Just. (CEPEJ), European Ethical Charter on the Use of Artificial Intelligence in Judicial Systems and Their Environment (Dec. 2018); Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, C.E.T.S. No. 225 (2024).
35. R (Bridges) v. Chief Constable of S. Wales Police [2020] EWCA Civ 1058 (Eng.).
36. K. Salma Jennath, ‘AI Tools Not Be Used to Arrive at Judgments’: Kerala High Court Issues Guidelines for AI Use in District Judiciary, LiveLaw (July 19, 2025), https://www.livelaw.in/high-court/kerala-high-court/kerala-high-court-ai-use-district-judiciary-298139 (last visited Nov. 1, 2026).
37. Sup. Ct. of India, Notice, Draft Regulations for Use of Artificial Intelligence (AI) in Courts, 2026, regs. 20(1)(b), (d), (f), 43 (June 3, 2026); Arushi Meshram & Alok Kumar, Analyzing Liability in 2026 Draft Regulations for Using Artificial Intelligence in Courts & Suggestions, LiveLaw (June 17, 2026), https://www.livelaw.in/articles/analyzing-liability-2026draft-regulations-using-artificial-intelligence-courts-538076 (last visited Nov. 1, 2026).
38. Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., Civil Appeal No. 11950 of 2025, 2026 INSC 668 (India).
39. Supreme Court Takes Cognisance of Trial Court Relying on AI-Generated ‘Fake’ Verdicts, The Hindu (Mar. 2, 2026), https://www.thehindu.com/news/national/supreme-court-takes-cognisance-of-trial-court-relying-on-ai-generated-fake-verdicts/article70694926.ece.
40. Use of AI Based Digital Content in Judiciary, Press Info. Bureau (Dec. 5, 2025), https://www.pib.gov.in/PressReleasePage.aspx?PRID=2199313®=48&lang=2 (last visited Nov. 1, 2026).
