The Invisible Algorithm: Why People Cannot Challenge Public Decisions They Don’t Know AI Helped Make

1. What Public Law Project actually found

On 20 October 2026, the Public Law Project (PLP) published Missing Pieces: Transparency and Accountability in Public Body Automated (and Automated Assisted) Decision-making, written by Caroline Selman and Joseph Summers with Nuffield Foundation funding. It is a study of information access and legal accountability when public authorities use algorithms to make, frame or inform decisions.

The researchers did not ask simply whether AI was technically impressive. They asked whether a person affected by a decision can find out if an automated system was used, how it affected their case and what information is available in time to challenge it. That is a public-law question with practical consequences for legal aid, discrimination claims and judicial review.

The report’s 125-register-entry review was undertaken in February 2026. Its methodology records that another 27 entries had been added by publication. It would therefore be inaccurate to present 125 as the entire October 2026 register. The authors also distinguish public bodies’ actual restrictions from participants’ perceptions of institutional resistance.

Among public-body interviewees, the report found support for transparency alongside real concerns: technical capacity, the information controlled by third-party suppliers, staff skills, security and operational effectiveness. That nuance is vital. A finding of inadequate disclosure does not, without more, establish deliberate concealment.

Read the Public Law Project’s report overview and the original 61-page report.

2. The invisible chain: how an algorithm can affect an apparently human decision

Much public debate imagines an AI delivering a final verdict: approve, refuse, investigate or remove. That is not the only, or even necessarily the most consequential, way automation can influence a public decision. A system can determine which cases receive attention, which data appear in a summary, what is prioritised, or which options the human is invited to consider.

Five points of influence

1. Data collection: information is pulled from records, communications, databases or referrals. Incomplete or inaccurate source records enter the pipeline.
2. Classification: a model or rules engine assigns categories, confidence scores or labels to the data.
3. Triage and prioritisation: the system determines which cases enter which workflow, how urgently they are reviewed, or whether a human considers them at all.
4. Presentation: a dashboard, summary or risk indicator frames what the decision-maker sees. The original material may be technically accessible but not actually examined.
5. Human adoption: a professional accepts, edits or rejects the suggested course. The decision is signed by a human, but its informational foundations may have been machine-shaped.

This is a generic analytical model, not a description of a verified Cafcass or Family Court product. It explains why transparency cannot stop at the name of the official who signed the final decision.

An algorithm might produce an accurate score from flawed inputs. A human might exercise genuine independent judgment yet rely on a summary omitting important context. Or a competent reviewer might challenge the output and reach a wholly different conclusion. The legal implications depend on actual design, data, governance and use; the phrase ‘AI was involved’ is not enough.

A forensic provenance chain

Without that chain, an appeal against a final conclusion may fail to reveal where the problem first entered the decision-making process. This is why explainability must be considered in operational terms, not simply as a reassuring paragraph in a procurement policy.

3. The Algorithmic Transparency Recording Standard: mandatory policy is not universal law

The government’s Algorithmic Transparency Recording Standard (ATRS) guidance provides a framework for publishing information about tools used in public-impact decision-making. It is broader than generative AI: it includes statistical models, complex algorithms and other technologies supporting operational decisions.

Under the current government policy, ATRS publication is mandatory for government departments and specified arm’s-length bodies providing public or frontline services or directly interacting with the public, for tools falling within the defined decision-impact or public-interaction scope. It is recommended more broadly, but is not automatically mandatory for every local authority, school, charity or private supplier.

A mandatory government policy is also not the same as a comprehensive statutory right enforceable by every affected individual. PLP therefore recommends giving the transparency regime a statutory footing, including stronger monitoring and enforcement.

What an adequate register entry should help answer

• Why is the tool used, and which administrative task or public service does it affect?
• Who owns the decision process, who supplies the technology, and what are their responsibilities?
• What inputs and operational data does the system rely on?
• What role does the human operator play, including review and override?
• What evaluations have been conducted for accuracy, discrimination, limitations and misuse?
• Which information cannot safely be made public, and why?

Government guidance itself recognises legitimate security and intellectual-property concerns, but generally prefers proportionate redaction with explanation over withholding an entire account. Information about who makes a decision and how a tool affects it is often a different matter from publishing source code or revealing security-sensitive detection thresholds.

4. The 2026 UK GDPR trap: a human signature can take a decision outside Article 22C

Here is one of the most legally important details in the report. The Data (Use and Access) Act 2025 replaced the earlier UK GDPR Article 22 framework. According to the Information Commissioner’s Office, all the Act’s data-protection changes were in force by 19 June 2026. The current provisions are Articles 22A to 22D, rather than the old Article 22 formulation often repeated by out-of-date websites.

Under section 80 of the Data (Use and Access) Act 2025, a decision is ‘solely’ automated for these provisions if it involves no meaningful human involvement. A decision is ‘significant’ if it has a legal or similarly significant effect on the person. Article 22B places particular restrictions on certain significant solely automated decisions, including those involving special-category personal data; Article 22C requires safeguards for qualifying significant solely automated decisions.

Those safeguards include information about the decision, an opportunity to make representations, the ability to obtain human intervention and the right to contest it. The reforms permit a wider range of solely automated significant decisions in other circumstances, subject to the relevant rules and safeguards.

But human-assisted automated decisions can fall outside these additional Article 22C safeguards if there is meaningful human involvement. That does not make all other processing lawful or immune from challenge. The UK GDPR’s wider principles concerning lawful, fair, transparent, accurate and appropriately secure processing remain relevant, alongside other legislation and public-law obligations.

A decision-making continuum

This creates a difficult governance question: if adding a meaningful reviewer removes a decision from the strictest statutory safeguards, what information must still be made available to show that the review was meaningful? PLP argues for extending suitable notification and information rights to automated-assisted decisions.

The answer is not to treat all human-assisted AI as secretly unlawful. It is to recognise that the distinction between the AI output and the final human decision may be hard for an affected person to test without a trace of what the human actually saw and did.

5. The discrimination problem: an assessment that is not visible is hard to challenge

PLP found that just one of 125 ATRS entries reviewed linked to an Equality Impact Assessment, while five more summarised one. That is a finding about information visible in register entries, not a determination that 124 organisations had failed to assess equality effects.

Under section 149 of the Equality Act 2010, public authorities exercising covered functions must have due regard to specified equality objectives, including eliminating unlawful discrimination, advancing equality of opportunity and fostering good relations. The obligation concerns decision-making practice, not merely producing a document titled ‘Equality Impact Assessment’.

For a technically sophisticated system, meaningful evaluation may need to examine error rates, false positives and false negatives, representativeness, differences between groups, operational changes and whether a disadvantaged person can obtain a correction. A crude overall accuracy figure can conceal unacceptable outcomes for smaller groups.

An error-rate example — illustrative, not a finding from PLP

Imagine a hypothetical tool correctly classifies 95 of every 100 referrals. That headline accuracy may sound reassuring. But if its errors cluster among disabled parents, particular language groups or people whose interactions with institutions are poorly documented, the fairness consequences could be unequal.

Accuracy is also not interchangeable with legality. A score may be statistically well calibrated but inappropriate for the precise statutory question. A risk category developed for allocating administrative resources must not silently become a factual finding or a legal threshold.

A published link to an assessment will not itself make an algorithm fair. But allowing affected individuals, representatives and independent researchers to see relevant evaluation and limitations can make challenge and correction possible.

6. Why this belongs in a family justice publication — without inventing a Cafcass scandal

The PLP investigation is principally about public-body automated decision-making across public services. It does not establish that Cafcass secretly uses a particular algorithm to determine child arrangements, that social workers currently rely on an unnamed automated risk model, or that a family judge has delegated welfare decision-making to AI. Those claims would require separate evidence.

The family-justice implications are nonetheless concrete. Information created by local authorities, police, schools and other public bodies may enter the evidential environment surrounding a child. Where technology is used to process or prioritise information inside a relevant institution, any failure to preserve provenance, qualifications or corrections can affect what another professional later receives.

Four distinct decision-makers; four distinct legal questions

Under section 1 of the Children Act 1989, the child’s welfare is paramount when the court determines specified questions concerning upbringing. Where domestic abuse is relevant to applicable child arrangements proceedings, Practice Direction 12J requires proper attention to allegations, disputed facts, harm and welfare. No generic risk score can substitute for that legal analysis.

A public authority’s administrative decision is not the same thing as the Family Court’s judicial order. The routes to challenge an inaccurate public record, an unlawful council decision, a professional assessment and a family court judgment are different. They must not be conflated.

A carefully labelled hypothetical

Suppose a local authority adopts a hypothetical technology tool to triage safeguarding referrals. The system flags certain records as high priority because of keywords and earlier database entries. A social worker then reviews the file and writes a report. Later, some information from that report is placed before the Family Court.

The immediate questions are not whether the algorithm is malicious or whether the parent’s account should automatically prevail. They are whether the original records were accurate, whether the worker knew the output’s limitations, what independent checks were performed, whether contradictory information was considered and how the court can distinguish the model’s flag from verified evidence.

In a coercive-control case, automated summarisation may miss context and pattern. In another case, a properly designed triage tool might help locate genuinely urgent concerns earlier. Both outcomes are plausible. Responsible analysis requires evidence about the actual system, its design and its use.

7. Freedom of information, subject access, disclosure and judicial review are not interchangeable

People often receive well-meant advice to ‘send a subject access request’ whenever they suspect an inaccurate or automated decision. A subject access request may be useful, but it does not necessarily reveal a model’s logic, supplier contract, testing data or the full decision-making pathway.

Ordinary FOI requests generally have a 20-working-day statutory response framework under section 10 of the Freedom of Information Act 2000, subject to its qualifications. Under ICO guidance, organisations generally must respond to a valid subject access request without undue delay and within one month, subject to applicable rules. Neither clock is a substitute for a litigation timetable.

The timing trap

Under CPR 54.5, judicial review claims must generally be filed promptly and in any event within three months after the grounds first arose, subject to shorter rules in some fields and the court’s power to extend time in appropriate cases. A pending FOI request, complaint or SAR does not automatically stop that clock.

This is the vicious circle identified by PLP: without information, a person may struggle to formulate a viable public-law challenge; without starting proceedings, they may struggle to obtain the disclosure needed to understand the system. Delayed access can therefore become a barrier to access to justice.

Because these deadlines and procedural routes can be consequential, potential judicial-review questions require urgent advice from an appropriately authorised public-law specialist. JSH Law’s family litigation support does not extend to conducting reserved judicial-review litigation.

For readers dealing specifically with Cafcass records, our guide to subject access and case notes explains a separate, appropriate route and its limitations. Do not use it as a template for demanding unrelated third-party information or protected children’s records.

8. The defence of secrecy: safety, commercial confidence and public trust

Public bodies do have legitimate reasons to withhold some technical detail. Revealing exact fraud-detection thresholds might enable evasion. Publicly exposing security architecture could create vulnerabilities. Personal information about children or third parties must not be casually disclosed. Supplier intellectual property may sometimes justify carefully limited restrictions.

Those arguments deserve a serious answer, not the assertion that every refusal proves misconduct. Equally, ‘commercial confidentiality’ must not become an automatic substitute for explaining the legal basis and consequences of a public decision.

What can be disclosed without releasing source code?

• The fact that a tool was used, its name, purpose and the public function affected.
• Whether the tool produced a recommendation, priority, score, classification or summary.
• The categories and provenance of data considered, without identifying protected third parties.
• The identity of the accountable public authority and the extent of human review.
• Relevant aggregate validation, known limitations and any meaningful equality testing.
• The process by which a person can correct inaccurate data or challenge the result.

PLP discusses proportional arrangements including confidentiality rings in litigation. These can enable scrutiny of sensitive material in appropriate proceedings without making it public. Their availability depends on the court’s powers and particular circumstances; they are not an automatic entitlement.

The hard governance point is that suppliers and decision-makers should anticipate lawful scrutiny at procurement and design stage. A public authority should not promise meaningful accountability while signing a technology contract that leaves it unable to discover what the system did.

9. What can a parent or litigant in person actually do?

A concern that an institution may have used AI is not, by itself, evidence that its decision was unlawful. A useful response begins by identifying the actual problem and preserving the records that matter, rather than trying to litigate an assumed algorithm.

1. Identify the decision. Who made it, when, under what apparent power, with what written reasons and what practical effect?
2. Separate records from conclusions. Record the difference between a report of an allegation, a professional opinion, a database entry, a recommendation and a finding by a court.
3. Ask a proportionate factual question. Where appropriate, ask whether any automated tool materially supported triage, assessment, prioritisation or preparation of the decision, and seek the tool’s name and purpose.
4. Check public information. Examine the relevant body’s published policies and, where applicable, the government’s ATRS register. Absence from a register is not proof of covert or unlawful use.
5. Choose the right information route. Consider whether an FOI request, SAR, ordinary records request, complaint or case-specific disclosure request is legally appropriate.
6. Protect confidential material. Avoid publishing children’s information, protected addresses, allegations or private court papers to campaign about the issue.
7. Do not miss procedural deadlines. Seek specialist legal advice early if a decision may need a statutory appeal, judicial review or an application within ongoing proceedings.
8. Make the correction request specific. Identify the original record, disputed entry, source material, the correction or clarification sought and the decision potentially affected.

An illustrative neutral question

Please confirm whether any automated or algorithmic system, including AI-assisted classification, triage, summarisation or risk scoring, materially informed the preparation or making of the decision dated [date]. If so, please identify the tool, its purpose, the stage at which it was used, the categories of information considered and the extent of meaningful human review. Please also explain the appropriate route for requesting correction of inaccurate personal information and for seeking further information where legally available.

This is a suggested enquiry, not a legal demand that overrides information exemptions, family-court restrictions or a live court order. Do not ask a protected witness or child to supply information outside the proper process.

Where records are already known to be inaccurate, the decisive point may be the original error rather than the software. See our guide to challenging inaccurate school safeguarding records for a practical example of careful source attribution.

10. The safeguards a public body should be able to demonstrate

My proposal is a decision provenance and challengeability standard: enough records should exist for an authorised reviewer to reconstruct how a material administrative conclusion was reached without compromising protected information. It is a governance proposal, not an existing universal statutory requirement.

1. System inventory: identify each deployed model, version, accountable service and lawful purpose.
2. Data lineage: document major source categories, dates, changes, missing data and correction mechanisms.
3. Impact evaluation: test for accuracy, false-positive/negative outcomes, group differences and foreseeable safeguarding or accessibility harms.
4. Meaningful review: define when a human must examine original source material, when they may override outputs and how this is recorded.
5. Retention: maintain proportionate audit evidence while complying with data-minimisation and retention limits.
6. Accessible notification: explain when a system has materially contributed to an individual decision, subject to legitimate and proportionate restrictions.
7. Challenge route: provide a real contact, correction procedure and timely escalation path that does not depend on technical literacy.

The PLP authors recommend statutory reinforcement of the ATRS, stronger and more timely information rights, improved monitoring and retention, expert oversight and updated guidance on disclosure in litigation. They also identify a need for additional study of the relationship between statistical techniques and legal decision-making. This is a substantive reform agenda, not simply a call for more press releases.

Five difficult questions for government and legal technology suppliers

• If the public authority cannot explain a supplier’s system, who is accountable for a wrong decision?
• If an algorithm changes the order in which people receive human attention, has it already exercised meaningful influence?
• How should a public body prove human review was real rather than a signature on a machine-shaped output?
• Can a system comply with a narrow data-protection rule yet still produce an unlawful or procedurally unfair public decision?
• What must be recorded and disclosed so that a person without a lawyer can identify whether a challenge is worth bringing?

Efficiency gains and genuine public benefit may justify investment in automated tools. They do not answer whether the tools are safe, reviewable or lawful. Those questions must be confronted directly, not treated as an inconvenient afterthought.

11. Frequently asked questions

Does the report prove that Cafcass is secretly using AI to decide cases?

No. The report analyses public-body automated decision-making and information-access problems. It does not establish that any named Cafcass system secretly decides child arrangements or that a judge has delegated a welfare decision to a model.

Does a public authority have to tell me every time it uses AI?

There is no universal rule requiring individual notification of every AI-assisted step. Specific duties depend on the organisation, the kind of tool, the processing, relevant statutory regimes and whether a significant solely automated decision is involved. PLP calls for stronger notification rights.

Can I demand the algorithm’s source code under a subject access request?

A SAR principally concerns personal data about the requester and applicable related information rights. It is not an automatic right to obtain source code, all procurement documents or other people’s confidential records.

Is an AI-assisted decision unlawful because a human relied on a computer-generated score?

Not automatically. Lawfulness depends on the public body’s powers, applicable duties, quality of information, the decision-making process, any meaningful review, fairness and other relevant legal rules.

If I have evidence that an underlying record is wrong, should I focus on the algorithm?

Often the first practical step is to identify the inaccurate source record, preserve evidence of the error and use the appropriate correction or procedural route. The automated processing may matter if it amplified, circulated or relied upon that information.

Could a judicial review challenge an automated decision?

Potentially, if the correct public-law requirements are met and there is no more appropriate statutory route. This is specialist litigation with costs and strict timing issues. It is not an appeal on the merits merely because a claimant disagrees with the result.

Is secrecy sometimes justified?

Yes. Legitimate confidentiality, child protection, fraud prevention, national security, policing and supplier interests may constrain disclosure. The question is whether the restriction is lawful, necessary and proportionate, and whether meaningful accountability can be maintained.

12. Primary sources, research and relevant JSH Law reading

The research at the centre of this article

• Public Law Project: Missing Pieces (20 October 2026), Caroline Selman and Joseph Summers
• Full Missing Pieces report (PDF, 61 pages), especially methodology, register review, public-law transparency and recommendations

Law, regulation and government policy

• Government Digital Service: ATRS guidance for public sector bodies
• Data (Use and Access) Act 2025, section 80 — replacement UK GDPR Articles 22A–22D
• ICO: Data (Use and Access) Act reforms and in-force status
• Equality Act 2010, section 149
• Freedom of Information Act 2000
• ICO: Subject access request response requirements
• Civil Procedure Rules, Part 54 (judicial review)
• Children Act 1989, section 1
• Family Procedure Rules Practice Direction 12J
• Family Procedure Rules Practice Direction 12G
• Judiciary: Position Statement on the Judicial Use of AI, 9 October 2026

Related JSH Law coverage

• The Next Frontier in Family Justice Is Decision Integrity
• A Human in the Loop Is Not Enough: Humanwashing and Judicial AI
• How to Obtain Cafcass Records and Case Notes
• School Safeguarding Records Are Wrong: How to Challenge Inaccurate Information