Imagine receiving a rejection letter from an employer.
You were qualified.
You had relevant experience.
Nobody interviewed you.
Nobody telephoned.
Nobody appears even to have read your application.
A machine assessed your résumé, employment history, perhaps your location, education, job changes and other data and concluded that you should not proceed.
Now imagine the same process determining whether you receive:
- a mortgage;
- an apartment;
- insurance;
- medical treatment;
- university admission;
- a public benefit;
- or access to another essential service.
The computer does not insult you.
It does not announce a discriminatory motive.
It may not even contain a rule saying:
“Reject people belonging to group X.”
It merely generates a score.
Or ranking.
Or recommendation.
And the answer is:
No.
This is where the next generation of discrimination law becomes difficult.
Human discrimination historically gave law something familiar to investigate.
What did the decision-maker believe?
What did they say?
How were comparable people treated?
Was a protected characteristic used?
Was the stated explanation pretextual?
Algorithmic systems disrupt that evidential structure.
The person making the discriminatory decision may no longer be sitting behind a desk.
The relevant “decision-maker” may instead be distributed across:
the company which developed the model;
the organisation which purchased it;
the data used to train or configure it;
the employee who set the parameters;
the software producing a score;
and the human who accepted the software’s recommendation.
Colorado has spent the past two years trying to determine how law should respond.
Its answer changed dramatically in 2026.
In May, Colorado enacted Senate Bill 26-189, the Automated Decision-Making Technology legislation, repealing and reenacting the state’s earlier 2024 AI provisions. Instead of centring the statutory framework principally upon a category called the “high-risk artificial intelligence system”, the new law focuses on automated decision-making technology used to materially influence consequential decisions concerning individuals. It was signed by the Governor on 14 May 2026 and its substantive requirements take effect on 1 January 2027.
That change of vocabulary may sound technical.
It is not.
It reflects a much more profound regulatory question:
Should law regulate artificial intelligence because it is artificial intelligence—or because of what somebody allows it to decide?
Colorado’s revised answer is increasingly the latter.
That makes Senate Bill 26-189 one of the most interesting American regulatory experiments of 2026.
Colorado’s first attempt was much more ambitious
To understand why the new law matters, one has to understand what came before it.
Colorado enacted Senate Bill 24-205 in May 2024.
The earlier law required developers and deployers of high-risk AI systems to use reasonable care to protect consumers against known or reasonably foreseeable risks of algorithmic discrimination.
Developers were expected to provide substantial documentation, disclose risk information and support impact assessments.
Deployers faced obligations including AI risk-management programmes, impact assessments, annual reviews, consumer notices, rights to correct inaccurate personal information, mechanisms for appealing adverse consequential decisions and public disclosures concerning high-risk systems.
It was an ambitious attempt to turn the fashionable expression “responsible AI” into actual legal governance.
And then lawmakers encountered the familiar problem of ambitious technology legislation.
The closer implementation approached, the more complicated compliance appeared.
Colorado’s Attorney General later publicly described concerns that the original framework imposed unnecessarily burdensome and, in places, unworkable requirements. The law’s operative date was consequently postponed during a 2025 special legislative session to 1 July 2026, expressly giving legislators further time to revise the framework.
Colorado therefore did something unusually revealing.
It regulated.
Paused.
Reconsidered.
And rewrote.
That is not necessarily legislative failure.
It may be legislative learning.
Senate Bill 26-189 changes the object of regulation
The 2026 Act no longer begins from the same conceptual question:
Is this a high-risk artificial intelligence system?
Instead, it defines automated decision-making technology, or ADMT, broadly as technology which processes personal data and uses computation to produce outputs such as predictions, recommendations, classifications, rankings, scores or other information used to make, guide or assist a judgment or determination concerning an individual.
That definition is important because it avoids an increasingly artificial distinction between:
“AI”;
machine learning;
statistical scoring;
algorithmic ranking;
and other computational decision systems.
A person’s life can be adversely affected by an old algorithm just as surely as by a fashionable generative-AI model.
The legal question should therefore sometimes be:
What function did the technology perform?
rather than:
Was the technology sophisticated enough for somebody to market it as AI?
That is a much more durable approach to legislation.
Technology labels age quickly
The term “artificial intelligence” is notoriously unstable.
A technology regarded as AI today may simply be regarded as ordinary software ten years from now.
Spam filters were once sophisticated.
Recommendation engines became commonplace.
Credit scoring predates contemporary machine learning.
Automated résumé screening may combine rules, statistical inference and modern AI techniques.
A statute dependent upon technical taxonomy can therefore become obsolete remarkably quickly.
Colorado’s ADMT formulation is more functional.
Does computation process personal data?
Does it generate a ranking, prediction, recommendation, classification or score?
Is that output being used to guide a decision concerning an individual?
If so, the legal enquiry can continue.
That is a far more interesting question than debating whether the underlying software is “really AI”.
But Colorado does not regulate every automated decision
The new law focuses upon a subset of systems:
covered ADMT.
Broadly, that is automated decision-making technology used to materially influence a consequential decision.
And Colorado defines consequential decisions by reference to areas where algorithmic decisions can materially alter a person’s life.
The statutory categories include decisions concerning access, eligibility or compensation related to:
education;
employment;
housing;
financial or lending services;
insurance;
healthcare services;
and essential government services or public benefits.
This is the regulatory insight at the centre of the legislation.
Not every algorithm deserves the same law.
An algorithm deciding which song Spotify should play next is not legally equivalent to an algorithm deciding whether someone should receive chemotherapy.
A recommendation engine suggesting shoes is not the same as a model deciding whether a family receives a mortgage.
A system ranking holiday photographs is not equivalent to one ranking job applicants.
The technology may be similar.
The consequences are not.
The law is therefore regulating consequences rather than computational sophistication
That distinction deserves wider adoption.
Technology regulation frequently becomes mesmerised by technical complexity.
Large language models receive attention because they are impressive.
But a relatively simple decision tree can cause enormous harm if government uses it to terminate someone’s welfare benefit.
Conversely, an extraordinarily sophisticated AI image generator may have little connection with any consequential decision about an individual.
Risk does not necessarily correlate with technical novelty.
Sometimes a simple algorithm sitting in the wrong place is more dangerous than the most powerful AI model in the world.
Colorado’s framework recognises that.
Employment makes the problem particularly vivid
Consider recruitment.
An employer receives 50,000 applications.
No human organisation realistically intends to read each one carefully.
Software therefore filters applicants.
Perhaps it ranks education.
Employment gaps.
Geographical location.
Vocabulary.
Previous titles.
Years of experience.
Psychometric answers.
Recorded video responses.
Performance on online assessments.
The efficiency case is powerful.
Automation may even reduce certain forms of human prejudice.
A machine does not become irritated because a candidate is pregnant.
It does not personally dislike an accent.
It does not recognise the applicant’s uncle from a political dispute.
But algorithms can reproduce discrimination through less obvious channels.
A model trained on historical hiring decisions may learn historical preferences.
A variable may act as a proxy for another characteristic.
Employment gaps may disproportionately affect caregivers.
Postcode can correlate with race or socioeconomic background.
Educational institutions can correlate with class.
Language patterns may correlate with national origin.
A seemingly neutral scoring system may therefore reproduce unequal outcomes without containing a line of code openly instructing:
“discriminate.”
This is the difficulty of algorithmic discrimination.
Machines can discriminate without possessing prejudice
This sounds paradoxical only if discrimination is understood exclusively as hatred.
Law has long recognised that unequal treatment can arise without personal animus.
Algorithms make the point obvious.
Software has no racial hatred.
No religious prejudice.
No misogyny.
No class resentment.
It has variables.
Weights.
Training data.
Objectives.
Thresholds.
Correlations.
And outputs.
Yet the output can still systematically disadvantage identifiable groups.
The absence of malice therefore does not mean the absence of discrimination.
Technology may remove discriminatory intention while leaving discriminatory effect.
The machine may even make discrimination harder to see
A prejudiced employer can sometimes be exposed.
Emails.
Witnesses.
Statements.
Patterns.
A computational system may produce a superficially neutral explanation:
Applicant score: 62. Threshold: 70.
But why 62?
Which variables mattered?
Which data was inaccurate?
Was the model validated?
Was the system used for the purpose the developer intended?
Did a human understand its limitations?
Were particular proxies inadvertently discriminatory?
The number can create an illusion of objectivity.
That is dangerous.
Precision is not the same thing as fairness.
“Computer says no” is not an explanation
One of the strongest aspects of Senate Bill 26-189 is therefore procedural.
Colorado does not attempt to guarantee that every automated decision will be correct.
No law can do that.
Instead it gives affected people rights surrounding the process.
A deployer must provide clear and conspicuous notice at the point where a person interacts with covered automated decision-making technology.
If the technology materially contributes to a consequential decision producing an adverse outcome, the deployer must subsequently provide a plain-language description of the system’s role within the statutory period, which the Act sets at 30 days after the relevant decision.
That changes the relationship between individual and machine.
The person is entitled to know that automation mattered.
This is more important than merely saying “AI was used”
Generic disclosure can become meaningless.
A company might say:
“We use artificial intelligence to improve our services.”
That tells the individual almost nothing.
Was AI used to spell-check the letter?
Or reject the mortgage?
Was it peripheral?
Or determinative?
Colorado’s concept of materially influencing a consequential decision directs attention towards decision relevance.
That is where transparency becomes useful.
The meaningful question is not:
“Did AI exist somewhere inside the company?”
It is:
“Did automated technology materially influence what happened to me?”
The applicant also gains a right to inspect relevant personal data
The new law permits consumers to request personal data used by the covered automated decision-making technology and to seek correction where that personal data is factually inaccurate.
That may sound like an ordinary privacy right.
In algorithmic decision-making it becomes especially important.
Suppose the system thinks:
your income is $40,000 rather than $140,000;
you were dismissed from a previous job when you actually resigned;
you have a criminal conviction belonging to somebody with a similar name;
your address is wrong;
you missed loan payments which belonged to another borrower;
your professional qualification has expired when it remains valid.
If the data is wrong, a mathematically perfect algorithm can generate a perfectly wrong answer.
Garbage in.
Garbage out.
Automation does not cure factual error.
It can industrialise it.
A mistake in a human file harms one person; a mistake in an automated pipeline can become systemic
This is one reason algorithmic decision systems deserve special attention.
A human employee misunderstands one document.
One case may be affected.
A faulty data field inside a large automated decision pipeline can affect thousands.
A defective model assumption may affect hundreds of thousands.
Scale transforms error into governance risk.
That is why correcting inaccurate personal data is not merely a privacy convenience.
It can be a form of substantive protection.
Then comes the most important right: meaningful human review
Where a covered ADMT contributes to an adverse consequential decision, Colorado gives the individual the right to request meaningful human review and reconsideration.
Three words matter enormously:
meaningful human review.
Not:
human confirmation.
Not:
human rubber stamping.
Not:
a customer-service employee reading the computer result aloud.
Review must have substance.
Otherwise the right is fiction.
What should “meaningful” mean?
The statute leaves implementation detail to a regulatory framework now being developed by the Colorado Attorney General.
The Attorney General filed proposed rules concerning automated decision-making technology on 11 August 2026, and formal rulemaking is presently underway ahead of the Act’s 1 January 2027 commencement.
But the basic jurisprudential idea can already be identified.
For human review to be meaningful, the reviewer should presumably be capable of doing something the machine cannot simply dictate.
The reviewer should understand:
what decision was made;
what material information was used;
what the automated technology contributed;
what limitations may exist;
what evidence the individual disputes;
and whether the outcome should be reconsidered.
Most importantly, the human should possess authority to change the answer.
A review process in which the human cannot depart from the algorithm is not genuinely human review.
It is theatre.
This problem is larger than Colorado
Organisations around the world increasingly tell regulators that there is a “human in the loop”.
The phrase has become a compliance incantation.
But one should always ask:
Where exactly?
Doing what?
At what stage?
With what information?
With what authority?
Does the human meaningfully examine the evidence?
Or does the employee see:
Algorithm recommendation: reject.
and click:
Approve.
If ninety-nine employees out of one hundred mechanically accept the system’s recommendation because challenging it damages their performance statistics, the human may exist physically while being absent functionally.
Law will increasingly have to distinguish human presence from human judgment.
The human reviewer must not become a liability sponge
There is another danger.
Companies might use the existence of a nominal human reviewer to transfer responsibility.
The developer says:
“We only supplied a recommendation.”
The employer says:
“Our human employee made the final decision.”
The employee says:
“I relied upon the validated system.”
Everyone participated.
Nobody accepts responsibility.
Colorado’s new architecture is interesting precisely because it allocates duties between different actors in the chain.
Developer and deployer are different legal characters
A developer creates or supplies the technology.
A deployer uses it.
This distinction is crucial.
A software company may know how the model was trained and tested but know little about the individual applicant.
The employer may know the applicant and how the tool is being used but know little about the underlying model architecture.
Neither possesses complete information.
Regulation therefore needs both.
Developers must tell deployers what they are buying
Beginning 1 January 2027, a developer providing covered ADMT must supply technical documentation addressing matters including:
the technology’s intended uses;
categories of training data;
known limitations;
and instructions concerning appropriate use and human review.
Developers must also notify deployers about material updates or modifications to the technology.
This is commercially significant.
AI vendors can no longer sensibly sell a consequential-decision tool as though it were an ordinary software licence.
The customer needs governance information.
“Black box—proprietary” is becoming a weaker commercial answer
Technology vendors naturally protect intellectual property.
They should.
A customer is not automatically entitled to source code or trade secrets.
But a company buying software to decide who receives loans or jobs needs enough information to use the system lawfully.
What was it built to do?
What should it not be used for?
What categories of data informed development?
Where are known weaknesses?
What human oversight is recommended?
Has the system materially changed since procurement?
These are not demands for every technical secret.
They are minimum information necessary for responsible deployment.
A vendor who says:
“Trust us, the model is proprietary”
may increasingly find that sophisticated buyers reply:
“Then we cannot responsibly use it for consequential decisions.”
Documentation may become as valuable as model performance
Technology procurement traditionally focused upon:
accuracy;
speed;
price;
integration;
and uptime.
Regulated AI procurement increasingly adds:
explainability;
auditability;
training-data governance;
bias testing;
documentation;
incident reporting;
change control;
and human-review design.
A slightly less powerful model with excellent governance may become legally more valuable than a spectacular model nobody can explain.
That is an important commercial change.
Compliance features themselves become product features.
Updates create another serious issue
Software changes constantly.
A model purchased in January may not be identical in June.
Training data changes.
Weights change.
Features change.
Thresholds change.
Integrations change.
A vendor may update functionality remotely.
Colorado’s requirement that developers notify deployers of material modifications recognises that compliance cannot occur only on the date of procurement.
Model governance requires lifecycle governance.
A compliance assessment conducted against version 1 may tell you nothing about version 7
This is obvious technically and frequently ignored legally.
Contracts should therefore define:
what constitutes a material change;
when notice is required;
whether revalidation is necessary;
whether the customer can reject an update;
whether an impact assessment must be repeated;
what documentation accompanies changes;
and who bears the cost of regulatory re-evaluation.
AI software cannot be treated as though it were a static machine delivered once and left untouched for ten years.
Records must survive
Both developers and deployers must retain records necessary to demonstrate compliance for at least three years.
That is an apparently modest requirement with significant litigation implications.
When a dispute occurs, the decisive question may concern a decision made years earlier.
Which model version was used?
What data?
Which notice was given?
What did the developer disclose?
Was a known limitation documented?
Did the consumer request correction?
Who conducted reconsideration?
What outcome followed?
Without preserved records, everyone reconstructs history imperfectly.
Good governance requires contemporaneous evidence.
Algorithmic litigation will be document-heavy
One can already imagine future disclosure disputes.
The claimant says:
“The algorithm discriminated.”
The deployer asks the developer for model records.
The developer invokes confidentiality.
The claimant seeks data.
The company argues trade secrets.
Experts dispute causation.
Different model versions existed.
Training data evolved.
Human review occurred after the fact.
The output was one of several factors.
The litigation will not be simple.
That is another reason legislation encouraging documentation before disputes arise is valuable.
Colorado does not create a new private cause of action
This is one of the 2026 Act’s most important limitations.
Senate Bill 26-189 does not create a new standalone private right allowing every affected consumer simply to sue for violation of the ADMT statute.
The Colorado Attorney General enforces the statutory requirements through the Colorado Consumer Protection Act, and a violation is treated as a deceptive trade practice.
That allocation reflects a compromise.
The State regulates compliance.
Existing discrimination law continues providing whatever private causes of action it otherwise supplies.
Yet the Act still matters in private discrimination litigation
Colorado did something clever here.
Although the legislation creates no new general private action, it establishes mechanisms concerning allocation of fault between developers and deployers in civil actions alleging unlawful discrimination under existing law.
That means the statute can influence litigation without itself becoming the cause of action.
The underlying discrimination claim comes from elsewhere.
The ADMT framework helps answer:
who in the technology chain bears responsibility?
That is exactly the question algorithmic cases need.
The developer cannot always blame the employer, and the employer cannot always blame the developer
Suppose a recruitment system systematically rejects qualified women.
Why?
Possibility one:
the model itself contains a defective design.
Developer problem.
Possibility two:
the employer configured the system improperly.
Deployer problem.
Possibility three:
the software was designed for graduate recruitment but was used for senior executive hiring contrary to instructions.
Deployer problem.
Possibility four:
the vendor knew of a material limitation and did not disclose it.
Developer problem.
Possibility five:
both contributed.
Shared problem.
Traditional software contracts frequently attempt to allocate these risks categorically.
Reality is rarely categorical.
Colorado’s legal architecture recognises that causation can be distributed.
The law also gives businesses a transitional enforcement period
Before initiating an enforcement action prior to 1 January 2030, the Colorado Attorney General must provide a developer or deployer with 60 days’ notice and an opportunity to cure an alleged violation where cure is considered possible.
This is significant.
The law does not begin with maximum punishment.
It creates an implementation runway.
That may encourage businesses to cooperate, repair compliance failures and learn the regulatory system while rules remain comparatively new.
There is virtue in that.
Technology regulation should not always operate through regulatory ambush.
But a cure period should not become a licence to ignore the law
There is an obvious danger.
Some companies may interpret:
“You get sixty days to cure”
as:
“Compliance is optional until the regulator writes.”
That would defeat the purpose.
The cure mechanism protects good-faith implementation.
It should not become a business model.
A mature organisation should be ready on 1 January 2027, not on the sixtieth day after the Attorney General discovers non-compliance.
The Attorney General is writing the rules now
This makes the subject particularly timely.
The Colorado Attorney General’s Office began formal rulemaking after the legislature rewrote the statute. On 11 August 2026, it filed proposed Automated Decision-Making Technology and Conversational Artificial Intelligence Service rules, with formal public comments presently being received and the rulemaking process continuing through October.
The final operational details therefore remain in development as this article is written.
That is important for businesses.
The Act gives us the statutory architecture.
The rules will determine how parts of that architecture operate in practice.
Lawyers advising affected companies should distinguish enacted duties from proposed regulatory interpretation.
Colorado illustrates why compliance cannot begin after the final rulebook arrives
Some businesses wait for complete certainty.
Technology law rarely provides it.
The model therefore has to be:
understand the statute;
map systems;
identify high-impact uses;
collect technical documentation;
design notices;
build human review;
prepare recordkeeping;
then refine implementation as final rules emerge.
Waiting until every regulatory sentence is final may leave no time to build the necessary operational systems.
Governance takes longer than legal drafting.
The statute is also less burdensome than its 2024 predecessor in important respects
The original 2024 law contemplated a considerably more elaborate compliance structure around high-risk AI, including formal risk-management programmes, impact assessments, annual reviews, broad public statements and risk disclosures.
The 2026 replacement retains meaningful protections but reorganises the framework around the concrete use of covered automated decision-making technology and procedural rights connected with consequential decisions.
That is not simply “weakening” the law.
It represents a different regulatory philosophy.
The legislature has moved from trying to govern a category of risky technology comprehensively towards placing more emphasis upon the decision transaction itself.
Who built the tool?
Who used it?
Was the individual told?
What data was relied upon?
Can inaccurate data be corrected?
Can a human reconsider the adverse result?
Those are administrable questions.
This may be a better model for small and medium-sized businesses
Complex AI impact assessments sound admirable.
They also require lawyers, engineers, compliance officers, data scientists and money.
A large bank can build that infrastructure.
A small employer may not.
If regulation becomes so complex that only multinational companies can comply, it can inadvertently create barriers to entry.
Colorado’s 2026 recalibration appears sensitive to this problem.
The challenge is preserving meaningful protection without creating compliance architecture disproportionate to risk.
That is the essence of good regulation.
The law should care most where the person has something important to lose
This is why the concept of consequential decisions is so powerful.
A machine decides which advertisement you see.
Low stakes.
A machine decides whether your kidney treatment is authorised.
High stakes.
A machine selects music.
Low stakes.
A machine decides whether you are offered housing.
High stakes.
A machine sorts photographs.
Low stakes.
A machine decides whether government suspects you of benefits fraud.
High stakes.
Regulation becomes more proportionate when it concentrates institutional energy where individual consequences are greatest.
Housing illustrates the danger of invisible exclusion
Imagine a landlord using software to screen tenants.
The system analyses:
credit data;
employment;
rental history;
criminal-history information;
address history;
payment patterns;
and other variables.
The landlord receives a simple score.
Applicant A: 83.
Applicant B: 61.
Threshold: 70.
Applicant B is rejected.
Nobody asks why.
But suppose inaccurate criminal-history data reduced the score.
Or the model disproportionately penalised people from particular neighbourhoods.
Or certain proxy variables created discriminatory effects.
Traditional discrimination may be visible at the front door.
Algorithmic discrimination can occur before anyone receives the address.
That makes transparency and review essential.
Lending creates similar problems
Credit scoring is not new.
Automated lending is.
Modern systems can analyse more variables, make faster decisions and operate at enormous scale.
That can improve financial inclusion.
A lender may consider information beyond conventional credit files.
People previously excluded may gain access.
But more data does not automatically mean less discrimination.
Alternative data may create alternative proxies.
A person’s device.
Browsing.
Location.
Purchasing.
Social connections.
Employment pattern.
All may correlate with characteristics law does not want lenders using unfairly.
The algorithm may discover correlations humans never consciously identified.
That is precisely its attraction.
It is also its danger.
A model can become discriminatory by being too good at prediction
This deserves attention.
Suppose an algorithm discovers that a seemingly innocuous variable predicts loan default extremely accurately.
The lender uses it.
Later, analysis shows that the variable strongly correlates with a protected group.
The lender never instructed the model to discriminate.
The model found an economically useful pattern.
What should law do?
This is where anti-discrimination principles confront optimisation.
The algorithm’s commercial objective is:
predict accurately.
Law’s normative objective includes:
do not distribute opportunities unjustly.
Those objectives may conflict.
Software cannot resolve that conflict alone.
It is a legal and ethical choice.
Accuracy is not fairness
This may be the single most important sentence in algorithmic governance.
An algorithm can be statistically accurate and still unjust.
A model could accurately discover that historically disadvantaged groups possess particular measurable outcomes because history itself was unequal.
Turning that correlation into tomorrow’s decision may perpetuate yesterday’s inequality.
That is why automated decision law cannot simply ask:
“Does the model work?”
It must sometimes ask:
“What does working mean?”
Insurance raises the same tension
Insurance is fundamentally predictive.
Who is likely to claim?
How much?
How often?
Pricing follows prediction.
Machine learning can improve actuarial accuracy dramatically.
But an increasingly granular system can create another problem.
If every individual becomes perfectly priced according to personal risk, the social pooling function of insurance begins to change.
Some variables may also operate as legally or ethically problematic proxies.
The algorithm’s optimal commercial price may not always be the legally permissible price.
Again, mathematical precision does not answer normative questions.
Healthcare may be even more sensitive
A healthcare algorithm could decide:
which patient receives attention first;
whether treatment is authorised;
which diagnosis is most likely;
whether someone qualifies for a programme;
where limited resources should be allocated.
Good algorithms can save lives.
Poor ones can institutionalise error.
The ethical stakes are obvious.
If an algorithm works significantly worse for a demographic population underrepresented in training data, deployment can create unequal healthcare without any doctor intending discrimination.
This is why developers need to disclose training-data categories and known limitations to deployers. Colorado expressly requires this type of documentation for covered ADMT.
The hospital needs to know what the machine does not know.
Known limitations may be more important than advertised capabilities
Technology marketing tells customers:
95 per cent accuracy.
Faster decisions.
Lower cost.
Higher throughput.
Better predictions.
Legal due diligence should ask:
On whom was it tested?
Under what conditions?
Against what baseline?
Where does performance deteriorate?
Which populations were underrepresented?
What constitutes error?
How does it behave outside intended use?
What human review is required?
What happens after model drift?
A vendor’s limitation section may become more important than its sales brochure.
Public benefits create a constitutional dimension
Government use of automated decision-making deserves particular scrutiny.
When a private streaming service recommends the wrong film, little happens.
When government incorrectly denies a benefit, a person’s survival may be affected.
Housing support.
Healthcare.
Education.
Income assistance.
Essential public services.
Government also possesses coercive authority private companies do not.
The procedural right to know and request human review therefore becomes particularly important where public institutions automate decisions.
Efficiency cannot become administrative invisibility.
Automation does not repeal administrative justice
Governments understandably want technology.
Backlogs are enormous.
Budgets are constrained.
Public servants handle millions of applications.
AI promises speed.
But administrative justice has values which are not merely inefficient remnants of a paper age.
Reasons.
Review.
Accuracy.
Consistency.
Opportunity to correct errors.
Human discretion where circumstances require it.
These principles should not disappear because government replaced the clerk with software.
A digital state still owes justice.
This is where Colorado’s experiment becomes relevant to Pakistan
Pakistan approved its National Artificial Intelligence Policy 2025 in July 2025, with the Ministry of Information Technology and Telecommunication describing the policy direction as promoting AI innovation, ethical AI development and economic growth.
Pakistan therefore has an official strategic commitment to artificial intelligence.
The harder question is what happens when adoption moves from policy documents into consequential decisions.
A bank uses AI to decide lending.
A hospital prioritises patients.
A university screens applicants.
A large employer filters job candidates.
An insurer prices risk.
A government department uses automated scoring to investigate benefits.
A housing platform ranks tenants.
At that point, the national conversation can no longer remain:
“How do we encourage AI?”
It must also become:
“What rights does the person have when AI says no?”
Pakistan does not need a giant AI Act before answering that question
This is an important lesson from Colorado.
Countries can spend years debating comprehensive AI legislation.
Meanwhile businesses deploy systems.
Pakistan could adopt targeted, sector-specific safeguards much sooner.
For example, regulators could require organisations using automated decision systems in high-stakes contexts to disclose that fact.
Permit correction of materially inaccurate personal data.
Require documented human reconsideration for adverse decisions.
Require vendors to disclose known limitations.
Preserve decision records.
Create audit rights for regulators.
Establish procurement standards for government AI.
Those principles do not require Pakistan first to solve every philosophical question concerning artificial intelligence.
The Pakistani Constitution already begins from equality
Article 25 of the Constitution of Pakistan provides the foundational principle that citizens are equal before law and entitled to equal protection of law, together with the Constitution’s further prohibition concerning discrimination on the basis of sex. The Constitution therefore supplies a powerful normative starting point for thinking about automated public decision-making.
But constitutional principle does not itself provide a complete operational framework for private automated systems.
It does not tell a recruitment-software vendor what documentation to supply.
It does not tell a bank how to explain an algorithmic denial.
It does not specify how a consumer corrects input data.
It does not design a human-review process.
That is where legislation and sector regulation become necessary.
Pakistan’s most immediate algorithmic risk may arrive through foreign software
This deserves particular attention.
Pakistani institutions need not develop their own AI systems to inherit AI bias.
They can buy it.
An employer purchases foreign recruitment software.
A bank licenses overseas fraud detection.
A hospital obtains a clinical decision product.
An insurer integrates international risk-scoring software.
A government agency procures an automated analytics system.
The model may have been trained primarily on American or European data.
Its assumptions may not translate.
Language patterns differ.
Names differ.
Documentation differs.
Employment histories differ.
Address structures differ.
Gender participation differs.
Socioeconomic structures differ.
Local data quality differs.
A model considered fair in Colorado may behave very differently in Karachi.
Imported AI can import imported assumptions
Technology transfer is never purely technical.
A model embodies choices.
Which data mattered?
Which objective was optimised?
What counted as success?
What threshold was acceptable?
What errors were tolerated?
Whose behaviour constituted the reference population?
These decisions may reflect another jurisdiction’s social and economic environment.
Pakistan should therefore not confuse foreign certification with local validity.
A serious procurement process should ask:
Has this system actually been tested on Pakistani data and Pakistani populations?
That question may become one of the most important parts of responsible AI deployment.
Names alone could create unexpected bias
Consider something as simple as résumé screening.
Names may reveal or correlate with:
gender;
ethnicity;
religion;
region;
language;
or socioeconomic background.
A model trained on historical employment data could learn patterns associated with particular names even where nobody deliberately programmed discriminatory rules.
Removing the explicit “gender” field does not necessarily solve the problem if fifteen other variables reconstruct it indirectly.
This is why modern discrimination law must understand proxies.
Removing protected characteristics can sometimes make fairness auditing harder
There is an uncomfortable paradox.
A business says:
“We do not collect gender, therefore we cannot discriminate by gender.”
Not necessarily.
The model may infer it indirectly.
And if the company refuses to collect any demographic information, it may be unable to test whether outputs disproportionately affect different groups.
Privacy and fairness can therefore pull in different directions.
Data minimisation says:
do not collect unnecessary sensitive information.
Bias auditing sometimes says:
we need enough demographic information to discover unequal outcomes.
There is no universal answer.
Responsible governance requires careful purpose limitation and safeguards.
Pakistani employers should begin asking AI vendors difficult questions
An employer buying recruitment software should ask:
What does the model score?
What data does it use?
Was it trained on historical employee data?
Can protected characteristics be inferred?
Has disparate-impact testing been conducted?
For which populations?
How are disabilities accommodated?
Can a human override the recommendation?
Is the candidate told automation is being used?
Can incorrect input data be corrected?
What records are retained?
Will the vendor cooperate in litigation?
What happens if discrimination is alleged?
Who bears contractual liability?
If the vendor cannot answer basic questions, perhaps the employer should not outsource consequential judgment to it.
Contracting becomes central
Colorado’s developer/deployer distinction is particularly useful for international contract drafting.
AI agreements should increasingly allocate responsibility for:
training data;
system documentation;
known limitations;
bias testing;
configuration;
lawful data collection;
candidate notices;
human review;
regulatory changes;
audit rights;
model updates;
incident notification;
record retention;
and discrimination claims.
The traditional SaaS contract was designed around uptime and cybersecurity.
Consequential AI requires more.
“Customer is solely responsible for all outputs” should attract suspicion
Technology vendors frequently insert broad clauses stating that the customer alone bears responsibility for decisions made using the software.
Sometimes that is fair.
The customer chooses how to deploy the product.
But what if the defect lies inside the model?
What if the vendor failed to disclose a known limitation?
What if the system was marketed as suitable for employment screening?
What if an update creates discriminatory effects?
What if the customer lacks access to information necessary to detect the problem?
Responsibility should follow control and fault.
Colorado’s statutory division between developer and deployer captures that principle more intelligently than generic contractual disclaimers.
Likewise, customers cannot outsource morality to vendors
The employer cannot simply say:
“Our software provider told us the system was compliant.”
The deployer chose to use it.
Selected the context.
Integrated the data.
Set the threshold.
Used the output.
A regulated organisation should understand the consequences of technology deployed in its name.
Procurement does not erase accountability.
Boards need inventories of consequential automation
Many organisations still do not know how many algorithmic systems they use.
HR has one.
Fraud has another.
Marketing another.
Finance another.
Customer service another.
A department purchased a cloud tool on a credit card.
Someone enabled an AI feature during a software update.
This phenomenon is sometimes described as shadow AI.
From a governance perspective, the first task is therefore remarkably mundane:
make a list.
Which technologies materially influence decisions about people?
Until that inventory exists, compliance is guesswork.
Do not inventory “AI”; inventory decisions
Colorado’s 2026 approach suggests a smarter methodology.
Instead of asking every department:
Do you use AI?
ask:
Do you use software to score, rank, recommend, classify or predict something about individuals in a way that affects jobs, housing, credit, insurance, healthcare, education or public benefits?
That question is much harder to evade accidentally.
People frequently do not know whether software qualifies technically as AI.
They do know whether software rejects applicants.
Regulation should follow that reality.
Human review needs organisational independence
Suppose the same employee who configured the automated system hears the appeal.
They may be institutionally invested in defending it.
Suppose the review process merely sends the application through the algorithm again.
Nothing changes.
Meaningful reconsideration may therefore require:
access to original information;
ability to consider additional evidence;
understanding of automated limitations;
authority to depart from the recommendation;
and perhaps escalation to someone not personally responsible for the original configuration.
Good process design matters.
Explanation should be comprehensible to the person affected
Technical transparency is not necessarily useful transparency.
A rejected borrower does not benefit from:
“The gradient boosted ensemble returned a negative classification following vector normalisation.”
True perhaps.
Useless.
The person needs to know enough to understand:
what the technology did;
what information mattered;
whether information was wrong;
and what can now be challenged.
Colorado’s statutory requirement for a plain-language description after an adverse outcome is therefore important.
Transparency should serve the recipient, not merely the compliance department.
There is also a dignity value in being told why
Procedural rights matter even where the outcome ultimately remains unchanged.
Someone rejected for a mortgage may accept the decision if they understand:
their debt-to-income ratio was too high;
the system used accurate information;
a human reviewed the result;
and reconsideration confirmed it.
What feels unjust is:
“Computer says no. Goodbye.”
Law is partly concerned with whether institutions treat people as participants in decisions affecting them rather than objects being processed.
Automation should not erase that dignity.
Algorithms may also reduce discrimination when used properly
A balanced article must acknowledge this.
Human decision-makers are inconsistent.
Tired.
Biased.
Distracted.
Susceptible to favouritism.
Influenced by appearance.
Names.
Accents.
Personal connections.
Automated systems can sometimes impose consistency and identify factors more rationally.
A properly designed system may be fairer than the humans it replaces.
The correct legal objective should therefore not be:
“Keep machines away from consequential decisions.”
It should be:
“Use machines where useful without allowing efficiency to erase accountability.”
That is a more mature position.
Human review is not automatically superior
Another uncomfortable truth.
A biased human reviewing an unbiased algorithm can make the decision worse.
Human intervention is therefore not inherently virtuous.
The legal value of human review lies in flexibility, explanation, contextual judgment and ability to correct exceptional cases.
Not in romanticising human decision-makers.
The ideal system may combine machine consistency with human responsibility.
The real objective is contestability
That is the concept I find most compelling in Colorado’s revised law.
The individual should be capable of contesting an adverse automated outcome.
Not necessarily winning.
Not necessarily forcing the institution to abandon automation.
But contesting.
Which data?
Which role did the system play?
Was information wrong?
Can somebody reconsider?
Contestability prevents automation from becoming absolute authority.
The machine must remain answerable to law
That phrase captures the larger jurisprudential point.
Algorithms will increasingly participate in decisions.
That is inevitable.
But institutional delegation to technology must not become delegation outside law.
A bank cannot avoid lending law by calling its decision a score.
An employer cannot avoid discrimination law by calling its decision a ranking.
Government cannot avoid administrative fairness by calling its decision an automated assessment.
The legal character of the decision survives the technological method.
Algorithmic discrimination may actually expose weaknesses in existing discrimination law
Traditional law often depends upon categories and evidential models developed around human decision-making.
AI raises difficult questions.
What if several protected and unprotected variables interact in ways nobody understands?
What if no individual decision-maker intended discrimination?
What if disparity appears statistically across a large population but no single person can prove why they were rejected?
What if the model changes continuously?
What if developers and deployers sit in different jurisdictions?
What if training data came from multiple sources?
These problems may eventually force anti-discrimination jurisprudence to evolve.
Colorado is beginning with procedure.
That is sensible.
There is another question: should individuals have a right to opt out of automation entirely?
Colorado’s 2026 law does not simply create a universal entitlement to demand that every consequential decision be made manually. Instead, it focuses upon notice, data correction and meaningful human review after relevant adverse outcomes.
That is probably pragmatic.
A bank receiving one million applications may not be capable of providing purely manual underwriting to everyone.
The right worth protecting may therefore be less:
“A human must make every decision.”
and more:
“A human must remain capable of correcting the machine when the machine matters.”
That distinction will become central internationally.
Excessive regulation could protect incumbent companies
There is also a competition concern.
Large technology companies can hire:
AI ethicists;
lawyers;
data scientists;
compliance teams;
model auditors;
and regulatory specialists.
Start-ups cannot.
Every compliance obligation has a fixed cost.
If lawmakers create a hundred-page impact-assessment process for every algorithm, small innovators may simply avoid the market.
Regulatory complexity can therefore become a competitive moat.
Colorado’s decision to rewrite its original law after concerns about burden should be understood partly against this reality.
Good regulation should protect people without accidentally guaranteeing that only giant companies can afford to innovate.
On the other hand, “innovation” cannot become an immunity word
Technology companies frequently respond to regulation by invoking innovation.
Sometimes correctly.
Sometimes strategically.
The fact that a system is innovative does not answer whether it unfairly denies someone housing.
Innovation is valuable.
So is fairness.
Law exists precisely because commercial incentives do not automatically optimise every social value.
A civilisation which can build extraordinary algorithms should also be capable of building procedures ensuring those algorithms do not casually ruin people’s lives.
Pakistan should avoid both extremes
Pakistan should not suffocate an emerging AI sector with regulatory bureaucracy copied wholesale from Europe or the United States.
Nor should it assume that “AI innovation” will regulate itself.
The sensible route is proportionate.
Concentrate first on consequential uses.
Employment.
Credit.
Insurance.
Healthcare.
Education.
Government services.
Public procurement.
That is where individual rights and institutional power intersect most directly.
Public-sector procurement may be Pakistan’s best first intervention
Government is one of the largest purchasers of technology.
Pakistan could therefore introduce responsible-AI obligations through procurement before enacting a comprehensive statute.
Any vendor supplying automated decision technology for consequential government functions could be required to provide:
technical documentation;
data provenance information;
known limitations;
local validation;
audit rights;
human-review mechanisms;
records;
security documentation;
and contractual cooperation in legal challenges.
Government can shape markets through purchasing power.
This is often faster than legislation.
Regulators can act sector by sector
The State Bank can consider automated lending and financial decisions.
SECP can examine insurance and regulated financial-sector uses within its remit.
Healthcare regulators can develop clinical AI standards.
Public-service authorities can address automated government decision-making.
Employment regulation can develop separately.
This may ultimately be more administratively realistic for Pakistan than immediately creating one giant central AI regulator expected to understand every industry.
AI risk is contextual.
Sector regulators already understand the contexts.
Pakistan should also insist upon local validation
This point deserves repetition.
Do not assume software trained in Denver behaves correctly in Dera Ghazi Khan.
Do not assume a model tested on American English understands Pakistani English.
Do not assume a recruitment model trained on Western employment patterns understands career interruptions common locally.
Do not assume facial or voice systems perform equally across populations.
Do not assume foreign credit variables translate.
Local deployment requires local evidence.
Imported technology still requires domestic responsibility.
The National AI Policy should eventually acquire enforceable procedural children
Pakistan’s National AI Policy provides strategic direction and has been formally approved by the Ministry’s policy framework.
Policies state aspirations.
Rights require procedures.
The next stage of Pakistani AI governance should therefore translate broad concepts such as ethical or responsible AI into modest, concrete requirements.
Tell people when consequential automation is used.
Keep accurate records.
Correct wrong data.
Maintain meaningful review.
Require vendors to disclose limitations.
Audit the most sensitive systems.
Those five rules would achieve more practical fairness than fifty pages of ceremonial AI ethics.
Frequently Asked Questions
What did Colorado change in 2026?
Colorado enacted Senate Bill 26-189, repealing and reenacting the state’s earlier 2024 AI framework with new requirements focused upon automated decision-making technology used to materially influence consequential decisions.
When was Senate Bill 26-189 signed?
The Governor signed the legislation on 14 May 2026.
When do the new requirements take effect?
The substantive framework takes effect on 1 January 2027.
What is automated decision-making technology under the Colorado law?
The Act broadly defines ADMT as technology processing personal data and using computation to generate outputs—including predictions, recommendations, classifications, rankings or scores—used to make, guide or assist a decision or determination concerning an individual.
What is a consequential decision?
The statutory category includes decisions relating to access, eligibility or compensation concerning education, employment, housing, financial or lending services, insurance, healthcare services and essential government services or public benefits.
Does the law regulate every piece of AI software?
No. The principal duties concern covered ADMT used to materially influence consequential decisions. The statute therefore focuses upon particular decision contexts rather than every possible use of computation.
What must AI developers provide?
Developers of covered technology must provide deployers with technical documentation including intended uses, categories of training data, known limitations and instructions concerning proper use and human review, and must notify deployers of material updates or modifications.
What rights do people have after an adverse automated decision?
The Act provides rights including access to certain personal data used by the covered technology, correction of factually inaccurate personal data and a request for meaningful human review and reconsideration after a covered ADMT contributes to an adverse consequential decision.
Must people be told that automated technology is being used?
Yes. Deployers have notice obligations, including clear and conspicuous notice at the point of interaction and a plain-language description of the covered technology’s role following a qualifying adverse consequential outcome.
How long must compliance records be kept?
Developers and deployers must retain records necessary to demonstrate compliance for at least three years.
Does the Colorado law create a new private right to sue?
No. The Act does not create a new standalone private right of action. The Attorney General enforces its requirements through the Colorado Consumer Protection Act. The legislation does, however, address allocation of fault between developers and deployers in existing civil discrimination cases.
Is there an opportunity to cure violations?
Until 1 January 2030, the Attorney General must generally provide sixty days’ notice and an opportunity to cure before commencing an enforcement action where a cure is considered possible.
Why did Colorado rewrite its earlier AI law?
The original 2024 framework imposed extensive high-risk AI governance obligations. The Attorney General subsequently raised concerns about burdensome and unworkable requirements; the legislature delayed implementation during a 2025 special session and then replaced the framework in 2026.
Are the implementing rules final?
Not yet as of 19 August 2026. The Colorado Attorney General filed proposed ADMT rules on 11 August 2026 and formal rulemaking is presently underway.
Does Pakistan currently have equivalent algorithmic-decision legislation?
Pakistan has an approved National AI Policy 2025 promoting AI development and ethical use, but it does not presently operate an equivalent Colorado-style statutory framework giving individuals the same specific notice, data-correction and meaningful-human-review rights for consequential private-sector automated decisions.
Could Pakistan adopt similar principles without copying Colorado?
Certainly. Pakistan could develop sector-specific requirements concerning consequential automation, including disclosure, data correction, vendor documentation, local validation and meaningful human reconsideration, while adapting them to Pakistan’s constitutional, institutional and economic circumstances.
The deeper problem: algorithms convert judgment into infrastructure
A human decision feels like a decision.
Someone interviews you.
Considers your application.
Writes a letter.
An algorithmic decision can disappear inside infrastructure.
The score appears.
The workflow proceeds.
The rejection email is generated.
Nobody experiences themselves as having decided anything.
That diffusion of responsibility is one of the great legal dangers of automation.
Everyone can say:
“The system did it.”
But systems do not owe duties.
People and institutions do.
Law must therefore reconnect automated outcomes with human accountability.
The worst automated decisions may be the ones nobody notices
A dramatic AI failure becomes news.
A chatbot says something absurd.
A self-driving vehicle crashes.
A deepfake goes viral.
Algorithmic discrimination can be quieter.
Applicant rejected.
Tenant declined.
Loan refused.
Insurance priced higher.
Benefit denied.
No scandal.
No viral clip.
Just thousands of individual people receiving slightly worse outcomes for reasons they cannot see.
That makes the problem less visible, not less serious.
Statistical injustice can hide inside individually plausible decisions
This is another challenge.
Every individual rejection may appear defensible.
Applicant A lacked experience.
Applicant B had insufficient income.
Applicant C had unstable employment.
Applicant D had a poor score.
But across 100,000 decisions, a pattern emerges.
One group consistently receives worse outcomes.
The system may therefore look rational close up and discriminatory from a distance.
Algorithmic regulation needs both perspectives.
Individual review.
Population auditing.
Colorado’s 2026 law emphasises the former more heavily than its predecessor.
Future regulation may need both.
Humans also hide behind numbers
People trust scores.
Credit score.
Risk score.
Suitability score.
Fraud score.
Performance score.
Once a number appears, decision-makers may feel absolved from judgment.
Numbers look scientific.
That creates automation bias.
A human reviewer may unconsciously assume:
the computer has more information;
the model was validated;
somebody else checked it;
therefore the result is probably right.
Meaningful human review requires cultural as well as procedural change.
The reviewer must be permitted to distrust the machine.
A score should be evidence, not authority
That may be the better organisational principle.
Automated output can inform judgment.
It should not automatically become judgment where the stakes are high.
A model should be capable of being challenged.
A human should be capable of departing from it.
An institution should remain capable of explaining why it followed it.
That is how technology becomes a tool rather than sovereign.
Colorado may have found a useful middle ground
The 2024 law attempted comprehensive governance around high-risk AI systems.
The 2026 law retreats from some of that regulatory machinery while preserving protections at the point where automation touches an individual.
Notice.
Information.
Correction.
Documentation.
Human reconsideration.
Regulatory enforcement.
The result may prove too weak.
Or too burdensome.
We do not yet know.
Implementation only begins in January 2027.
But the conceptual shift is worth studying.
Regulate the decision, not the marketing label
That may be Colorado’s most important contribution.
A company should not escape regulation because its lawyers argue that the scoring engine technically is not AI.
Nor should a harmless application become heavily regulated merely because its marketing department enthusiastically calls every feature AI.
Ask instead:
What does the software do?
What information does it process?
What decision does it influence?
How important is that decision?
What happens if it is wrong?
Can the individual challenge it?
That framework may survive technological change much better than another statutory definition of artificial intelligence.
Pakistan has an opportunity to arrive at the same insight earlier
Pakistan does not need to spend two years debating whether every machine-learning system qualifies as “high-risk AI”.
It can begin with consequential decisions.
If an algorithm materially affects:
employment;
credit;
housing;
insurance;
health;
education;
or government benefits,
then certain basic procedural protections should follow.
That is intuitively understandable.
And administratively manageable.
The principle is older than AI
This is worth remembering.
The newest technology is exposing one of law’s oldest concerns.
A person affected by power should ordinarily know that power has been exercised.
They should have a fair opportunity to correct material error.
They should have some avenue of review.
And those exercising power should remain accountable for how it was used.
None of those principles were invented by machine learning.
AI merely gives us a new reason to remember them.
The computer can assist the decision; it should not erase responsibility for the decision
That is ultimately the central legal proposition.
A business may choose automation.
A hospital may choose automation.
A bank may choose automation.
Government may choose automation.
But each remains responsible for what its systems do within its institutional authority.
Efficiency is not absolution.
Outsourcing is not absolution.
Complexity is not absolution.
And the sentence:
“the algorithm rejected you”
should never become the end of legal analysis.
It should be the beginning.
About the Author
Barrister Aemen Zulfikar Maluka is the founder of Josh and Mak International, an Islamabad-based law practice advising Pakistani, overseas and international clients on cross-border commercial, regulatory, technology and public-law matters.
Her international legal commentary focuses on emerging regulatory developments across the United States, United Kingdom, European Union, Australia, Asia and other jurisdictions, particularly where apparently foreign developments illuminate legal problems that businesses and public institutions connected with Pakistan will shortly confront themselves.
Artificial intelligence makes this comparative perspective especially important. A Pakistani employer may procure American recruitment software; a local bank may licence an overseas scoring model; a technology company in Pakistan may develop decision-support software for a foreign client; and an international company operating in Pakistan may apply global automated-decision systems locally. The law governing those systems increasingly crosses technical, contractual and jurisdictional boundaries.
Barrister Aemen’s advisory approach is therefore practical as well as comparative: identifying who controls the technology, who bears responsibility for decisions, what documentation and contractual protections are required, what rights individuals possess and how international regulatory developments should inform Pakistani legal and commercial strategy.
For further insights, international AI regulatory analysis, cross-border technology contracting, algorithmic governance advice or legal advice concerning the implications of emerging foreign regulation for businesses connected with Pakistan, contact Barrister Aemen at Aemen@joshandmak.com.
Josh and Mak International
www.joshandmakinternational.com
This article is intended as general legal and regulatory commentary and does not constitute Colorado, United States or Pakistani legal advice concerning any particular automated decision system, employment process, lending decision, healthcare system or transaction. The Colorado Attorney General’s implementing rulemaking remains in progress as of 19 August 2026.