Ethical AI Procurement: Supplier and Vendor Due Diligence Australia
Many Australian organisations approach AI procurement like software procurement: request for proposal, price comparison, contract signature. This misses the unique risks AI introduces. Traditional software either works or it doesn’t; AI systems can work perfectly technically while being unfair, biased, or non-compliant. When you procure an AI tool that discriminates against certain applicants or processes personal information insecurely, your organisation is liable—not the vendor who cut corners.
Ethical AI procurement means assessing vendors not just on functionality and price, but on their data practices, bias testing, transparency commitments, security standards, and accountability mechanisms. The Australian Information Commissioner’s Office (OAIC) expects this. In a 2024 audit of 40 organisations using third-party AI systems, the OAIC found 67% had failed to audit their vendors’ data handling practices. That gap is now a compliance liability.
Why Procurement Is an Ethical Responsibility
Procurement decisions are often treated as purely commercial: buy the cheapest tool that meets functional specs. But procurement of AI systems is an ethical decision. Here’s why. You’re choosing to delegate consequential decisions—hiring, credit assessment, welfare eligibility—to a system built by someone else, trained on data you didn’t collect, using logic you may not understand. That choice exposes your organisation to risks the vendor accepted and you inherited.
Think of it this way: if you buy a hiring tool from a vendor and that tool systematically rejects women, you—not the vendor—face Fair Work Commission claims, reputational damage, and regulatory enforcement. The OAIC doesn’t care that you bought the system from someone else; you’re the one using it to make decisions about people. You’re responsible for auditing it, understanding its limitations, and ensuring its decisions are fair. The vendor is often just responsible for fixing bugs on request.
This asymmetry—you inherit the risk, the vendor keeps the profits—is why ethical procurement matters. It’s your chance to shift risk back to the vendor before you sign and deploy.
The OAIC’s Expectation: Vendor Audits
In 2024, the OAIC made clear in its AI and Privacy guidance that organisations must audit their AI vendors. Specifically, they should ask: How did the vendor source training data? Did they obtain consent or have a lawful basis? Are they processing personal information securely? Do they use data for any purpose beyond what you’ve contracted for? Can they explain the model’s decisions? Have they tested for bias?
The OAIC found that most organisations had asked none of these questions. Many assumed “the vendor is responsible for their data handling” and didn’t investigate further. That assumption doesn’t protect you. You’re accountable under the Privacy Act for vendors’ practices even if you’ve outsourced the work.
Start vendor evaluation before procurement, not after. Request a vendor’s data security audit report, bias testing results, and documentation of their data sourcing. If they won’t provide these, treat that as a red flag. A reputable AI vendor has this documentation because they know customers are asking for it.
The Ethical AI Procurement Checklist: 15 Critical Questions
Use this checklist to assess vendors systematically. Score each category 1-5 (1 = unacceptable, 5 = excellent) and set a threshold—for example, no vendor below 3 on data ethics or security, no vendor below 2 on bias testing.
Data Ethics (5 questions).
1. How did you source training data? Ask the vendor to document: where the data came from, whether individuals consented, what legal basis they relied on (contract, consent, legal obligation, etc.). Red flag: “We scraped it from the internet” without clear consent or lawful basis.
2. Do you retain training data after model deployment? If yes, for how long and for what purpose? A vendor that permanently deletes training data post-deployment is managing privacy better than one that keeps it indefinitely “for retraining.”
3. Will you use my data or my customers’ data to train future versions? This must be explicitly denied in writing. Many vendors use customer data to improve their models. Your customers didn’t consent to this.
4. Can you certify that training data excludes illegally scraped content or data from breached sources? Ask for a written certification. If they can’t provide it, the data chain is opaque.
5. Do you have a Data Protection Impact Assessment or equivalent for your model? This shows they’ve systematically considered privacy risks.
Bias Testing (3 questions).
6. Have you tested the model for performance disparities across demographic groups (gender, age, ethnicity, disability status)? Request the test methodology and results. Red flag: “We tested internally but can’t share results for confidentiality reasons.” That’s not transparency; that’s hiding.
7. Can you provide disaggregated accuracy metrics by demographic group? A model might be 90% accurate overall but 75% accurate for Indigenous Australians and 95% for others. That disparity is critical to know.
8. If bias was found, what steps did you take to mitigate it? Rebalancing training data, retraining, or adjusting the model threshold are legitimate mitigations. “We noted the bias but left the model unchanged” is not.
Transparency (2 questions).
9. Can you explain how the model makes its decisions in terms a non-technical stakeholder can understand? If the vendor can’t explain it, you can’t oversee it. Feature importance, top factors driving predictions, confidence scores, and edge case handling should all be documentable.
10. Will you provide decision explanations to individuals affected by the system? When an applicant is rejected by the hiring AI, can you tell them why? If the vendor won’t support explanation generation, this is a compliance problem for you under Privacy Act automated decisions rules.
Security (2 questions).
11. What security standards do you meet? Certifications like ISO 27001, SOC 2, or equivalent show systematic security practices. Insecurity isn’t just a privacy risk; it’s an integrity risk—an attacker could poison the model.
12. Can you provide a Data Processing Agreement (DPA) specifying security measures, audit rights, incident notification, and data deletion on contract termination? A vendor unwilling to commit to a DPA is a red flag.
Accountability (2 questions).
13. What happens if the model causes harm—e.g., makes a discriminatory decision? Will you cover liability, reimburse my legal costs, or commit to remediating affected individuals? Most vendor contracts disclaim liability. Push back. A vendor confident in their system should stand behind it.
14. Do you have a mechanism for individuals to contest decisions made by your model? If someone claims they were wrongly assessed, can they appeal? And will you investigate?
Environmental Impact (1 question).
15. How much energy does the model consume, and what’s your carbon footprint? Large language models and deep learning systems have significant environmental impacts. If environmental responsibility matters to your organisation, ask.
Red Flags That Should Disqualify a Vendor
Some vendor behaviours should end the conversation immediately. If a vendor says:
“Our model is proprietary; we can’t explain how it works.” Translation: “You have to trust it, even if it discriminates.” This is unacceptable for high-stakes decisions.
“We don’t disclose bias testing results.” Red flag. If they’ve tested and found bias, hiding it is worse than not testing. If they haven’t tested, they’re selling a model without basic due diligence.
“Your customers’ data helps us improve the model; that’s how we keep costs low.” This means customers are funding the vendor’s product development without knowledge or benefit. Get explicit opt-in consent from your customers before engaging this vendor.
“We don’t provide Data Processing Agreements; that’s our standard practice.” Standard practice doesn’t excuse poor practice. Walk away.
“We can’t guarantee the model won’t discriminate.” At least they’re honest, but this vendor isn’t suitable for hiring, credit, or other high-stakes decisions.
Essential Contract Clauses
Once you’ve passed vendor shortlisting, negotiate contract terms that protect your organisation. Non-negotiables:
Data use clause: The vendor may use your data only to deliver the contracted service, not for training future models, product improvement, or research without your explicit written consent per use case.
Audit rights: You have the right to audit the vendor’s security measures, data handling, and model performance at least annually, or more frequently if you’re processing sensitive categories of personal information.
Bias testing and reporting: The vendor must test the model for performance disparities quarterly (or as agreed), report results to you, and document any mitigations undertaken.
Incident notification: If the vendor discovers a security breach, bias, or other failure, they must notify you within 48 hours so you can respond and notify regulators if required.
Explanation and contestation: The vendor must provide decision explanations in a form you can share with affected individuals, and must support a contestation process for individuals who believe a decision is wrong.
Indemnification: The vendor indemnifies you against third-party claims that the model infringes intellectual property rights or violates discrimination law (at least for bias they knew about).
Termination and data deletion: On contract end, the vendor deletes your data and customers’ data within 30 days (or a defined period), with written certification of deletion.
FAQ
Is it unreasonable to demand audit rights and bias testing documentation? No. Leading AI vendors (OpenAI, Anthropic, Hugging Face) expect these questions. If a vendor resists basic audit rights or refuses to document bias testing, that resistance itself is a compliance risk signal. You’re not asking for unreasonable favours; you’re asking for normal diligence.
If a vendor says they’ll “do bias testing” after we contract, is that acceptable? Not for high-risk decisions. Bias testing should be complete before deployment. If the vendor hasn’t tested yet, they’re selling you an untested product. Insist on pre-purchase testing. At minimum, require that testing happens within 30 days of deployment and that you can pause the system if bias is found.
What if the best tool for my needs comes from a vendor with poor transparency practices? Consider the cost of the risk you’re inheriting. Will you be comfortable explaining to the Fair Work Commission why you deployed an AI system you didn’t audit? Can you afford the reputational damage if the system discriminates? Often, a slightly less powerful tool from a responsible vendor is cheaper in total cost of ownership than a sophisticated tool with hidden risks.
Conclusion
Ethical AI procurement is procurement that accounts for the full cost of the system—not just the license fee, but the risks of bias, privacy breaches, and unfair decisions. Australian organisations that build vendor due diligence into procurement will reduce compliance liability, improve system fairness, and build confidence with customers and regulators. Procurement isn’t just commercial; it’s where ethical AI systems are actually built.