AI and Children’s Online Privacy: Special Obligations in Australia

Children leave digital footprints earlier than ever—and AI systems are watching. When a five-year-old engages with an AI-powered educational platform, watches recommendation algorithms personalise video content, or uses a chatbot designed for learning, that child’s behaviour, preferences, language patterns, and even emotional states are being logged, processed, and stored. In Australia, this data collection isn’t unregulated grey space: organisations deploying AI systems that collect, use, or infer information about children face overlapping obligations under Privacy Act 1988 (Cth), the Online Safety Act 2021 (Cth), and the eSafety Commissioner’s regulatory expectations. Yet enforcement remains patchy, and many organisations misunderstand their legal and ethical duties. Understanding the special status of children’s data in AI systems is essential for any Australian firm targeting young users.

Why Children’s Data Is a Special Category in AI

Children’s data demands different handling than adult data for two reasons: developmental vulnerability and temporal duration. Young brains are still forming; they’re susceptible to manipulation, less capable of understanding long-term consequences, and more prone to privacy harms they cannot anticipate. An AI system that infers a child’s learning disabilities, mental health struggles, or family instability from interaction patterns creates a persistent record that shapes future algorithmic decisions—college admissions, employment screening, insurance pricing—long into adulthood.

The temporal problem amplifies this risk. An adult might disclose personal health information to an online forum and later delete the account. A child who does the same in 2026 might find that data encoded in an AI training dataset, resurface in a decade via a data breach, or shape algorithmic predictions about their employability or creditworthiness at age 30. Think of children’s data as a tree ring: every interaction adds a layer of inference that cannot be unwritten. This permanence makes children’s data fundamentally different from adults’.

Developmentally, children cannot meaningfully consent to the algorithmic complexity that AI systems introduce. A six-year-old cannot understand how behavioural data fed into a machine learning model might drive filter bubbles or personalised advertising that shapes their worldview. A thirteen-year-old cannot foresee how interaction patterns collected via a social platform’s AI could be weaponised for social engineering or manipulation. Australian privacy law recognises this asymmetry; organisations deploying AI to children must navigate that reality.

The eSafety Commissioner and AI Platform Accountability

The Australian eSafety Commissioner holds broad powers to regulate how platforms—particularly those using AI systems—handle content and safety. Under the Online Safety Act 2021, the eSafety Commissioner can issue removal notices requiring platforms to delete harmful material within specified timeframes. Platforms that fail to comply face civil penalties of up to AUD 555,000 (as of 2026). Crucially, the Commissioner’s remit now extends to AI-driven recommendations and content filtering systems that disproportionately affect children.

In 2024–2025, the eSafety Commissioner released updated expectations for social media platforms, explicitly addressing algorithmic personalisation, age-gating, and AI moderation. The Commissioner expects platforms to demonstrate that AI systems managing children’s content exposure operate within child-protection parameters. This isn’t optional guidance; platforms failing to meet these expectations face public shaming campaigns, removal notices, and reputational damage that drives regulatory escalation.

For organisations building AI systems used by children—from educational tools to social platforms to parental-control applications—the eSafety Commissioner’s jurisdiction is expanding. If your AI system facilitates access to harmful content, fails to age-restrict appropriately, or uses child data in ways the Commissioner deems inconsistent with child safety, enforcement action is increasingly likely. The Commissioner publishes an annual transparency report and actively uses media attention to drive compliance; non-compliance is a reputational liability.

Privacy Act Obligations for Organisations Handling Children’s Data

The Privacy Act 1988 (Cth) imposes strict requirements on organisations collecting children’s personal information. Organisations must collect personal information only for lawful purposes directly related to their functions, and collection must be fair, transparent, and limited to necessary information. For AI systems, this means you cannot collect children’s interaction logs simply to “improve algorithms”; collection must align with disclosed, legitimate purposes.

The critical obligation: consent. Organisations collecting personal information from children typically require parental consent, not just child consent. The Privacy Commissioner’s guidance clarifies that children under 13 lack legal capacity to give valid consent; parental authority is required. For teenagers aged 13–17, the position is murkier—some child-psychology experts argue that adolescents cannot meaningfully consent to algorithmic complexity—but Australian privacy law currently permits organisations to accept direct consent from teenagers 13 and above for non-sensitive personal information.

Data minimisation is equally important. An AI system trained on thousands of data points about a child’s learning behaviour, location, social interactions, and viewing patterns violates the Privacy Act’s minimisation principle if those data points exceed what’s necessary for the disclosed purpose. If your AI tutor requires access to a child’s family income data, device GPS location, and browsing history to deliver maths tutoring, that’s excessive and unlawful—regardless of your terms of service.

The Privacy Act also grants children (and their parents) data access rights. A parent can request all personal information held about their child and demand it be deleted or corrected. If your AI system cannot explain why a particular data point was collected, you’re breaching the Act’s requirements for data governance. Documentation of collection purposes, storage, inference logic, and retention schedules is mandatory.

Online Safety Act Requirements and Age Restriction

The Online Safety Act imposes platform-level duties that directly affect AI system design. Platforms must take reasonable precautions to prevent children accessing age-inappropriate content, maintain systems for removing harmful material quickly, and demonstrate that algorithmic recommendation systems don’t amplify illegal or harmful content. For AI-driven recommendations, this means implementing guardrails: your system cannot recommend content to a child that would constitute harassment, sexual abuse material, or violence.

Age-appropriate design is now a regulatory expectation, not optional. The eSafety Commissioner’s guidance aligns with UK ICO recommendations and international best practice: AI systems used by children should be designed with age-appropriate features, reduced data collection, disabled personalisation (or heavily restricted personalisation), and parental transparency. Dark patterns—design choices that manipulate children into sharing more data, spending more time, or disclosing more information—are increasingly scrutinised as breaches of both the Online Safety Act and Australian Consumer Law.

Best Practices: Age Verification and Consent Mechanisms

Effective age verification is non-negotiable. Relying on checkboxes (“I am 13 or older”) is inadequate; regulators now expect documentary evidence or third-party verification. Options include identity document upload (driver’s license, passport), third-party age-assurance APIs (which perform cryptographic verification without storing copies of documents), and custodian-gated access (requiring parental email verification). Acceptable options vary by risk profile; educational apps might use basic custodian gates, while social platforms should implement stronger verification.

Consent mechanisms must be genuinely informed. A 200-line privacy policy buried in legal jargon does not constitute meaningful consent. Best practice involves: plain-language explanation of what data you collect and why, age-appropriate explanation of how AI uses that data to personalise experiences, explicit opt-in for data-processing activities beyond the minimum necessary, and easy opt-out mechanisms. For children under 13, parental consent forms should be separate from child sign-up flows and should explain risks, not obscure them.

Data retention and deletion policies matter significantly. Retaining children’s data indefinitely to “improve algorithms” is indefensible under the Privacy Act and eSafety Act expectations. Best practice involves: deletion of raw interaction data after it’s aggregated for model training, removal of linkages between a child’s identity and their training data (de-identification), and scheduled deletion of all children’s data upon account closure. If a parent requests deletion, compliance must be swift—not “we’ll deprioritise it”.

Frequently Asked Questions

Q: Can I collect children’s data without parental consent?
A: Parental consent is generally required for children under 13. For teenagers 13+, direct consent from the teenager may suffice for non-sensitive information under Privacy Act guidelines, but best practice is to obtain both teen and parental consent to reduce legal risk and build trust.

Q: What counts as “harmful content” under the Online Safety Act?
A: The eSafety Commissioner defines harmful content as material that threatens, abuses, depicts violence or sexual abuse, or constitutes cyberbullying. For AI systems, this includes algorithmically amplified harmful content and personalisation that exposes children to such material.

Q: What happens if the eSafety Commissioner investigates my platform?
A: The Commissioner can issue removal notices (requiring material deletion within a set timeframe), negotiate compliance agreements, and refer matters to ACMA (Australian Communications and Media Authority) for civil penalty proceedings. Penalties can reach AUD 555,000+ for systemic non-compliance.

Q: How should I handle parental access requests?
A: Respond to documented parental requests for access to their child’s personal information within 30 days. Provide data in a clear, readable format. If a parent requests deletion, delete raw data and remove linkages to training datasets (though aggregate, de-identified patterns may be retained for model improvement).

Key Takeaway

Children’s data is not a scalable commodity for AI training; it’s a special category requiring heightened protection. Australian organisations deploying AI systems used by children must implement age verification, obtain meaningful consent, minimise data collection, restrict algorithmic personalisation, and maintain deletion protocols aligned with Privacy Act principles. Failure to do so risks eSafety Commissioner enforcement, reputational damage, and regulatory escalation that outweighs the algorithmic gains of unconstrained data collection.

Navigating compliance for child-safe AI systems? Contact Anitech to design age-appropriate AI systems that protect children while meeting Australian regulatory expectations.