Australia’s Digital Transformation Agency Issues New Guidance for Agentic AI

What privacy and AI governance professionals need to know about Australia’s new agentic AI standard.

Why Agentic AI Demands Its Own Guidance

Artificial intelligence has been evolving rapidly, but few developments signal a more fundamental shift than the emergence of agentic AI. Unlike conventional AI systems that respond to a single prompt and return a result, agentic AI can plan, reason across multiple steps, use tools, take actions in the real world, and collaborate with other AI agents. And all of this happens usually with minimal human intervention along the way.

That shift from AI as a passive analytical tool (which a lot of us are using like an advanced Google search) to AI as an active, decision-making participant changes everything, particularly from a governance perspective. It raises new questions for AI governance like: Who is accountable where there is a chain of autonomous agents taking an action that ultimately causes harm? How do you audit a decision that emerged from dozens of intermediate reasoning steps? What happens when one agent passes sensitive data to another without any human reviewing that exchange?

These are no longer hypothetical concerns, as agentic AI moves from research labs into ordinary operational environments, including in government. Recognising the real risks of getting agentic AI governance wrong, Australia’s Digital Transformation Agency (DTA) has published an Agentic AI Addendum to its existing AI Technical Standard, giving dedicated guidance for government agencies deploying or exploring these systems. But useful for all organisations going down the agentic AI path, particularly those operating in regulated environments. 

This blog posts looks at DTA’s new Agentic AI guidance in more detail.

Summary of DTA’s Agentic AI Addendum

For those interested in the highlights, the following are the main points for privacy and governance professionals from the new guidance:

• Human accountability is non-negotiable, with defined responsibilities and a named accountable human. Accountability must be traceable, documented, and auditable.
• Calibrate human oversight to risk (HITL vs HOTL) and document those calibration decisions.
• Memory is a privacy risk that must be actively governed. What agents can store, for how long, under what access controls, and with clear purge and conflict resolution mechanisms must be defined.
• Workflow design must embed guardrails from the start including mapping task sequences, assigning least-privilege access to each agent, defining decision points and conditional branches, and built in kill switches and automatic rollbacks.
• Data governance (including quality and security) must be confirmed before you build. Agentic workflows must not be built on top of ungoverned data, and agent-to-agent data exchange must be protected by authentication, encryption, and sovereignty controls.
• Model and technology selection is a governance decision, not just a technical one. The choice of model affects explainability, auditability, and ultimately compliance.
• Prompts must be treated as governed artefacts. System prompts should be logged, versioned, approved, and capable of rollback.
• Evaluation is ongoing, not a one-off exercise.
• Tool and protocol governance is essential as the ecosystem matures. Tools must be selected based on authorised scope, accessed through approved interfaces with full audit logging, and governed for interoperability.
• Invest in a control tower from day one. A centralised control tower providing real-time dashboards, audit trails, and compliance reporting is strongly recommended, more than a nice-to-have.

The DTA’s Role in AI Governance

The Digital Transformation Agency is the Australian Government’s lead body for digital policy and transformation. Among its responsibilities is the development and maintenance of technical standards that guide how government agencies design, procure, and use technology including AI.

The DTA’s AI Technical Standard provides a lifecycle-based framework covering everything from design and data management through to evaluation, integration, monitoring, and decommissioning. It sits alongside broader obligations including the Australian Government’s Policy for the Responsible Use of AI, the Australian Privacy Principles, the Information Security Manual, and the Protective Security Policy Framework.

The new Agentic AI Addendum extends that existing standard rather than replacing it. All prior requirements continue to apply; the addendum layers on additional criteria specifically tailored to the distinctive risks and characteristics of agentic systems. This approach is sensible: it avoids creating siloed compliance pathways and ensures that agentic AI is assessed through the same foundational lens as other AI, while acknowledging where it genuinely differs.

Understanding Agentic AI: Capabilities, Use Cases, and Risks

What Makes a System ‘Agentic’?

The addendum defines an agentic AI system as one that perceives its environment, maintains an internal state, reasons across multiple steps, and executes a series of actions independently to achieve predefined objectives. Crucially, these systems operate with varying levels of autonomy and can coordinate with other agents, humans, or external services.

The fundamental operating cycle is described as Perceive, Reason, and Act (PRA): the agent takes in inputs, evaluates them against its goals and constraints, and then executes actions, using APIs, databases, code execution, or other tools, before incorporating feedback and repeating the cycle.

How Agentic AI Might Be Used

In government and regulated industries, the potential applications of agentic AI systems are significant. These systems can handle high-volume administrative tasks, process complex multi-document queries, coordinate compliance workflows, triage correspondence, or manage procurement processes, all tasks that currently require significant human time and are prone to inconsistency.

The addendum identifies specialised agent types that might work together within a single system: planner agents that decompose goals into tasks; executor agents that carry out permitted actions; researcher agents that retrieve and synthesise information; supervisor agents that monitor progress and trigger rollbacks; and monitoring agents that track anomalies, costs, and failures in real time.

The Governance Challenges

The very features that make agentic AI powerful also make it risky to deploy without rigorous governance controls. Key concerns include:

• Accountability gaps: When multiple agents act sequentially or in parallel, tracing a harmful outcome back to a specific decision point becomes complex. Human accountability must be explicitly assigned and documented.
• Data leakage and exfiltration: Agents passing information between themselves and to external tools create new pathways for sensitive data to move outside controlled environments, often invisibly.
• Cascading errors: A mistake in one agent’s output can propagate through an entire workflow before any human has the opportunity to identify and correct it.
• Prompt injection and tool misuse: Malicious inputs or poorly validated tool interfaces can manipulate agent behaviour in ways that are difficult to detect.
• Memory and retention risks: Agents that retain information across sessions create ongoing privacy obligations that must be actively managed.
• Autonomy calibration: The appropriate level of human oversight must be calibrated to the risk of each decision: a one-size-fits-all approach will either over-burden humans or expose the organisation to unacceptable risk.

A Detailed Review of the Guidance

The addendum is structured around eight statements, each tied to specific lifecycle phases and supported by mandatory criteria (what agencies ‘must’ do) and recommended criteria (what they ‘should’ do). The following summarises the key requirements.

Governance and Accountability (AGT.1)

The foundational requirement is that every agent must have clearly defined responsibilities and every decision must have a human accountable for it. This is not merely about assigning a responsible officer at the system level, it requires documentation of what each agent is authorised to do, how accountability traces back through agent actions, and how accountability is handled when external systems or third-party agents are involved. Agencies are also required to explain agentic decision-making processes, including capturing reasoning chain outputs where LLMs are used, while ensuring that reasoning logs do not themselves expose personal or sensitive data.

Memory Management (AGT.2)

One of the addendum’s more distinctive elements is its focus on memory governance. Agentic systems may retain information across sessions in short-term buffers, long-term databases, or vector stores. The guidance requires agencies to define what may be stored, for how long, under what sovereignty and access controls, and with what purge mechanisms. Conflict resolution rules must be established for contradictory memory states, and mechanisms must exist to remove outdated information to prevent agents from acting on stale or incorrect data.

Workflow Design and Guardrails (AGT.3)

Before deploying an agentic system, agencies must map the logical sequence of tasks and agents, define each agent’s role and access privileges, identify conditional branches and decision points, and embed self-correction mechanisms including kill switches, automatic rollbacks, and alerting systems. Each agent must be assigned a unique identity with least-privilege access: agents should not be able to self-assign elevated permissions. Fail-safes and feedback loops must be auditable.

Data Flow and Orchestration (AGT.4)

Agent-to-agent data exchange must be governed by robust authentication and encryption protocols, with classification and sovereignty controls applied throughout. The guidance requires agencies to define which components are responsible for routing decisions and to ensure that data flows between agents, tools, APIs, and memory stores remain secure, governed, and traceable.

Model and Technology Selection (AGT.5)

Agencies must evaluate the appropriate agent type and underlying model for each use case, explicitly weighing trade-offs including capability, cost, latency, data residency, FOI exposure, and vendor lock-in risk. Prompt engineering is treated as a governance matter: prompts should be logged, versioned, approved, and capable of rollback. The guidance recommends techniques such as chain-of-thought prompting and ReAct to improve reliability, transparency, and explainability.

Evaluation (AGT.6)

Continuous evaluation is described as essential, not optional. Requirements include testing agents against adversarial scenarios, measuring success rates for task completion, evaluating tool selection accuracy, assessing agent-environment interactions, and reviewing how effectively agents manage and retrieve information from memory. Evaluation must be ongoing, not just at initial deployment.

Integration and Protocols (AGT.7)

Tools used by agents must be selected based on authorised scope, provided with only necessary permissions, and accessed through approved interfaces with full audit logging. The guidance acknowledges the rapidly evolving landscape of inter-agent communication protocols — including MCP, Agent2Agent (A2A), and Agent Communication Protocol (ACP) — and requires agencies to ensure these are governed for alignment and interoperability.

Monitoring and Control Tower (AGT.8)

Continuous monitoring must cover individual agents, inter-agent interactions, tool use, memory integrity, and environmental changes. The guidance recommends establishing a ‘control tower’: a centralised governance layer providing end-to-end observability, real-time dashboards, audit trails, error tracing, and compliance reporting across the entire agentic system.

Steps for Australian Organisations Considering Agentic AI

Whether you are in government or the private sector, the addendum provides a useful blueprint for responsible adoption of agentic AI.

The following steps reflect its key requirements translated into practical action:

• Start with governance design, not technology selection. Before choosing a framework or model, map accountability: who is responsible for each agent’s behaviour and who is accountable for outcomes? Establish documentation and escalation pathways from the outset.
• Confirm data readiness. The addendum is explicit that data quality, governance, and security must be confirmed before progressing beyond early design. Do not build agentic workflows on top of ungoverned data.
• Apply least-privilege principles rigorously. Each agent should have only the access it needs to perform its defined task. Prevent agents from self-assigning elevated permissions or accessing data and tools outside their authorised scope.
• Design human oversight into the workflow, not as an afterthought. Identify which decisions require human-in-the-loop pre-approval, which warrant human-on-the-loop monitoring, and which can be post-hoc audited. Document these calibration decisions and review them regularly.
• Treat prompts as governed artefacts. Log, version, and approve system prompts. Ensure they are testable, maintainable, and can be rolled back if they produce unsafe or unintended behaviour.
• Establish memory governance policies before deployment. Define what agents may retain, for how long, where it is stored, and how it is purged. Address conflict resolution for contradictory memory states.
• Build in kill switches and rollback capability from day one. Fail-safes are not optional additions — they are a design requirement. Ensure that the system can be safely halted and the last known safe state restored.
• Invest in continuous evaluation and monitoring. Plan for ongoing testing against adversarial scenarios, anomaly detection, and a control tower capability that provides real-time visibility across the entire agentic system.

Agentic AI has the potential to transform how organisations deliver services and manage complex processes. But that potential will only be realised responsibly if governance keeps pace with capability. The DTA’s addendum is a meaningful step in that direction and for any organisation operating in a regulated environment, it provides a practical and principled framework worth taking seriously.

For more information on how Privacy 108 can assist your organisation with AI governance, privacy impact assessments, or agentic AI readiness, please contact us via the below form.