The Agentic UX Audit Checklist: 50+ Checkpoints for AI Agents

By the UXreviewer.io team3 min read

An agentic UX audit checks whether users can safely delegate tasks to an AI agent: whether they can express intent, understand the plan, control execution, see what happened and recover from mistakes. The checklist below contains more than 50 checkpoints in six groups. Mark each as pass, partial or fail, and add a screenshot for every fail.

Use it for a self-review before launch, for comparing your product with competitors, or to prepare an external audit. It complements, rather than replaces, classic heuristic evaluation, so run both if your agent sits inside a larger product.

How to run the audit

Pick three to five realistic tasks, including one ambiguous request and one that is likely to fail. Run each task at least twice, since agent behavior varies. Record the session so you can revisit exact wording. Score each checkpoint per task, then summarize by group to see where the weaknesses cluster.

As a rule of thumb, fix fails in recovery, permissions and transparency first, because they carry the highest risk of harm and lost trust.

Involve at least one person who did not build the agent. Builders know what the agent is supposed to do and unconsciously fill gaps that real users will fall into. If possible, let a support colleague take part too: they know which questions and complaints already reach the team.

For each fail, write down the observed behavior, the expected behavior and a proposed fix. This turns the audit into a backlog the team can act on, rather than a list of opinions. Our articles on error recovery and transparency explain the reasoning behind many of the checkpoints below.

Intent and onboarding (10 checkpoints)

• The product explains what the agent can and cannot do, in concrete examples.
• Users can see example requests that work well.
• Key constraints such as budget, deadline or recipients can be set explicitly.
• The agent asks clarifying questions when a request is ambiguous.
• Clarifying questions are focused, one at a time, with suggested answers.
• Sensible defaults are used and shown, not silently applied.
• The agent restates its understanding before high-impact tasks.
• Out-of-scope requests receive an honest answer and an alternative.
• Users are told they are interacting with AI, consistently across channels.
• Onboarding does not request broad permissions before value is shown.

Planning and control (10 checkpoints)

• Multi-step tasks show a plan before execution.
• The plan is written in user language, not tool names.
• Users can edit, remove or reorder plan steps.
• Autonomy level can be set per task type.
• Approval gates appear before costly, irreversible or external actions.
• Approval prompts state consequences and alternatives.
• Routine low-risk steps do not require confirmation.
• Users can pause and stop a running task at any time.
• Stopping halts server-side actions, not only the interface.
• Spending or volume limits can be set and are enforced visibly.

Progress and transparency (10 checkpoints)

• Current step and progress are visible during execution.
• Waiting states explain what the agent is waiting for.
• Background tasks notify users only when attention is needed.
• Every run ends with a summary of what changed.
• Skipped and failed items are listed with reasons.
• Users can see why the agent made key choices.
• Sources and data used are cited and linked to the relevant passage or record.
• Uncertainty and assumptions are stated honestly.
• A persistent, readable activity log exists.
• AI-generated content sent to third parties is identifiable where appropriate.

Errors, recovery and handoffs (10 checkpoints)

• Failure messages state what succeeded, what failed and the current state.
• Reversible actions offer undo, with a visible time window if limited.
• Compensating actions such as cancellations are guided and state their costs.
• Irreversible actions are clearly labelled as such before execution.
• Partial completions offer retry, rollback or manual continuation.
• Retries do not duplicate completed actions.
• Users can request a human at any point.
• Handoffs to humans carry full context.
• Unanswered approvals have a defined timeout behavior.
• Users can flag a wrong result directly from the summary.

Permissions, privacy and accessibility (12 checkpoints)

This group overlaps with compliance topics. The checklist is a UX aid, not legal advice; see our article on permissions and consent UX for design detail.

• Permissions are requested just in time, with a reason.
• The narrowest useful scope is offered by default.
• Users can see all granted permissions in one place.
• Permissions can be revoked easily, with consequences explained.
• The agent does not access data beyond what the task needs.
• Users can see and delete stored memory or preferences.
• Chat and agent interfaces work with keyboard only.
• Streaming responses and status updates are announced sensibly to screen readers.
• Focus is managed correctly when approval dialogs appear.
• Text contrast and size meet WCAG requirements.
• Time limits for approvals or undo can be extended or are generous.
• Plain language is used throughout, including error messages.

Frequently asked questions