Agent UX (AUX): Designing for the Next User

The definition of 'user' has changed. AI agents are no longer tools for the user — they are the user. And our design practices haven’t caught up.

We are entering a new era where intelligent AI agents, autonomous, goal-driven, and web-native, are navigating digital experiences alongside us. These agents are not simply assisting users; they are acting as users. They shop, search, click, compare, and decide. And increasingly, they do so invisibly, on our behalf.

To understand what this looks like in practice, watch Google’s Project Mariner in action. In this demo, the AI agent receives a goal: order missing ingredients for a recipe, and completes the task through a series of autonomous interactions with a real e-commerce interface.

What you are watching is a new kind of user: one who reads the DOM, not the copy. One who reasons across semantic categories, not screen layouts. One who does not care if your button is pretty, only if it is clear.

I call this new discipline Agent UX, or AUX: the practice of designing digital experiences for both humans and intelligent agents. AUX is not speculative. It is becoming foundational. It challenges many of our existing assumptions about how interfaces function and introduces a new layer of design responsibility for automation, semantics, and structural clarity.

Within AUX, we evaluate agentability—the degree to which an interface is understandable, navigable, and operable by AI agents. Just as we measure accessibility or usability, we must now consider agentability as a core metric of experience quality, of crafting digital experiences that are understandable, navigable, and actionable for intelligent agents and humans. It challenges many of our existing assumptions about how interfaces function and introduces a new layer of design responsibility for automation, semantics, and structural clarity.. Because for the first time, product design must consider the needs of two personas: the human and the agent.

How AI Agents Interact With Interfaces

Let us begin with the agent. Using Project Mariner as our reference point, we can map how such systems understand, plan, and act within a user interface. This view is critical for designers to internalize.

1. Information Hierarchies

• Semantic Understanding: Mariner interprets content with contextual meaning. It recognizes that "flour" belongs to baking, and "chicken breast" is a protein, without relying on string-matching.
• Ingredient Recognition and Parsing: The agent comprehends structured information like "2 cups of flour" or "500g of chicken" by parsing numerical data, units, and context.
• Recipe Awareness: It understands that recipes have structured roles: ingredients, steps, and categories.
• Website Structure (DOM): The agent parses HTML tags like <input>, <button>, and <img>, while ARIA attributes enhance semantic clarity. CSS classes like product-title or add-to-cart-button help it form associations.
• Navigation Paths: Elements like breadcrumbs and nav menus convey logical site structure.

2. Architectural Components

• Perception Module: Processes DOM, text, and visual signals, identifying interactive elements.
• Cognitive Module (Gemini 2.0): Plans tasks, breaks goals into sub-goals, and reasons with contextual knowledge.
• Action Module: Executes clicks, inputs, and scrolls using a browser API.
• Learning Module: Learns from feedback and demonstration to improve over time.

3. Consistency as a Design Advantage

• UI consistency (labels, patterns, ARIA use) improves both human usability and AI comprehension.
• Predictable data structures allow faster parsing and greater task completion accuracy.

4. Component Awareness

• Vision models interpret images and layout.
• LLMs process language and formulate plans.
• Structured APIs help verify state transitions or interpret edge cases.

Designing for Dual Personas

This shift is not theoretical. It impacts automation reliability, accessibility, SEO, and even conversion rates. If we do not design for agents, we are designing for obsolescence.. How can we, as designers, create experiences that are usable, meaningful, and interpretable for both humans and intelligent agents?

This duality introduces a real problem: we are no longer designing for a single interpretation of interface elements. Humans bring emotional context, cognitive flexibility, and visual interpretation to UI. AI agents, on the other hand, rely on structured semantics, consistent patterns, and deterministic feedback loops. What works intuitively for one may be ambiguous or even invisible to the other.

If we fail to consider the agent as a legitimate user, we risk building products that perform poorly when automated tasks are introduced, or worse, exclude a growing class of assistive and automated tools that users depend on. Overlooking this tension means missed conversions, broken automations, inaccessible experiences, and an overall lack of resilience in the interface.

As AI agents take on increasingly critical tasks such as shopping, scheduling, searching, and assisting, designing without them in mind introduces significant risk. It limits the scalability of experiences, undermines automation reliability, and leaves products brittle in the face of new interaction models. In the same way that accessibility was once viewed as a niche concern but became a hallmark of quality, agent-awareness must become a baseline for resilient, forward-thinking design.

1. Semantic Clarity and Structure

• 🧑🏽 For humans: intuitive layout, visual hierarchy, readable labels.
• 🤖 For agents: use semantic HTML5 (<main>, <nav>), consistent ARIA attributes, and structured data (e.g. Schema.org or JSON-LD). Avoid dynamic DOM patterns that confuse parsing.

2. Explainability and Feedback

• 🧑🏽 For humans: feedback messages, progress indicators, success confirmations.
• 🤖 For agents: clear state changes ("Added to cart"), consistent error states, and potentially APIs that confirm outcomes.

3. Predictability and Robustness

• 🧑🏽 For humans: smooth UX, graceful degradation, accessibility.
• 🤖 For agents: stable IDs, predictable DOM structure, tolerance for minor UI changes. Avoid misleading or duplicate buttons.

4. Oversight and Control

• 🧑🏽 For humans: undo flows, preview steps, privacy choices.
• 🤖 For agents: transparent audit trails, confirmation checkpoints, and clearly separated task scopes (browse versus purchase).

5. Performance and Visibility

• 🧑🏽 For humans: responsive UI, minimal load time.
• 🤖 For agents: consistent rendering and accessibility of core elements. Avoid lazy loading critical buttons without triggers.

Designing for Agentability

Designing for AI agents is not a novelty. It is an emerging requirement. Just as accessibility standards improved the web for all users, so too will designing for agents.

To get there:

• Make semantic HTML and structured data a design system norm.
• Teach engineering teams why consistent labels, ARIA, and DOM stability matter.
• Introduce agentability testing alongside usability testing.
• Prioritize agent-aware flows for high-value tasks like shopping, booking, and scheduling.

The AI agent is a new kind of user. It deserves design attention not as an abstraction, but as a persona.

The future of UX is not just human-centered. It is agent-aware.

If you lead a design or product team, start here: pick one high-impact flow and audit it for agentability.

You will uncover gaps that matter more than you think.