Interfacing Humans with Intelligent Machines

As machines grow increasingly intelligent, human-machine interfaces (HMIs) are evolving in ways that profoundly reshape how humans interact with technology. This transformation is particularly evident in the realm of user experience (UX) design for websites, software interfaces, and digital tools. The addition of AI is profoundly reshaping how humans interact with technology including how we evaluate the effectiveness of UX designs.

Consider the seemingly simple act of adding an icon or widget to introduce a new software feature. At first glance, this might appear to be a straightforward UX enhancement. However, the history of so-called “assistant” features reveals a pattern of missteps. From the infamous Windows paperclip to more recent attempts by Google to integrate large language model (LLM) AI into Google Workspace applications like Gmail and Google Docs, designers have often prioritized broad utility over thoughtful, context-driven implementation. As a result, users frequently find themselves questioning why these tools insert themselves into workflows uninvited, while simultaneously demanding users provide structured, nuanced inputs to achieve basic results.

This dynamic presents a troubling trend: users seeking AI assistance must increasingly act as managers of the AI’s capabilities, tasked with crafting precise prompts to achieve the desired outcome. This shift occurs partly because modern AI tools are now advanced enough to avoid the glaring error messages of the past. Yet, this newfound sophistication has led software designers and product managers to abdicate their responsibility to streamline the user journey. Instead of guiding users through intuitive, seamless experiences, they’ve left them to navigate a labyrinth of potential AI interactions without a clear map.

Broadening the Scope of UX to HMI Design

For those familiar with Steve Krug’s seminal UX book, “Don’t Make Me Think,” the guiding principle of UX design is clear: reduce cognitive load and deliver what the user wants with minimal effort. Modern AI implementations often do the opposite. Imagine a word processor with a toolbar button labeled “AI Assistant” that opens a broadly capable tool, but offers no hints on how to use it effectively. Worse still, the tool might be constrained behind the scenes in ways that users aren’t informed of, leading to a frustrating guessing game about what the AI can and cannot do. Early iterations of Google Workspace’s AI assistant fell into this trap—users were left confused and frustrated by the tool’s limitations and inconsistencies.

The unfortunate consequence is many users bypass these built-in AI tools in favor of more generalized solutions like ChatGPT. While ChatGPT may be overly broad, at least it doesn’t come with hidden zones of incapacity that disrupt user workflows. This preference underscores a critical lesson for UX designers: providing users with an assistant that appears universally capable but behaves inconsistently is a recipe for frustration. Instead, the focus should be on creating contextual, task-specific AI integrations that anticipate user needs and reduce the cognitive burden—not increase it.

Quick aside: This series is mostly discussing user experience (UX) design and product management implications of AI. AI is often generically referred to as machine intelligence, and it has already transcended our rectangular screens, making its way into audio interfaces, phones, and robots - humanoid and not. We touch on HMI here to begin broaden our thinking to include these user experience contexts.

Here are some other key considerations to keep in mind given the changes in how people perceive and use machines, as well as some specific UX techniques we can apply to meet the needs of both human and machine in today’s HMI context.

From Tools to Partners

The evolution of machines from tools to full-fledged partners is dependent on clear bilateral communication, reliability of the machine, and trust of the human. All of these factors are currently works in progress. Thoughtful design can help bridge the gaps and foster productive partnerships.

Key Considerations:

• Ensure the system maintains transparency in its actions and suggestions to foster the users’ trust.
• Balance control by letting users override or guide the system when needed.
• Avoid overwhelming users with unnecessary suggestions or complexity.

UX Techniques:

• Implement explanatory UI elements (e.g., "Why we suggested this" tooltips) for transparency.
• Use progressive disclosure to simplify user options while allowing access to more advanced features.
• Employ contextual guidance, such as tailored onboarding or real-time help, to align with user goals.

Natural and Intuitive Interfaces

Key Considerations:

• Prioritize accessibility and inclusivity when designing voice, gesture, or tactile interfaces.
• Account for errors or misunderstandings in recognition, ensuring a forgiving and flexible interface.
• Design for natural interaction without requiring users to learn complex commands or patterns.

UX Techniques:

• Use conversational design principles for voice interfaces, such as turn-taking and error recovery strategies.
• Incorporate gesture libraries for intuitive, efficient, and culturally appropriate physical interactions. Also, if accessibility is important in your application, consider that gesture accessibility is a developing concept and that extra effort and new strategies may be required of the UX designer(s).
• Implement feedback mechanisms, such as visual or auditory cues, to confirm successful inputs. Whether your application enables parallel processing of multiple human requests, linear interactions only, or a combination, ensure you let the users know clearly which modes they are experiencing and the progress of their request. If parallel request processing is supported in your application, ensure clarity of thread status and navigation.

Personalization and Adaptability

One of the most seductively easy ideas is personalization, driven by the AI’s ability to learn our patterns through interactions with us, and then cater experiences accordingly. There is an unproven and potentially annoying side effect worth discussing: it is difficult to make AIs forget selectively enough to ensure the user has granular control over what the AI thinks that the user likes or wants. Here are some things to think about:

Key Considerations:

• Ensure users have control over the degree of personalization and can easily reset or modify preferences.
• Avoid over-reliance on assumptions that might result in incorrect or intrusive personalization.
• Prioritize security and user consent when collecting and using personal data.

UX Techniques:

• Provide dashboards where users can adjust personalization settings.
• Use AI-driven recommendation systems with clear explanations and options to refine results.
• Conduct user testing to validate personalization features and identify edge cases.

Proactive and Predictive Interactions

AI-generated predictions can sometimes be inaccurate, making it essential to design for error management. This is why it’s particularly important to ensure that when the AI makes a mistake, it does not alienate the human.

Key Considerations:

• Proactively-offered suggestions should feel relevant, timely, and unobtrusive.
• Ensure the system gracefully handles incorrect predictions to maintain user confidence. This topic warrants a dedicated discussion, which we plan to cover in future articles. A partial list of such techniques includes transparent explanation of rationale behind the prediction, ensuring the user has the ability to ignore or reverse predictive recommendations or AI actions and provide feedback so the AI can do better next time.
• Design fallback mechanisms for when the predictive model fails or isn't applicable.

UX Techniques:

• Use non-intrusive notifications (e.g., banners or subtle suggestions) to present predictions.
• Implement undo functionality to correct or revert system predictions.
• Use A/B testing to refine when and how proactive features are deployed.

Seamless Integration Across Devices and Platforms

If we aspire to create a sense of a highly present and engaged AI assistant, accessibility, continuity, and consistency of interactions is critical.

Key Considerations:

• Maintain consistency in interaction patterns and visual design across all devices.
• Design for continuity so users can transition tasks smoothly between devices.
• Account for varying hardware capabilities and screen sizes.

UX Techniques:

• Use responsive design principles for adaptable layouts.
• Implement cloud-based state synchronization to save and resume tasks across devices.
• Conduct cross-device usability testing to ensure a cohesive user experience.

Challenges and Ethical Considerations

Teams work best when team members adhere to a common set of values. The same can be said of the human - machine team. To the degree that the context makes AI expression of values meaningful, care should be taken to create that feeling of value alignment.

Key Considerations:

• Prioritize transparency about how data is collected, stored, and used.
• Ensure that AI-driven decisions align with ethical guidelines and user expectations.
• Minimize the risk of over-reliance on intelligent systems by supporting user autonomy.

UX Techniques:

• Include privacy dashboards where users can view and manage data usage.
• Provide ethical design choices, such as clear opt-in mechanisms for data collection.
• Use human-in-the-loop frameworks to allow users to review and confirm critical decisions.

Blurring Boundaries Between Physical and Digital

Multiple streams of R&D are bringing AI and virtual environments ever closer to widespread public use. Beyond the obvious entertainment value, this convergence opens up promising opportunities for training and productivity applications in high-stakes contexts. To deliver on this promise, we in the traditional UX world will need to extend our thinking and our techniques into the third dimension.

Key Considerations:

• Design for intuitive interaction in VR or mixed-reality environments, ensuring clear affordances.
• Avoid overwhelming users with excessive virtual elements or disjointed transitions between physical and digital.
• Consider motion sickness, fatigue, and physical comfort for immersive interfaces.

UX Techniques:

• Use spatial design principles for VR/AR interfaces to guide users through digital environments.
• Implement haptic feedback to reinforce digital interactions with tactile sensations.
• Test with usability labs for XR to evaluate interaction intuitiveness and comfort.

Democratization of Technology

Google started the simplicity revolution with its single input box. AI tools are making that single box do more, and maybe it will eventually do everything. But, humans are visual creatures, so will we really go back to the age of the command line simply because the command line can now deliver better output? Perhaps, but before we in the product design community give up the ghost, it’s worth remembering that people use all their senses all the time. We can be reading something while listening to music, or speaking with a friend while walking (or not walking) across an intersection based on the traffic light color. Let’s see if we can blend the conversational interfaces with visual and other cues and interactions to make for a richer, more productive experience.

Key Considerations:

• Simplify interfaces without compromising functionality for novice and advanced users alike.
• Use visual and language-based cues to support universal comprehensibility.
• Design inclusively for diverse demographics, including non-technical users.

UX Techniques:

• Create low-code/no-code toolkits with templates and drag-and-drop functionality.
• Use internationalization and localization techniques to adapt interfaces for global audiences.
• Conduct field studies to gather insights on how diverse user groups interact with the system.

Conclusion

You might have noticed a repeating theme across these categories: he most potent obstacle to a business’s ability to capitalize on AI utility is distrust and even fear, and this will become a more and more acute issue as AI capabilities grow. Ensuring the user always feels in control is key, and in most cases, that means that - going beyond feelings - the user must be in control. This will require a collaboration between UX, Engineering, and Product Management that is focused on this challenge.

We are still in the early days, but as the landscape evolves and our experience with this new wave of technology grows deeper, we will update our perspective and recommendations.

If you haven’t read our previous blogs in this ongoing series, you can read them here:

UX Meets AI: What Designers and Product Managers Need to Know

Exploring the Challenges AI Brings to UX Design