• Criticality – There is a high risk associated with making the wrong decision. Once taken, actions are hard to reverse.
  By contrast, if the risk is low and actions can easily be rolled back, users might not need an explanation for the system proposal.

• Complexity – It’s hard for users to assess the impact and quality of their decisions. This might be due to a lack of process insight because of the automation of this task with machine learning.
  By contrast, if users can easily tell when a proposal fits (without training), they probably don’t need additional input.

• Transparency – Your business case is subject to extensive auditing requirements. Auditors must be able to trace transactions back and see the underlying reasoning for each execution step. Auditors may require more information than end users.
  By contrast, if there are no auditing requirements, explanations might not be necessary (assuming they are also not required by end users).

• Volatility – To adapt your machine learning model to changing conditions or requirements, your AI application relies on continuous feedback.
  By contrast, if the feedback has little or no effect on the output of the algorithm or the user experience, it can be more distracting than helpful to provide an additional explanation.

Level 1: Indicator (What?)
The minimum explanation level. An indicator is required whenever AI (machine learning) output is provided. The indicator is also the access point for the next explanation level (if required).

Level 2: Abstract (Why?)
A condensed view of the relevant properties, amounts, and contextual information. An abstract helps users to better understand the AI proposals. It can contain links to the last and most detailed explanation level.

Level 3: Detail (How?)
An extended report specifically for advanced users. It covers all aspects processed by the intelligent system, the AI performance, and any further context and conditions that help users to monitor AI operations.

Global – Model level
Explains the AI output for a group of elements, such as a list, table, or whole section. The global explanation provides the overall context for more complex constellations affecting multiple objects or aspects.

Local – Item level
Explains the AI output for a single value or item.

Global Indicator

Local Indicator

Default Style
The default style provides the minimum amount of indication about the type of AI output. It uses explicit and descriptive labels and blue coloring for clickable explanations.

The default style offers sufficient visual affordance for non-critical AI applications.

Extended Style
The extended style uses semantic colors and to offer more contextual insight at the first explanation level. It can also include other semantic elements, such as an icon to indicate a trend.

The extended style can further improve the affordance for the user and emphasize the importance of the information provided.

Confidence Indicator – Best Practices 

Level 2: Simple Explanation

Explanation Popover/Overlay

Users access the explanation popover by clicking the explanation indicator. The popover is a canvas to provide contextual information beyond quality ratings, and to list or visualize key factors, inputs, or criteria that led to the AI result (for example, you could use a model induction algorithm or “Explainer” to reveal which data best supports the prediction).

Explanation popovers, screen overlays or conversational items can include the following elements:

• Self-explanatory header title
• Minimum: Short explanation of what the AI has performed in natural language.
• Minimum: For AI outputs that are part of a scale, offer a key or legend to explain how the current value should be read.
• Recommended: A timestamp indicating when the result or prediction was computed. If the output is time-critical, show when the result or prediction was last updated.
• Recommended: A list of the 3-5 most important parameters that influenced the result being explained. This can include a comparison with averages or reference values, if required.
• Optional: A short explanation in natural language of how the AI has performed in comparison to a given reference (historical cases, given data, average prediction accuracy, or similar). This can be combined with performance indicators or a trend.
• Optional: In-place feedback controls if you want to collect explicit user feedback. For more information, see User Feedback.

Explanation Page

Dedicated Explanation App

Static Explanations

Static explanations are suitable for “offline learning” or instance-based AI services. We assume that the need for a repeated explanation of the model can decrease or even disappear over time as the user gains more experience with the system.

Dynamic Explanations

Dynamic explanations are required for “online learning” AI services. This type of explanation is generated (refined) every time the AI has learned and adapted its model. In most cases, it’s not necessary to change the appearance and content of the explanations completely. However, if an AI service discovers new rules to process items in the system, this newly-acquired knowledge must be reflected in its explanations.

• Does the user expect an explanation?
  If the risks associated with an action are quite low and the results can easily be rolled back, users are not normally interested in an explanation of the system proposal.
• What level of automation are you aiming for?
  Depending on the automation level, the use case and target user roles and app functionality can change dramatically.
• Which other roles are involved in the experience besides your target business user?
  The interaction with AI systems also involves technical, non-business roles. Also consider roles involved in development, support, or maintenance.
• How transparent or traceable does your machine learning service have to be?
  Some use cases are more critical than others. Due to legal and regulatory provisions, enterprise applications are often subject to stricter auditing requirements than consumer software.
• Are your users generally able to relate to the displayed information and data, and can they infer the follow-on actions and impact (detail and explicitness)?
  Although AI can help optimize even simpler tasks, it might be overengineered to provide AI explanations when the user fully understands the results and implications.
• How harmful can wrongly processed data be for the business of your client?
  Some actions in processes are critical by design and can have serious side effects. AI explanations must safeguard such scenarios and help prevent any damage or disruption.
• How easy or difficult is it to revert changes applied to the system or process?
  In scenarios where the user faces deadlines, period-end closing, or other actions that must succeed first time, providing information to support the user is crucial. However, may not need explanations if it is possible to just revert everything instantly in case of failure.
• Does your business case require continuous adaptation?
  We assume that the need for a repeated (static) explanation of the model can decrease or even disappear over time as the user gains more experience. However, if the AI learns dynamically, users must always be informed of changing conditions (dynamic explanations).