Einstein Analytics (EA) provides Salesforce customers with the ability to build analytics dashboards and apps that deliver business insights. While it’s incredibly powerful, EA can feel intimidating for novice users who have little experience in analytics and/or Salesforce. With this gap in mind, how could we make Einstein Analytics more accessible so users can reap the benefits regardless of their skill level?
Interacting with a machine using plain, non-technical language is tricky. When it works, it’s magical; but when it doesn’t, it’s incredibly frustrating and inefficient. As the team began working on this project, we realized that achieving a completely “natural” language interaction would be extremely difficult to pull off, given the limited resources we had at hand.
We have five Analytics personas across the board and for this feature in particular, we’re primarily focused on the analyst and line-of-business-user, who are consumers of analytics apps and dashboards. Our secondary focus is the admin, who build apps and dashboards for their organizations.
In our initial research around the topic of natural language query, we talked to users who fit these profiles and asked them what they would expect from it. Without seeing any UI, we found that users had extremely high expectations, wanting it to answer complicated questions such as “how do I get more profit?”
With this in mind, the UX team suggested that we start with a simpler first version. High-level queries such as “how am I doing?” is difficult for the machine to parse and answer without a high quantity and quality of data, analysis, and intelligence. On the other hand, simpler template-based queries such as “show me top 5 accounts by revenue” can be more easily parsed, even in the absence of “smart” machine analysis.
This type of query is also metadata-based, so it’s scalable across different types of datasets. The underlying logic between “show me top 5 accounts by revenue” and “who are the bottom 10 agents by customer satisfaction score” are similar, i.e. the sentence pattern is reusable. Last but not least, this query type covers a good portion of potential questions a user may have about their data, which makes it ideal as a starting point.
(Original concept by JD Vogt)
We also realized that the right UI context would be critical in grounding the user’s expectations. The higher it sits in the UI, the higher the expectation is. We decided to embed the query field in two specific parts of EA: the explorer (where the user explores/plays with a chart to discover further insights) and the builder (where the user builds apps and dashboards). The specificity of these contexts communicates to the users that they can only ask questions that are narrower in scope.
I started with two main user flows:
- Creating a new chart using conversational query (builder use case)
- Exploring a chart within an existing dashboard using conversational query (explorer use case)
I also defined some restraints. First, creating a chart and constructing a query is not reciprocal, i.e. the user can create a chart by submitting a query, but they cannot create/modify a query by creating/modifying a chart. Second, each query is independent of the next, i.e. the user can’t ask related questions one after another, e.g. “What’s the average deal amount?” > “What about ACME?” > “In East Coast?”
Though desirable, these functionalities are more “nice-to-have” than critical. Doing away with them for the first iteration significantly reduces engineering complexity without severely impacting the experience.
One Field, a Thousand Micro-Interactions
Designing for conversational query presents some unique challenges due to its minimal UI. The user is basically interacting with one field instead of a host of UI components (buttons, fields, toggles, dropdowns, etc).
As a result, every single keystroke and every move of the cursor is highly consequential—any small bump in the interaction feels much more significant than they would in a full-screen UI.
Additionally, after showing a rudimentary prototype of this feature during the second round of research, we found that trust remained a major issue for users. For conversational query to be successful, it has to guide users every step of the way and gradually guides them towards insights.
I’ve seen features like this in the past and they usually give generic answers.
—Call Center Supervisor
After trying out some approaches, I eventually created a set of interaction flows to show all the different micro-interaction patterns, including base patterns for keyboard and mouse as well as more complex query logic (e.g. how to show filter values, relative dates, etc.).
This method led to more productive discussions between UX, Engineering, and PM; in particular, it helped Engineering understand and plan their work more effectively. More importantly, however, it helped us focus on delivering a highly guided experience for the users and lessen their concerns. These schemas went through multiple rounds of iterations, which were informed by continuous feedback, both internally and from users.
As the base experience took shape, we were able to gradually add improvements that enhance the feature.
- Robust Synonyms
A salesperson might call potential sales “opportunities” while another might say “deals.” We added a set of synonyms to accommodate variations like this. - Efficient Shortcut Phrases
In sales, an “opportunity won” is the same as “opportunity where stage is ‘closed won.’” We added shortcut phrases that are tailored to frequent scenarios so the queries are more efficient and natural-sounding. - Specific Error Messages
A submitted query may contain errors; for instance, the question “opportunities greater than” is invalid without a numeric value. We identified a number of common but specific error scenarios like this and provided targeted error messages to help the users.
To create/edit analytics apps and dashboards, an EA user previously had to either write code or use declarative tools. Although declarative tools have made it easier to build charts, it still takes 12 clicks on average and requires some prior knowledge of said tools. Conversational queries breaks down this barrier by enabling less technical users to ask questions about their data directly.
The beta version of conversational query was officially unveiled in early 2018 and the initial response has been incredible. That said, there’s plenty of room for improvement. To begin with, we’re exploring ways to answer higher-level questions (e.g. “how do I maximize profit?”) so users can get even deeper insights from their data. We’re also looking into organization-level customizations that would enable custom synonyms that are unique to each org (think of your company’s internal acronyms, for example). There is a ton of potential within this space and we’ve only just begun to scratch the surface.
Conversational Queries is going to be a huge game-changer in the AI and BI space. Having the ability to ask questions and gain key insights into our data just by typing or speaking a few words is amazing.
—Rick Nania, Director of CRM Operations at Active International