How to scale chatbot development with Google Dialogflow and Snorkel Flow

Zencore, a premier Google Cloud partner, provides expert guidance in integrating advanced cloud and AI technologies with an insider’s understanding of Google Cloud. Co-founder Sean Earley recently investigated how financial institutions could use Google Dialogflow with Snorkel Flow to build better chatbots for retail banking. His investigation produced this guide.

Businesses use chatbots to handle an increasing share of customer interactions. Building these applications presents many challenges. The consequences can be especially steep for financial institutions, where misreading intent can ruin a long-standing customer relationship. In this example, we will outline how Google Cloud solutions and Snorkel AI”s data development platform can be combined to develop highly accurate, large-scale intent-matching capabilities inside of DialogflowCX for a typical retail bank support use case.

Why accuracy matters

Customer service chatbots represent a critical evolution for the financial sector, with significant implications for customer experience and institutional credibility. Financial institutions increasingly rely on chatbots as a cost-effective customer service solution. While chatbots are effective for basic inquiries, their efficacy diminishes with more complex problems, such as interpreting the root cause of the customer’s problem.

As highlighted by research by the Consumer Financial Protection Bureau (CFPB), the stakes are high in the financial domain. Chatbots must provide accurate, compliant information as well as personalized, contextually appropriate responses. Inadequacies in chatbot design or failure to offer support can lead to negative outcomes for customers, including receiving inaccurate information when the chatbot fails to understand their intent or feeling frustrated at receiving an unhelpful, pre-built response. Moreover, poorly designed chatbots that fail to comply with federal consumer financial laws can lead to legal liabilities for financial institutions, erode customer trust, and cause significant consumer harm.

The power of Google’s Contact Center AI

Dialogflow, a pivotal component of Google’s Contact Center AI, has proven to be a game-changer in the consumer financial services sector. Innovative financial institutions like HSBC and Commerzbank have successfully utilized DialogflowCX to ease call burden on policy experts and enhance customer service experiences.

This AI-driven platform is particularly effective in handling the wide spectrum of customer interactions typical in consumer finance, from straightforward account inquiries to intricate financial consultations. DialogflowCX excels in enabling natural, multi-turn dialogues, allowing virtual agents to navigate various topics, conduct transactions, and provide consistent, around-the-clock service across numerous channels. This feature is indispensable in satisfying the modern financial consumer who expects quick, accurate, and personalized responses.

Using a data-centric approach to building chatbots

Chatbot performance, particularly in the nuanced domain of consumer financial services, fundamentally relies on the quality of the datasets used to build them. These datasets are critical as they train the chatbot to accurately recognize and respond to customer intents, a core functionality of DialogflowCX. The better the annotation and coverage of the dataset, the more capable and responsive the chatbot is in handling a diverse array of customer inquiries, from simple transactional requests to complex financial consultations.

This is where Snorkel AI comes in. With the Snorkel Flow data development platform, teams can annotate conversations with a level of efficiency and scale that was previously unattainable. Snorkel’s programmatic approach to labeling data allows a typical team to process and annotate as many conversations as a team ten times its size, and address a long-tail of hundreds of possible intents or rapidly changing responses.

Snorkel provides the capability to integrate generative AI technology alongside humans to amplify their effectiveness, such as in this example using Google’s PaLM 2 model. This efficiency translates to a reduction in labeling time from months to just days, a dramatic improvement. Snorkel also has experience working with seven of the top 10 US banks on many use cases, including on chatbots. JP Morgan Chase bank used an early version of Snorkel technology to update its intent model to handle rapidly changing customer needs during the COVID-19 pandemic.

Integrating Snorkel Flow and Google Dialogflow

Snorkel Flow integrates natively with BigQuery to access unstructured data in Google Cloud.  Intents can then be labeled programmatically using a data-centric AI workflow to quickly generate high-quality training sets over complex, highly variable data. A quick overview of the workflow is as follows.

1. Enable Interaction Logging to BigQuery

First, you will want to configure Dialogflow interaction logging, which will store redacted end-user queries that you can use as input to your training dataset.

From within the DialogflowCX console, go to the agent settings page for your agent. From there you will want to select “Enable interaction logging” and “Enable BigQuery export”. Provide the details of your destination BQ dataset

2. Configure the Snorkel Flow BigQuery integration to ingest your interaction data

From within the Snorkel Flow GUI, select “Google BigQuery”’ as a data source to create a new dataset. Enter BigQuery connection details and credentials.

Snorkel Flow will then ingest the dataset, making it immediately referenceable throughout the platform.

3. Prepare your annotated Training Dataset in Snorkel Flow

Snorkel AI’s data development platform equips machine learning teams to efficiently work with internal experts and programmatically label queries, ultimately generating high-quality training sets.  This data-centric approach has enabled Snorkel customers to build AI services 100x faster than manually labeling data.

4. Export Data from Snorkel Flow

Within Snorkel Flow, data is divided into two or more “splits” depending on the complexity of the project.  Labeled data can be exported from Snorkel Flow using the “Export” button at the top of the page.

Typically data is exported in a comma delimited *.csv file, though users can extensively customize the export (including exporting additional information such as how each labeling function voted, comments left by SMEs etc.) using the Snorkel Flow notebook.  

In this case, a simple *.csv file works perfectly.

The *.csv can be exported natively from Snorkel Flow with the following columns:

• Intent Display Name
• Language
• Phrase

Each entry contains either the display name, language, and first training phrase or just a training phrase for the previously declared intent.

Dialogflow encodes annotations in exported training phrases so that annotations are restored when importing. The format for this encoding is:

(annotated part)[entity, parameter]

For example:

Intent Display Name,Language,Phrase

Shirt Selection,en,I want a (green)[@sys.color, color] shirt

,,I would like a (yellow)[@sys.color, color] shirt

Store Hours,en,When are you open?

,,What are your hours?

5. Import your annotated intent dataset to Google Dialogflow

After the creation of your prepared *.csv, this can be imported to your Dialogflow agent from the intent management interface.

When importing intents, there may be merge conflicts when the display name for an intent in your existing agent matches that of an imported intent. You can control the merge behavior when intent display names match by selecting one of the following:

• Replace existing intents: An imported intent overwrites any same-named intent in your existing agent.
• Rename and import as new intents: The imported intent is renamed by appending “_1” to the display name.
• Merge with existing intents: The training phrases of an imported intent are added to the existing intent. If the same training phrases exist, they will not be duplicated.
• Keep original intents: The existing intent remains unchanged, and the conflicting intent is ignored. Nonconflicting intents are imported.

6. Train your Dialogflow Model

For a large complex agent with many intents, you will likely want to use the Advanced NLU type. Auto-training may make the console lag for large flows, so it is recommended for this use case to have it disabled. To manually train your flow from the console select Train.

How Zencore can help with Google Dialogflow and more

At Zencore, we leverage our extensive experience as former Google Cloud leaders, architects, and engineers to provide expert guidance in integrating advanced AI technologies like Dialogflow. Our approach involves hands-on, collaborative work alongside your team, ensuring strategic planning, execution support, and valuable knowledge transfer. With our insider’s understanding of Google Cloud technologies, we simplify complex processes and de-risk technical decisions, preventing costly delays and technical debt. Our deep expertise in AI and cloud solutions, combined with a focus on client success, makes us an ideal partner for projects requiring sophisticated, AI-driven solutions.  To learn more, set up a free consultation.

Sean is a co-founder at Zencore and leads the AI, Infrastructure, and Cloud Migration services practices. Prior to Zencore, Sean worked at Google helping customers build innovative products, applications, and capabilities on Google Cloud. Sean lives in Brooklyn, NY. e