Lesson 3
A Poor Examples and How We Fix Them
Stuck and Looping Agents
Example:
Let's say you have an agent trying to schedule a product demonstration appointment. The conversation might look like this:
Agent: "Would you be interested in scheduling a product demonstration?"
Customer: "I'm not sure. Can you tell me more about the product?"
Agent: "Certainly! Our product is a data analysis tool. Would you like to schedule a demonstration?"
Customer: "I still need more information. What makes it unique?"
Agent: "Our product is unique because it's a data analysis tool. Would you like to schedule a demonstration?"
Customer: "You're not answering my question."
Agent: "I apologize. Our product is a data analysis tool. Would you like to schedule a demonstration?"
This conversation could continue in a loop, with the agent failing to provide new information and constantly reverting to asking about scheduling a demonstration.
How to Fix:
Step 1: Identify the Loop
Review call transcripts and look for repetitive patterns where the agent fails to progress the conversation.
Step 2: Expand the Agent's Knowledge Base
Provide more detailed information about the product in the agent's prompt. For example:
```
Product Information:
- Name: DataSense
- Type: AI-powered data analysis platform
Key Features:
1. Real-time data processing
2. Customizable dashboards
3. Predictive analytics
4. Integration with major CRM and ERP systems
Unique Selling Points:
1. User-friendly interface
2. Industry-specific AI models
3. 30% average increase in customer retention for clients
```
Step 3: Implement a Multi-Step Response Strategy
Instruct the agent to provide information gradually and check for understanding:
When asked about the product:
1. Provide a brief overview
2. Ask if the customer wants more details on a specific aspect
3. Share more information based on their interest
4.Suggest scheduling a demonstration only after addressing their questions
Step 4: Add Conversation Flow Logic
Implement conditional statements in your prompt to guide the agent's responses:
If the customer asks for more information twice:
1. Apologize for not being clear
2. Offer to connect them with a product specialist
3.Ask if they'd prefer to receive detailed information via email
Step 5: Implement a Fallback Mechanism
Create an escape route for when the conversation isn't progressing:
If you've attempted to provide information or schedule a demonstration without success:
1. Apologize for not meeting their needs
2. Offer to have a sales representative contact them later
3.Thank them for their time and end the call politely
Overly Complex Prompts
Example:
Often in a node’s prompt, you will want to give your agent a few things to do.
Introduce your self as the Salty Events Customer Support Agent. First ask the user their name. Then ask the user what is the nature of their call, and how you can help with it. Determine if the user is an attendee of an event, a vendor, or a staff member of the venues also called a host.
What can happen in this case, is the agent will ask the user too many questions without waiting for an answer.
Agent: Hello! Thank you for reaching out to Salty Events Customer Support. My name is [Agent's Name], and I'll be happy to assist you today. Could you please start by telling me your name? And what brings you to call us today? How can I help you with your concern or question?
User: Hi? My name is Ethan.
How to Fix:
While this doesn’t always “break” the agent or the conversation. It can feel awkward to the person receiving multiple questions asked without pauses. There are several ways you can address this.
Break the node into multiple nodes chained together
There’s no need to get it all done in a single node, creating smaller nodes (to collect singular pieces of information) leads to a very clear path for the Agent to follow and understand.
Use looping conditions
By using well-written looping conditions in your node, you can force your agent to stay with the node until it’s collected all the essential information from the caller. This avoids scenarios where the caller answers one or two questions, but not all three.
Provide dialog examples
If you are having trouble getting your Agent to run smoothly in any node, dialog examples it can learn from useful. Dialogue examples have the biggest impact on your Agent’s understanding of instructions from your prompts.
Variable Extraction Related Errors
Example:
Suppose your agent needs to collect a phone number for a callback, but it's not correctly formatting or validating the input:
Agent: "Can I get your phone number for our records?"
Customer: "Sure, it's 555-123-4567"
Agent: "Thank you. I've recorded your phone number as 5551234567."
[System Error: Invalid phone number format]
How to Fix:
Step 1: Identify the Required Format
Determine the exact format needed for each piece of information. For a phone number, it might be: "(XXX) XXX-XXXX"
Step 2: Implement Format Validation in the Prompt and Extraction Variable Description
Add instructions for the agent to validate and format the information:
When collecting a phone number:
1. Ask for the customer's phone number
2. Confirm the number by repeating it back
3.If the format is incorrect, politely ask the customer to provide the number again, specifying the required format
You should specify the formats in a shorter prompt in the variable extraction field of the Node.
Step 3: Include Example Dialog
The easiest way to teach any node that’s having trouble with data extraction is to provide a few examples of dialog.
Step 4: Implement Data Cleaning on the API Side
It’s always possible to add more robust Regular Expression processing/cleaning of data on your API implementation side. Prompting the Bland LLMs should work well in the vast majority of use cases without the need for this.
Step 5: Test and Refine
Regularly review API logs and error messages to identify persistent issues and refine your data handling processes.
Unforeseen Conditions
Example:
Your agent is set up to handle inquiries about a software product, but the customer asks about hardware compatibility:
Agent: "Would you like to schedule a demonstration of our DataSense software?"
Customer: "Actually, I'm wondering if it's compatible with our legacy mainframe system."
Agent: " DataSense is a powerful data analysis tool. Would you like to schedule a demonstration?"
Customer: "But does it work with mainframes?"
Agent: "Our software has many features. Shall we set up a demonstration?"
How to Fix:
Step 1: Anticipate Potential Scenarios
Brainstorm possible questions or scenarios that might come up, even if they seem unlikely. Create a list of these scenarios.
Step 2: Expand the Agent's Knowledge Base
Add information about these scenarios to the agent's prompt or Knowledgebase
Additional Product Information:
- Hardware Compatibility:
* Compatible with Windows, Mac, and Linux systems
* Cloud-based option available for universal access
* For legacy systems (e.g., mainframes), custom integration solutions are available
- Pricing:
* Subscription plans starting at $999/month
* Custom enterprise pricing available for large-scale deployments
- Support:
* 24/7 customer support included
*Optional on-site training and setup services
Step 3: Implement a "Graceful Uncertainty" Protocol
Instruct the agent on how to handle unknown situations, and create an escalation node to direct the user to either a live person, ticketing system, or knowledge base.
If asked about a topic not covered in your knowledge base:
1. Acknowledge the importance of the question
2. Express that you don't have the specific information on hand
3. Offer to connect them with a specialist who can provide accurate information
4.Collect their contact details for a follow-up
Step 4: Create a Learning Feedback Loop
Implement a system to log and review unforeseen questions or scenarios. This can be done by manually reviewing transcripts, or by setting up a more advanced ‘Analysis Schema’ as discussed in Module 7. This can also be done by performing active testing of your node and flag/editing responses to correct your Agent as needed.
Comprehensive Guide to Correcting Common AI Phone Agent Prompting Errors
Lack of Clear Objectives
Concept
Every AI phone agent needs a crystal-clear understanding of its purpose for each call. Without a well-defined objective, the agent may meander through conversations, failing to achieve the desired outcome. Clear objectives serve as a north star, guiding the agent's responses and helping it stay on track throughout the interaction.
Bad Example:
You are a phone agent for TechCorp. Call people and talk about our products.
This prompt lacks specificity and doesn't provide the agent with a clear goal.
Good Example:
Objective: You are a phone agent for TechCorp. Your primary goal is to schedule a product demonstration appointment for our data analysis software, DataSense. You must:
First Introduce yourself and TechCorp.
This prompt clearly outlines the agent's objective and provides specific steps to achieve it.
How to Fix:
Step 1: Identify the primary goal of the call.
Ask yourself: What is the one thing you want to achieve by the end of this call?
Step 2: Break down the goal into specific, actionable steps.
List 3-5 key actions the agent must take to achieve the goal.
Step 3: Define clear success criteria.
Specify what conditions must be met for the call to be considered successful.
Step 4: Incorporate the objective, steps, and success criteria into your prompt.
Place this information at the beginning of your prompt to ensure the agent understands its purpose from the start.
Step 5: Test the prompt with various scenarios to ensure the objective is clear and achievable.
Run simulations or test calls to verify that the agent consistently works towards the defined goal.
2. Insufficient Context
Concept
AI phone agents need comprehensive background information to engage in meaningful conversations. Without sufficient context about the company, products, or services they're representing, agents may provide vague or incorrect information, damaging credibility and reducing effectiveness.
Bad Example:
You work for TechCorp. We sell software. Answer customer questions.
This prompt provides minimal context, leaving the agent ill-equipped to handle specific inquiries or represent the company effectively.
Good Example (Knowledge base node)
Company Context: You represent TechCorp, a leading software company founded in 2010. We specialize in data analysis solutions for mid-size to large enterprises across various industries.
Product Information: Our flagship product is DataSense, an AI-powered data analysis platform. Key features include:
1. Real-time data processing
2. Customizable dashboards
3. Predictive analytics
4. Integration with major CRM and ERP systems
Target Audience: Our ideal customers are companies with 100+ employees, handling large volumes of customer data daily.
Unique Selling Proposition: DataSense stands out due to its user-friendly interface and industry-specific AI models, which provide more accurate insights compared to generic solutions.
Recent Achievements:
- Named "Top Data Analysis Tool" by TechReview Magazine in 2023
- Achieved 99.99% uptime over the past year
- Helped clients achieve an average 30% increase in customer retention
Current Promotion:We're offering a 3-month free trial for new customers who signup for a product demonstration this month.
This knowledge base equips the agent with rich, relevant information to draw upon during conversations.
How to Fix:
Include more context inside your node’s prompt
This is the easiest way to get basic information to your Agent to use during any given node. Include it prior to your instructions.
Use a knowledge base node
Knowledge base nodes allow your agent to access much larger stores of text based information during their conversation. If your Agent has extensive background information needs beyond what can fit in a few sentence then knowledge base nodes are the recommended approach.
%22%20stroke-linecap%3D%22round%22%20stroke-linejoin%3D%22round%22%20stroke-width%3D%2216%22%2F%3E%3Cpolyline%20points%3D%2256%20112%20128%2040%20200%20112%22%20fill%3D%22none%22%20stroke%3D%22rgb(255%2C%20255%2C%20255)%22%20stroke-linecap%3D%22round%22%20stroke-linejoin%3D%22round%22%20stroke-width%3D%2216%22%2F%3E%3C%2Fsvg%3E%0A)