This guide explains how to use the Assessor's web chat testing capability to automatically evaluate text-based chat agents. The AssessorChatFlow conducts end-to-end conversations with your AI Employee over web chat, verifying responses, conversation flows, and business logic.
How web chat testing works
The AssessorChatFlow simulates a customer interacting with your AI Employee through web chat. The flow:
Initiates a chat session with your AI Employee using the
assessor_chatconnector.Role-plays as a customer based on the generated test scenario (e.g., a customer named John asking about appointment availability).
Sends messages and evaluates responses turn by turn, reasoning about each response before replying.
Scores the conversation based on whether the agent completed the required steps and reached the expected CTA.
Prerequisites
Before running web chat assessments, confirm:
Your AI Employee is configured with an Intent Type Map (ITM) that defines the intents to test.
The agent has a web chat channel enabled.
The Assessor module is active for your account.
Run a web chat assessment
Navigate to your AI Employee's assessment configuration.
Verify that the agent's Intent Type Map includes the intents you want to test.
Trigger an assessment run with the chat channel selected.
The AssessorChatFlow initiates a web chat session for each test scenario.
Monitor the assessment progress as conversations are executed.
Review the results in the assessment report.
Assessment flow details
The web chat assessment follows this sequence for each test scenario:
StartChat: Opens a new chat session with the AI Employee.
ReceiveConversation: Listens for the agent's initial greeting or response.
Reason turn: The Assessor's LLM analyzes the agent's message and determines the appropriate customer response based on the test scenario.
Reply: Sends the customer response back to the AI Employee.
Repeat: Steps 2–4 continue until the conversation reaches a natural conclusion or the session times out.
Evaluate: The Assessor scores the conversation based on step completion and CTA achievement.
Differences from voice testing
Aspect | Voice testing | Web chat testing |
Channel | Phone call via pooled numbers | Text-based chat session |
Resource management | Requires phone number allocation from Assessor Server | No phone numbers needed — uses chat connectors directly |
Conversation format | Spoken dialogue (speech-to-text) | Written messages |
Concurrency | Limited by available phone numbers in the pool | Can run many chat sessions in parallel |
🗒️ NOTE
Web chat assessments typically run faster than voice assessments because they do not require phone number allocation or speech-to-text processing.
Review results
Assessment results for web chat tests include:
Intent scores: 0–100 score for each tested intent.
Conversation transcripts: Full text of the chat conversation between the Assessor (as customer) and your AI Employee.
Step completion: Which required steps the agent completed or missed.
CTA achievement: Whether the agent reached the expected Call-To-Action.
QA analysis: For failed intents, a detailed explanation of what went wrong.
Results are saved in the agent's Knowledge Base with the label report alongside voice assessment results.