The Virtual Agent can do more than just type; now, it can speak!

Our idea here is to enable all the same skills being utilized by chat for the Voice channel. In the future, this will mean speaking with the VA through your Expedia app, your Alexa Dot, or even your car’s entertainment system. For now, it’s a phone call away.

In trying to create the best possible experience for Voice users, we quickly learned that the content should differ from Chat in a few primary ways.

Nuts and bolts of the Voice channel

First, a bit of context on how these skills work under the hood. We map skills the same as Chat, but there are a few tools/concepts we employ to make said skills work for Voice.

Input types: Where Chat has clicks and taps, Voice has DTMF and dictation. DTMF (Dual Tone Multiple-Frequency) means mapping options to numbers on a phone’s keypad, similar to Quick Replies in Chat. Think, “Press 8 to speak to an agent.” Dictation means speaking to the device to choose options, like typing a message in Chat.

Speech-to-text: When a user dictates an option, their speech is converted to text, which then enters our NLU engine the same way a typed message would. Currently, we use Amazon Polly for this speech-to-text conversion.

Speech Synthesis Markup Language (SSML): We’ve all had to listen to dry, robotic voices when calling an automated service. To add intonation, we can use a markup language called SSML. For example, we can add to pause for effect after a statement.

When designing for Voice, we need to consider the component-level differences and tone-of-voice differences from Chat.

Below is the project I have done in the early of 2021 to contribute to Expedia’s Global English Routing goal. The goal is to provide travelers an opt-in and opt-out options to record their conversations during the call.