conversational ai

Creating an enhanced experience for vacation hosts and their guests

client



role



tools



methods

Ansa AI- a conversational AI startup in Sydney

UX Designer Researcher

Trello, Sketch, Illustrator, Google Docs

A/B testing, 1:1 interwiews, user testing, LRT (likelihood ratio test), feature matrix

summary & problem 

Ansa AI had created a conversational AI product designed for Google Home. The startup needed to test v1 of their product and was exploring how it would work as a chatbot on Facebook Messenger. The idea was that the product would minimise communication between vacation rental guests and hosts by answering basic queries about a rented property and it's surrounding area therefore enhancing the guest experience and saving time for hosts who manage high volumes of vacation rental properties through AirBnB, Stays, and other similar providers.



My role in this project was to work with the UX Lead to conduct research in order to understand the value that v1 could provide to guests and therefore hosts.

seamless instructions

I tested multiple versions of the instructions to access the Ansa chatbot on Messenger prior to conducting formal user testing of v1 in order to understand if the setup process could be made easier for guest users. The original version of the instructions already existed when I came onboard. The idea was for guests to scan a Facebook Messenger QR code then go through a logon process. I tried multiple tactics to guide users through the setup process, however, users did not universally identify the QR code as such so the instructions were modified accordingly.

moderation guides

I wrote the user interview and testing moderation guide in order for the UX Lead to revise and modify. We knew the first half of the interview was to understand the guest and their needs when in a vacation rental. The second half was to be a scenario where the guest would engage with the voice assistant.

participant recruitment

I recruited five tech savvy participants between the ages of 25 and 55 who had been the managing guest of a vacation rental stay in the past three months in order to conduct user research.

user research

I assisted the UX Lead in conducting 45 minute user interview and testing sessions in order to understand the value of the product’s v1. Interview audio was recorded and I took notes during the interview sessions. At the end of each session participants were given an LRT scale.

data extraction, analysis and workshop

I worked in conjunction with the UX Lead to extract and compile data in order to group and analyse it and come up with recommendations. Once we had extracted data interview audio recordings and notes, we held a brainstorming workshop in which we grouped results into categories. Then we used a strategic framework feature matrix in order to understand how to move forward with v2.

research insights

• Guests would like the voice assistant to provide them with on-request tips about activities in the vacation rental’s local area. They would like these tips to be left by other guests and the property host and would see particular value in these being special suggestions that they would not be able to easily find by other means such as a google search. 



• Guests did not want to feel that the voice assistant was replacing the host, with some finding it impersonal.



• The biggest perceived value by guests for the voice assistant was for the service to be personalised with regards to the property. Guests wanted the assistant to offer them property feature tips and troubleshooting information specific to the property they were staying at.



• Guests saw value in receiving push notifications with regards to the checkout process. 



• Guests would have liked more personalisation in the communications received from Ansa (i.e: onboarding email and voice assistant interactions). They reported that they would have liked Ansa to include profiling details in its interaction with them (i.e: reason for trip, whether guest had been to that location before, etc).



• Guests did not intuitively know how to use Google Home and this made it more difficult for them to get the hang of using the Ansa voice assistant.



• Guests wanted the escalation process to a human when the assistant could not help to be instant (i.e: for the system to get a human on the phone straight away) and did not find it useful to wait for the property host to write back.



• Guests would’ve liked a chat feature to loop other members of their party into useful property info such as directions, key location, checkout instructions, etc.
• Guests would have found it useful to be able to extend checkout times easily through the app.



• There was no perceived value for the voice assistant prior to guests checking in to a rental property in spite of its pre-checkin features. In spit of this, guests would have liked for the assistant to provide them with in depth directions to the rental property, such as landmarks and photos, in the event that it was difficult to find.



• Of crucial note was the fact the voice assistant was unable to identify what a user with a mild to moderate Italian accent was saying most of the time in spite of multiple attempts for clear pronunciation. This posed a major issue as the market for the product is vacation rentals, a segment rife with international users.

I learned about the ins and outs of voice assistant / chatbot technology and its current abilities and limitations. I also learned the importance of testing with participants that are non-rejectors of the technology in question as this pre-fixed bias negates any usefulness for the product in the eyes of the participant therefore making it difficult to collect objective data.

We were especially surprised to find that guests did not think the chatbot was of value before arriving at the property. Most of the insights from our research brought to light issues with feasibility and viability with regards to the product. On the one hand, participants found little value in the features that the technology could provide at this early stage and, on the other, they found little value in the product unless it was deeply personalised. The latter being a logistical nightmare for property managers that look after large numbers of properties, therefore greatly diminishing value for the customer. These insights had a major influence on Ansa’s post-testing pivot in the direction of their product.

The biggest challenge for this project was testing a product that sits on a technology that remains in its early stages, such as was the case with voice assistants. Users had expectations beyond what the technology could currently do and our research showed that users did not find value in engaging with the technology with its current capabilities.

During this project, I particularly enjoyed working with conversational technology alongside an experienced UX Lead and conducting in depth user testing.

other work