Social Voice Agents in Homes

Brief

How can we deepen our understanding in which social agents can exist in a family home?

This research aims to uncover the boundaries between human and robots, particularly a social voice agent in a multi-person home. We were not trying to find what the agent can do, but rather understand human attitude towards these agents and their role in the future.

We posed questions like: Can your child ask Alexa for help with their homework? Can your sister or brother ask Google Home to lie for you?

Our creation of our agents are more future-forward, allowing us to pose questions on security, privacy, and surveillance. Ultimately, our findings will be published and used for future researchers and developers of voice agents.

Step 1

Generating Agent Themes and Scenarios

We did a quick brainstorming session to come up with several scenarios in which voice agents could exisit. We individually came up with scenarios of families interacting with a social agent around the themes that we generated.

Defining themes to guide our exploration

After the exercise, we decided to focus on the following 4 themes to guide our research:

Step 2

Producing storyboards from our scenarios

We broke into smaller groups to hone in on our scenarios and pick the narratives that were provocative. With my group, we gathered scenarios of the categories of Permission, Social DDD, Deception. We drew up our storyboards and as a whole team, we generated 22 well-defined storyboards.

Step 3

Speed Dating Outcomes

Recruiting was extremely difficult since our target research participants were 2-parent households with children 12 years or older. We wanted this demographic specifically in order to better understand how an agent can function in a family home. We were lucky enough to gather 16 interviews with families in the Pittsburgh area.

Each family we interviewed took around 2 hours and we interviewed about 20 families. We probed and asked questions to get to the root of their thoughts and feelings on the storyboards.

After a hefty period of transcribing each interview, we selected key quotes that would help us with a thorough analysis. As we read each quote, we continued to categorize popular opinions and strong feelings in an affinity diagramming exercise.

Step 4

Final Analysis of Storyboards

Families had very similar opinions and reactions to certain storyboards, both good and bad. Overall, we learned ways in which an agent could exists among a family. We concluded several insights about voice agents, but here are the top three I believe to be most significant in our findings.

Seamless household management and interactions

Parents particularly enjoyed the idea for the Agent to manage the household more seamlessly, allowing the agent to be able to transfer messages, to delegate tasks, or to figure our scheduling for family activities. “To have the agent be able to sort of track activities and calendars and due dates and things like that for all the variety of things that are going on and where people are supposed to be and stuff. I mean that’s actually very helpful.”

Bridging difficult interactions

In some situations, participants appreciated the agent’s involvement in difficult interactions and its ability to bridge between people. For example, asking for help, resolving sibling conflict, support an argument with evidence or for softening some unpleasant interactions. On the other hand, if the agent took sides with bias it would feel unnatural. People struggled understanding who it would be accountable to in a conflict situation, and generally struggled with the idea that an agent might have more than one social roles.

Agent making judgement

Information over recommendation: People specifically did not like when the agent was there to make social judgements: critique their lifestyle choices or giving any parenting advice. People also were more comfortable getting information from the agent, rather than the agent providing them with recommendations.

Conclusion

Final Thoughts

I’m so grateful for the opportunity to work with such an amazing team on such a thoughtful and intriguing project. I really enjoyed learning about voice agents and how people react to situation they might be in, in the future. I’ve always had a curiosity in how humans and robots interact, since I am an avid user of Alexa. Through learning new research methods and interviewing folks outside of my school bubble, I feel as though this project helped me grow a more holistic point of view, in human- computer interaction.

If I were to continue this study, I would actually conduct another round of interviews but with live prototyping. Perhaps someone could act and speak what a voice agent would say, and this might bring a whole new perspective and insights in an environment that simulates it.