Where creativity meets clarity, and vision meets action.
Driven by curiosity and built on purpose, this is where bold thinking meets thoughtful execution. Let’s create something meaningful together.
Background & Business Context
With the rapid rise of AI, the company's upper management prioritized adopting cutting-edge technology to stay ahead of competition. They were moving quickly to implement transcript and summarization features for call center representatives without first understanding employees’ workflows.
To support this initiative, they requested the UX team for help with visual design, layout, and usability for a transcript and summarization tool. After discussing their objectives, I identified their primary goal: leverage new technology to improve employee efficiency.
How I Reframed the Problem
Understanding these business goals, I advocated for leveraging insights from prior ethnographic research to identify AI solutions that could meaningfully improve efficiency. While stakeholders initially requested a visual redesign and usability study, I recommended first evaluating whether these features were useful and effective in achieving their efficiency goals, while also exploring additional opportunities to address the underlying need.
My Role in This Initiative
I led research strategy for this initiative, while my design partners developed the design direction and Wizard of Oz prototypes, and an AI team built working prototypes. As one of three researchers supporting the product, I led the initiatives outlined in this case study, while my research teammates focused on usability and other features. We collaborated closely to ensure research insights and designs decisions were implemented cohesively across the product.
Diverging
I grounded our team in the current experience by sharing key videos of actual call center representatives intaking calls with customers, and highlighted pain points and opportunities. Based on these insights, the designers developed two key new features:
Data Extraction: Automatically filling out the call center reps’ forms by surfacing relevant information from the conversation’s transcript.
In-call Guidance: Real time tips and suggestions for employees based on the conversation, including scripts, language recommendations, and guidelines for next-steps, such as when to transfer a call.
Assessing Functional Value with Wizard of Oz
I conducted our company’s first Wizard of Oz research study and discovered that data extraction was highly valued by participants as it would directly alleviate the busywork in their call intake process. Conversely, participants would not use a call summary. They found it redundant with information they already tracked during the call, and not timely enough to be actionable in the moment.
They would use the transcript only if the data extraction was incorrect or inconsistent. In-call guidance was considered less useful than data extraction, though still helpful for key parameters that management wanted employees to follow precisely.
These research findings directly challenged the project’s initial direction. Instead of investing effort in a summary feature, we focused our AI resources on implementing data extraction.
Evaluating Implementation with Live AI Prototype
After validating the concept of the data extraction feature, our AI team built an early prototype, and I evaluated its impact through cognitive walkthroughs.
I found several participants consciously abandoning the data data extraction feature because it was hallucinating incorrect information and failing to extract data for key fields. I also discovered that they delay between when a customer spoke and when the data was captured was so long that participants were unable to verify whether the extracted data was correct without writing it down themselves manually.
This study provided insight into key priorities and performance criteria needed before release. The team is currently working on optimizing queries and schema, focusing on improving accuracy, developing consistency, and reducing lag time.