Improving Communication on a Chat Service
By using several common user experience discovery methods, we uncover recommendations for service improvements.
Project Background
POWER Library is the online portal to all that Pennsylvania libraries offer. Research everything from healthcare to education and much more. Chat with a Librarian is here to help if you're unsure where to start.
Hosting Solutions and Library Consulting (HSLC) wanted to gain a fresh opinion on the user experience of their web-based chat service, Chat with a Librarian (CWAL), funded by the Pennsylvania Department of Education. HSLC was a valuable partner throughout the experience audit by providing existing research and background information. Their familiarity with the background processes of the system made them a helpful sounding board to validate or cast doubt on our assumptions.
Project Role: UX Strategy and Design, Presentation Design, Project Lead
Deliverables: Service Comparison, Personas, Service Blueprint, Messaging Recommendations, Reporting
Creative Director: Adam Englehart
Team: Laura Olverd
Client: Hosting Solutions and Library Consulting
Funded by: Pennsylvania Department of Education
Vertical: Education, Non-profit, Government
Created at Neff
Screenshot of the existing POWER Library Chat with a Librarian web page before our project. This page serves as a primary entry point to the CWAL service.
Framing the Problem
Acquiring New Users: Though the service is taught in schools, it was not reaching its user acquisition goals. This was attributed to potential users that were unaware of the benefits and capabilities that the service provides.
Operator Challenges: Many users were not aware that the service was staffed with live operators. This led to misconduct, ineffective communication, frustrating operators, and increased queue times.
Understanding the Constraints
Though interface design was outside the scope of the project, there was an opportunity for our recommendations to influence the messaging and user experience. There would be a limited ability to customize the chat interface itself because a third party handled it.
Aspects of the chat experience we could improve include:
The landing page information and messaging
Automated “canned message” suggestions for operators to choose from during their chat
Training protocols for operators
Designing a Solution
After identifying the gaps in communication throughout the experience, new messaging was recommended to set accurate user expectations.
Educational guidelines were also created for chat operators to encourage consistent interactions.
Objectives of the Review
Examine the experience for opportunities to increase the volume of users
Look for points of friction for users or operators to improve efficiency
Identify opportunities to dispel misconceptions users frequently have to improve the experience
Defining Target Personas
Learning About Our Chat Operators
To improve the user experience, it was imperative to consider the chat operator’s experience as well. Operators are librarians working at public libraries or universities within Pennsylvania who answer chat queries in real-time. The operators have varying levels of expertise with the chat service, and technology in general.
When local operators are unavailable, questions are outsourced to a broader network to provide consistent service.
Prioritizing User Groups
Though the chat service is for the public, and the types of users can vary widely, students and their caregivers were prioritized. The education level of student users may vary from elementary through high school.
Our client shared that this audience group makes up a majority share of users because this chat service is taught within the public education curriculum. During our review of device analytics, the outsized share of Chromebooks (commonly used in schools) accessing the service correlated with this assumption.
Public school librarians were also a targeted user group because they accessed the service while teaching students.
Personas at a Glance
Detailed Personas
Service Comparison
Method
Since CWAL is designed to serve Pennsylvania residents, we examined some similar services that operate in other states to understand better the space that CWAL performs in. We tested each service several times based on the personas developed for CWAL and recorded our observations.
We rated the services based on five uniform criteria based on our experiences of successful interactions. The rating system asserted generalizations based on our observation notes. We used the rating system as a construct to draw more direct comparisons between the qualities of different services.
Ratings Criteria
Introduction: Provide a clear introduction, including the operator's name, profession, and location. Let the user know that operators are real librarians here to help.
Setting Expectations: Let users know what types of questions can or cannot be answered, as needed. Inform users that chat operators offer resources but cannot always provide direct answers.
Preliminary Questions: Did the operator ask good clarifying questions to accurately understand the user's needs?
Language / Tone: Did the operator use friendly, professional, and age-appropriate language?
Helpfulness: Did the resource and interaction meet the user's needs?
The Journey Mapping Process
Overview
Journey maps were constructed for each persona. This exercise was intended to reveal obvious stumbling blocks or miscommunications that make take place during the chat experience.
Goals
Build empathy with the persona groups
Identify instances of friction in the user experience
Lay a foundation for targeted messaging and content
Method
User journey maps were created by inventing fictitious scenarios for our user personas that would precipitate an interaction between users and operators on the chat service. These scenarios were validated as plausible and authentic by HSLC’s existing dataset. Our team then tested these scenarios by role-playing each persona within their defined scenario and recording our interaction and observations. This data was analyzed and used in our journey maps.
Unfortunately, we couldn't access the operator software to role-play their experience. Operator journey maps were created based on composites of previous interactions we had on the service and research provided by HSLC.
Journey Maps
Building a Service Blueprint
This service blueprint documents the process of a typical user through all phases, from awareness of the chat service to the conclusion of the interaction with their operator. At first, we believed the service blueprint should only map the chat interaction itself. However, we found it relevant to pursue the full scope of this user flow to understand what knowledge and expectations a user may accumulate before the chat interaction.
The beginning stages of the diagram reference levels in a typical marketing funnel from “Awareness” to “Decision.” This is related to increasing the volume of users on the service.
In the service blueprint, the sequence of steps during the experience is displayed above, and the explanation of the actions is detailed below.
The user completes nine steps in the service blueprint, from awareness of CWAL to completing the chat interaction and providing feedback on their experience. At each stage, they interface with a form of technology or media. The operator shares data with the user during steps 7–9 of the interaction.
Delivering Recommendations
The Final Report
Our process, artifacts, and recommendations were delivered in a 50-page final report to HSLC. It also included pages on fundamental UX principles for operator training sessions and critiques of marketing materials. Individual suggestions for improvement were prioritized based on the estimated effort it would take to accomplish and their potential level of impact on the experience.
Suggestions for Improvement: Low-Hanging Fruit
Revise the CWAL landing page by setting the expectations of prospective users with service capabilities, constraints, and instruction
Chat operators should consistently deliver warm and informative introductions to humanize the interaction.
Messaging on the question entry form can reinforce guidelines and confirm users’ expectations.
Taking Recommended Action
Though the implementation of our recommendations was handled internally by HSLC and its partners, we can readily see that some changes were adopted. The largest one is separating target user groups before entering the chat experience.
Initially, the CWAL landing page did not have much user contextual information. We identified this as an issue because it assumes existing knowledge and does not adequately set users’ expectations for their interaction.
Even chat services can improve their communication!
Recommended messaging was added to provide adequate context for users before they engage in a chat:
When should users choose to interact via email, chat, or text?
Who is answering the chat?
What is the expected duration of the interaction?
What results can users expect?
What are the limitations of the chat service?
By separating the audiences, targeted messaging for K–12 students was able to be added to remind users that their operators are real people and to encourage behavior within the acceptable use guidelines. This addresses a key pain-point we heard from operators.