Implementing change management practices to de-risk new feature releases for internal users

co-author: Mathias Scholtysek

Introduction

Customers of HelloFresh are faced with a steady flow of improvements in our product offering, as well as the digital interface they work with. Even though it is great for our customers, it can get complicated for our customer care team which needs to solve requests and issues based on our evolving product. Having an efficient, fast, and user-centric solution to help our customers is as important as the evolving product offering itself, creating a seamless and customer-centric service recovery ecosystem. The Agent Experience Squad is focusing directly on our agents, our Freshies, which is the lovely name we give our internal users at HelloFresh.
Freshies are not only faced with a variety of problems they need to solve for our customers, but also changes that we conduct internally towards their processes and tools. Given these two factors, we needed to find an easy and user-centric solution on how to handle the risks and pain points that arise through continuous releases for them.

Problem statement

As the Agent Experience Squad, we are responsible for creating and improving customer care agents tools. Our users are quite diverse, they sit in 9 countries, across 4 continents and service all 18 markets and 6 brands the HelloFresh group currently operates. We regularly introduce new features and new tools according to customer feedback and agents requirements which lead to some problematic situations such as:

• Rolling out huge features as one bulk for the agents can lead to disturbances in the agent’s workflow especially if the time of training doesn’t align well with the full feature release or agents are not aware of the features which are being rolled out.
• Agents can be quite change-averse, especially experienced agents that are used to working in the old environment and create their own workarounds and processes based on it.
• Rollbacks can be costly in time and effort, decrease the user experience, and create misalignment with the training schedule and content, in addition to being a step back for the squad work and having a downtrend in morale from developers to stakeholders.

Resolution

By introducing a beta environment for third-party applications that communicate with the HelloFresh services we are able to test new features on a group of our agents, called change champions. That way, we can de-risk the release by identifying important edge cases for singular markets, gain information on bugs, and gather feedback directly from agents which enables us to make quick changes as well as build our roadmap out further. Our goal is to know where to improve and where to make changes to make the agents’ user experience as smooth as possible.

What is Beta Testing?

Beta Testing is a method of introducing change in a controlled environment to a small group of users to gather information about adaptability and introduce gradual changes while gathering feedback in order to ensure the best possible experience for both agents and customers.

What are change champions?

Change champions are what we call a group of agents that are excited to be the spearhead of testing new features and give feedback through a variety of channels and methods, such as focus groups, surveys, and hotjar feedback. We also created a chat channel through which the agents could directly reach out to the tech team (Developers — UX — Product Owners).

The change champion concept was taken from common change management practices, which usually focus on big organizational changes. We found it quite fitting for our product development and decided to adapt it based on our smaller scale.

The Kübler Ross change curve

How we defined the reasoning of our change champions framework in detail was:

• Gather UX/UI Feedback
• Learn about bugs
• Identify important edge cases and discuss them with users and local markets
• Co-formulate the vision of owl with our users

These points were used to govern our approach and helped us to identify the technical implementation, as well as how we want to collaborate with our users.

How would Beta testing resolve our problem?

• Improved decision-making: Beta Testing provides data-driven insights that help businesses make informed decisions about their products’ design, functionality, and user experience. This leads to better outcomes and more customer satisfaction.
• Optimized user experience: By testing different versions of a product or feature, businesses can understand what users prefer and optimize their experience, leading to higher engagement and retention.
• Better product development: It can help businesses to understand what features are valuable to users and prioritize their development, leading to more effective and efficient product development processes.
• Reduced risk: Beta Testing reduces the risk of introducing disruptive changes to a product or feature. By testing changes on a small group of users, businesses can assess the impact before rolling out the changes to their entire user base.
• Faster iteration: By quickly and easily testing different variations of a product or feature, businesses can iterate faster and make improvements more quickly, leading to a more agile development process.

Challenges and growth opportunities

It wouldn’t be an epic story if we didn’t have challenges and come up with solutions for them 😀

The challenge for the squad was, how do we create a sustainable environment to test future changes and tools, across all markets and brands we have, without disrupting the workflow of agents as well as ease the transition towards new tooling.

To break things down a little further, the challenge for the engineers is in the shape of a call center application that the customer care agents use to receive incoming customer chats or calls we needed to test rolling out a new and revamped customer care tool which is referred to as owl against the old tool which is referred to as bob. Both are web applications for displaying customer information and reporting errors on the customer’s behalf. The question raised is how to have a group of agents land from the call center application to the owl instead of bob as a landing page for our users. This is regulated by a setting change on the call center application which defines a link for each customer so you can get redirected to your assigned tool.

The challenge for UX and product is to align with our markets on how the process will be used in terms of training, identifying what a representative sample of users looks like, creating an environment for the users to give us feedback seamlessly and easily, and how to break the findings into short- and long term improvements.

All of these changes should fit seamlessly into the agent’s workflow and mustn’t add any additional workload to them. In addition, changes and findings should be communicated openly with users and local markets alike to keep up transparency and alignment. Lastly, we expected the changes to have a positive impact on two of our most important metrics, the average handle time (AHT) and our usability metric UMUX which we tracked throughout our program. The UMUX metric consists of 3 dimensions, namely:

• Effectiveness: Allow Freshies to do what they do without mistakes.
• Efficiency: Allow Freshies to do what they want to do — a service that meets their needs.
• Satisfaction: Allow Freshies to do what they want to do while enjoying it.

Defining the solution

After a brainstorming session, we tracked down two requirements for the solution that will be introduced for the beta testing environment:

• The flexibility to change the feature rollout for a vast amount of users or a specific group
• Fast implementation and flexibility to change users due to the changes in agent groups
• Removing the dependency on the call center application mentioned above

The solution

We came to the conclusion that we will need a proxy page to redirect our users effectively.
This page will be responsible for:

• Consuming the agent’s bearer token
• Utilize the Agent’s token to define which market the agent is working on as one agent can serve more than one market and decide if the market is part of beta testing or not
• Define within a market if the agent is part of beta testing
• Decide to direct the agent to bob or owl

How did we do it?

Engineering work was divided between the backend and frontend:

1. Backend

On the backend side, we needed to configure and update the call center application according to the market and the agents who are participating in the Beta testing to do that we did the following:

• Update the existing link from the call center application to reflect the new redirect page (New customer care tools or old customer care tools) we chose Denmark as the market to roll out the new change as it provides a small sample and is not a peak market
• We update based on the market which a mapping between (customer_id, UUID)
• We fetch the customer using the UUID and update the external link using the 3rd party application developer’s API

2. Frontend

On the frontend side, we needed to redirect the agent according to the token and the group that they are in

• Have a list of change champions that need access to new customer care tools
• The new page is through a new entry point (Proxy page) which is responsible for consuming the token
• Check if the agent is part of the change champions list that is part of the feature rollout
• By using the tokens to check the role/country
• We check the ID of the agent belongs to the change champions group

3. UX & Product

From a UX and product perspective, we needed to ensure a smooth start, running and end of the program. We clearly communicated expectations to stakeholders and users alike, the expectations from our users were:

• Honesty: “We can only improve the tooling and processes with honest feedback, please be brutal with us. We will never feel attacked nor will there be consequences for you being honest.”
• Proactive feedback: “Let us know how you feel often and about everything. We want to steer our development into being even more user-centric, which will also depend on you. And don’t forget that there are no hierarchies!”
• Take part in the sessions & use the new features — “We have focus groups and retro workshops scheduled, please attend them. We will coordinate with the local ops teams so you don’t have to worry about anything.“

A big part of the program relied on the feedback we received through the three channels we set up. These channels were:

• Hotjar feedback — immediate feedback that can happen during an interaction
• Google group — was used to give us more elaborate feedback, discuss problems and suggestions with us, as well as helped us to reproduce bugs.
• Focus groups — we set up bi-weekly focus groups throughout the program so that we can have more elaborate face-to-face discussions. These were prepared by UX and focused on open questions from either the hotjar feedback or the google group.

The result

The program was running actively for six weeks in total, during which we planned our meetings and actively searched for feedback. The users kept their access and provided feedback after we officially ended it, which was great to see given that we created an environment in which the users were reaching out even after the program was finished.

The highlights for us were:

• Creating a sustainable community and technical implementation to de-risk large-scale releases
• Improving our usability metric UMUX from 61.86 to 69.17 (higher score = better usability as defined by effectiveness, efficiency, and satisfaction)
• Decreasing our average handle time from 544 seconds to 516 seconds. During the same period, non-beta agents AHT increased from 611 seconds to 626 seconds
• No rollback was needed, and no important edge cases needed to be accounted for post-launch

Contact

And that’s all folks, Please contact HelloFresh’s agent experience squad for more information and questions. We are looking forward to hearing from you:
agent-x-squad@hellofresh.com

Thanks to everyone who worked on this Hagag Mathias Scholtysek Ivan da Rocha Fernandes Amey Patil Waqas Mumtaz Brighton Chabula Arvind Kumar