How business experiments can help your company to improve customer experiences and boost growth

When it comes to improving customer experiences, trying out new business and revenue models, or developing new products, even the most experienced managers get it wrong. After spending months of development time with decisions based on intuition, experience, and opinions, they discover that their new concept doesn’t work. That’s a lot of resource allocation and investments down the drain. What does work, though? We’ve learned that running disciplined business experiments creates significant value. And what if companies roll out new products or introduce new customer experiences without running these experiments? They’re basically flying blind.

Opinions, intuition, and (sometimes even) experience are useless

Today’s managers are stuck with a striking dilemma: despite being flooded with information coming from every direction, they operate in an uncertain world where they lack the right data to inform strategic and tactical decisions. So, as humans always do in situations of uncertainty, our actions tend to rely on experience, intuition, and beliefs. This often doesn’t just cut it, especially when companies want to go for big results with big amounts of resources and/or investment. Additionally, truly innovative ideas go against our experience and assumptions, or conventional wisdom.

A quick example to illustrate the point that intuition and experience don’t always bring you where you need to be, and that experiments can lead to the discovery of both new markets and technical solutions.
Taking you back to 1964, the year 3M discovered the Post-it note. Mr. Spencer Silver, a chemist at 3M whose job it was to create new types of glue, had experimented with a new glue that had a high level of “tack”, but low adhesion. Silver tried to sell his new glue to other departments in 3M, who just didn’t see a future in the glue because they were focusing on creating a stronger glue that formed an unbreakable bond, not a weaker one that only supported a piece of paper. Silver’s concept was hopeless - the adhesive just didn’t solve any known customer problem – until he met chemist and choir director Arthur Fry. Fry observed that members of his choir would often drop bookmarks on the ground when switching between songs, and he wondered that if these bookmarks had a little bit of adhesive, that would be just the next big thing for them. And boom, this “Eureka moment” led to a series of experiments with the new glue that led to a paper product that could be attached and removed without damaging the original surface. It’s was the solution to a problem nobody realized existed, leading to a market where 3M produce about 50 billion Post-its today.

This example shows that the innovative idea of the Post-it went against industry assumptions & experiences (everyone was trying to make stronger glues), and that new markets and new products can be found by running some disciplined experiments.

Don’t fly blind, use experiments to gather evidence and gain clarity

All experimentation – whether in a laboratory or on your business’ online retail channels - works because it harnesses the power of continuous learning, making it possible for your organization to capture small incremental learnings and improvements quickly, which accrue to large substantial growth over time.

Think of experimentation as a good ol’ game of Battleship. You know, the one where you play 1v1 and put ships on a grid, take turns ‘bombing’ the location of the other player’s ships and try to sink their fleet. With each turn, whether a hit or a miss, you generate knowledge. The more shots you fire (or the more experiments you run) the more you learn about the playing field and the more you know about what works, and equally important, what doesn’t work.

Experimentation embraces uncertainty and removes the guesswork by giving you evidence. You could test different campaigns and landing pages with different content and see which one converts to more sales, test a new digital queuing system in your store to increase customer experiences, or experiment with different flows in your app to improve user satisfaction. By running experiments and making decisions based on the hard facts they return, you can more accurately forecast the impact of new experiences, campaigns, features, and functionalities.

As Stefan H. Thomke depicts in his book Experimentation Works, organizations such as Amazon, Booking.com, Visa, and Netflix (the ‘Experimenters Index’) who deploy rapid experimentation perform significantly better than those who don’t.

Of course, there is a lot more in play than just the use of experiments in their business operations. Such as strategy, operational efficiency, employee performance, culture, external industry factors, etc. Still, it’s noteworthy that these organizations have performed exceptionally well commercially over the last 10 years.

The point is that with experimentation, you generate extensive knowledge from both failures and successes, generating a compounding effect of more experiments and more knowledge. In short, you stop flying blind.

Experiments can improve customer experiences and boost growth

At a time when digital & web channels are vital to almost all businesses, especially in a post-COVID world, rigorous online experiments can help businesses to increase sales, customer satisfaction, experiences, and journeys in a fairly easy way. Today, companies who already have a few thousands of users or visitors can assess not only ideas for websites but also potential business models, strategies, products, services, and marketing campaigns—all relatively inexpensively. The ability to access large customer samples, to collect huge amounts of data about user interactions, and to run concurrent experiments gives companies an unprecedented opportunity to evaluate many ideas quickly, with great accuracy, and at a negligible cost per experiment. Running these experiments can transform decision-making into a scientific, evidence-driven process—rather than an intuitive reaction. Without them, many breakthroughs might never happen, and many bad ideas would be implemented, only to fail, wasting resources.

A quick example of how companies can improve their customer experiences or boost sales through robust experiments.
Imagine a national airport that just re-built its parking experience. The airport shuffled the parking site around and changed the drop-off points, creating more space and more efficient parking options. As an airport, you want travellers to have the most accurate information available through your website and email notifications. As an experiment to optimize the customer experience, you can create a website, app and email copy with accompanying flow to inform travellers about the new features. To get insights on clarity, comprehension rates, and potential improvements, the airport could run a UX test. In a UX test, real customers are given an assignment, or mission, to complete and give real-time feedback on their experience. In this case, the customer could be asked to book a premium parking spot for a specific amount of time, returning qualitative data on the experience and quantitative clicking data to confirm or debunk clarity of the flow.

Another team of the airport has launched a new e-commerce platform where travellers can easily order duty-free items from the shops online and grab them at a pick-up point, skipping lines and general airport stress. To create awareness and grow sales on the platform, the team can launch different variations of their awareness campaigns on different traveler touchpoints and generate data on which campaign copy and touchpoints work best. This is what is called an A/B/n test, where different versions are tested against each other in parallel. Additionally, once the traveler has converted on the campaign, they can test various user journeys or product placements throughout the e-commerce platform and get insights on what works best. The airport could allocate a small number of their website visitors and email contacts to the new variants and discover if the conversion or traveler experience has indeed improved, and use this as a prediction for a full rollout. The final result is a conversion-optimized and data-driven user journey, with increased platform sales.

Use experimentation to guide the way

The main message we want to convey with this article is that experimentation is a tool that helps you to navigate uncertainty. It’s a process that helps you test out different ideas and assumptions, and introduces data and evidence into decision-making – steering away from opinions and intuition. The idea is to kiss as many frogs as possible, because this increases your chances of finding the prince.