On the 21th of September the best work in CRO and A/B testing will be awarded with the first and only CRO award of our country: the DDMA Dutch CRO Awards! Of the 70 submitted cases, 16 have been nominated! Philips is nominated with two cases for an award in the category ‘ Mobile’.
To learn more about CRO within Philips and how the case came about we spoke to Soraia Cardoso (CRO specialist).
What is your definition of CRO?
Conversion Rate Optimization is art and science coming together to optimizing digital platforms and increase the likelihood that visitors will complete a desired action. The CRO process involves understanding how users move through a digital platform, what actions they are likely to take, and what’s stopping them from completing their goals.
It is a process of finding answers that no one knows for sure. It is the constant search for a better option, a better idea, and a positive outcome. It’s about re-evaluating false knowledge, gaining new insights and celebrating discovery. It is about coming to a greater understanding of our users through constant experimentation.
What is the biggest trend or development you see in CRO?
The advance in technology and research techniques. The power of data science and artificial intelligence enabling us to focus on personalization based on customer behavior and preferences. The ability to power customer choices and enhance their experiences by anticipating their needs, desires, and emotions; showing relevant content and/or functionality at the right time, place and in the appropriate format.
When looking at CRO here in the Netherlands, what do you notice? What do you think is going well and what needs improvement?
The benefit of having a capability driven by the desire to better understand customer behavior and act upon that by exploring the design of experiences that are essentially right for them is something more businesses are starting to recognize as a game changer.
Many businesses in the Netherlands are adopting a culture of experimentation, providing space for trial and error as a way of learning while creating great value from it. Focus on user experience is being taken seriously and with CRO there is no longer the need to feel fear from being wrong, there is the belief that we can and should always strive to do better for our customers.
Overall, the field is getting more mature and an increasing number of CRO specialists are using structured processes, tools, methods and a prioritization framework. Related to that, optimization programs are also getting increased budgets and respect from stakeholders.
On a less positive note, there are still a lot of organization that think CRO is only about a/b testing and increasing revenue.
Where did it all start with this particular case ? What was the reason for creating this hypotheses?
Case: Retailers Logos
Testing, within our organization, is a process of finding answers that no one knows for sure. No matter the particulars of a given test, in this culture of constant optimization openness and stress relief where none of us have the answers, but all of us have great ideas, we celebrate being “wrong”.
We all agree that it is great to find something that works even better than what we have. It is never about a single test or idea; it is about the constant search for a better outcome, a better experience for our customers. It is about that marginal gain of finding the best option from the continuous research/ testing approach, and not from any single outcome.
Furthermore, questioning if and where we can do better is part of our global continuous optimization strategy at Philips. Customer Obsession is the focus of our global optimization efforts: we are continuously working on building meaningful and relevant online experiences for our customers.
With this in mind, and with the belief that visual cues and affordances play a critical role in the success of an online experience, we went on a journey of iterative design and testing. Our main question was simple: “Does the visualization of the logos on the Product Detail Page have an impact on the overall user experience and ultimately alter the visitor’s perception of the usefulness of the website based on analysis of KPI’s such as sales?”
What were the biggest obstacles that you encountered with your case?
The human factor: dealing with a great number of stakeholders with different interests and approaches and the need to get everyone to forget their personal vision. We solved this by bringing everyone to understand that our focus was not on the individual needs but, on the common goals which would benefit everyone: building meaningful and positive experiences for our customers benefits everyone in every way.
The technology: We had to face the challenge of real-time data updates related to the product stock availability on the Philips Flagship Store and in each of the Partner Retailers stores (visually represented by their logos), meaning if the product was not in stock on a certain retailer website, the logo should not appear. We solved this by developing an API call that allowed us to dynamically update the retailers’ logos, matching on each page load, with the represented retailers and the product availability online. All in all, a mi-nuncius task, which ultimately helped us achieving excellent results.
What advice would you give someone to ensure that his or her case gets nominated next year?
Embrace a culture of experimentation. No matter the particulars of a given test or business goal, keep in mind that any kind of change can be a positive one. Disregard right and wrong answers and celebrate discovery – it is the collective of all changes that will make a difference in the end.
Show that you are never satisfied with one answer, one test, one approach. Continue exploring ideas and possibilities. This will provide you endless insights, learnings, discoveries, and innovation. It is hard to fail using an approach rooting these core values.
What was the value of your case for the organization (what did it deliver)?
Customer obsession is a fundamental change to our business model. It is not just about numbers. It is about embedding consumer feedback into the digital development cycle at every stage and ensuring it’s a central part of our global continuous optimization strategy at Philips. We embrace a culture of experimentation where we continuously learn, discover and innovate by reinventing ideas and finding new ways.
With this experiment, we not only got very positive results with the test variant (at 99% significance): an uplift of 10,83% on orders, but we are also a step further in our drive to gain full understanding of our consumer’s journey’s to be able to offer relevant experiences at each (micro)moment.
Since we operate on a global scale, when a test is successful we pass the results and learnings to other teams worldwide (16 markets, +/- 200 countries) so that changes are implemented and the benefits captured globally.
Why should your case win?
Our case shows that a structured and well-defined approach of research and data analysis followed by user testing can be beneficial in creating A/B-tests that in can deliver the results we are aiming for. With our broad range of methods and tools applied to CRO, we strive to gain new insights and keep on learning so that we can delight our customers every day, in every way.
In the end, winning this award would clearly show/prove to the CRO community that putting the consumer at the heart of every CRO process is the best and only way forward.
What is the most important thing that you learned which you will apply again in the following cases?
Overall we’ve come to the realization and understanding that, positive outcomes are, seldom, based on a single idea, but are the result of a constant search for something better. It is about finding the best option from the various iterations. With constant experimentation, we can transform decision making into a scientific, evidence-driven process—rather than an intuitive reaction.