The power of A and B
Any internet/ecommerce product manager worth his or her salt knows that the best way to learn about your customers is to ask them. However, operating in the equivalent of an earth-sized shopping mall with millions of customers walking in and out every day makes one-to-one conversations impractical. Thus emerged the practice of A/B testing. By showing features or changes to just a proportion of your customers every day and holding them back from the control group and collecting the appropriate amount of behavioural data (more on this later), one can easily and precisely measure the impact of the change. The change can then be as large and complex as a full-page redesign or as simple as changing the colour of a button. Using a vast arsenal of statistical tools one can quickly arrive at clear data led evidence of the value of something. In my own experience it is often the smaller changes that provide the largest upside as they tend to remove minor but annoying and value depleting friction in the customer’s journey.
Is the brouhaha about Facebook’s emotion test justified?
So why is Facebook in a PR storm over the recent discovery of their “emotion” A/B test in 2012 that controlled negative or positive posts in your newsfeed? There has been a lot of opinion about this in the press already and it is always a delicate matter when an Internet behemoth is caught doing something deemed unethical. Is it because customers could not opt in or out of the test and it was done without their knowledge? If it were, that would hardly be a fair test and any results relatively meaningless. There were no privacy concerns, no data theft nor any unfair practices. The only marginal issue was around the influence of a large Internet company on people’s mood and emotions. Facebook’s newsfeed is a cornerstone of their product and understanding what the content of the newsfeed does to people (assuming that was their only intention) should be fair game.
How different is this from a reputed newspaper making editorial decisions about what makes front-page news? Or a wine seller bending the mood of the shopper with music from specific regions to push sales of wines from that region (see this report in Nature). Testing products with customers is not new news. Customers should probably not feel that they have been subjected to something unfair in the process. In fact they should feel good. Every day they contribute directly to the process of product invention and evolution via thousands of tests on the internet. Understanding cognitive biases and using them to improve business performance has captured the imagination of academia for the past couple of decades (Thinking fast and thinking slow by Daniel Kahnemann is an example) and is now being actively put into practice.
With (data) power comes great responsibility
That brings us back to the dreaded D word – Data. I made comments recently about the relative benignity of data collected by Internet companies as long as the data is handled safely and securely. I am not talking about credit cards or personal data here. That is all well secured and regulated and any serious Internet firm needs to ensure they handle this data securely. However, click activity and other customer behaviour are both vital and valuable data which when collected, aggregated and used properly, gives valuable insight to product managers that they can use to improve their product and business proposition. The key word there is aggregated – it is patterns that emerge from “big” data that provide the value being sought. Individual customer data, if used, would be dangerously misleading. As an industry insider, I can safely say that customer and regulator concerns here are exaggerated. {Tweet this!} Even so, products developed using aggregated data also often tend to be polarizing. Customers either love it or hate it and I would urge product managers to be prudent.
In summary, I urge the general public not to get over emotional (no pun intended) about the Facebook test or what they perhaps perceive to be unfair exposure to experimentation or data collection practices. They tend to be harmless, are incredibly valuable to both the businesses and future customer experiences and make the (internet) world a better place one test at a time.
Your reactions are welcome!
Guest post by Manish Gajria, Seasoned e-commerce professional.
I (Minter) had the pleasure to meet Manish Gajria while chairing a panel on eCommerce at the London Business School. I look forward to any comments you might make about Manish’s post and will make sure that he gets them, too!
