Split Testing and How it Can Evolve Your Product to Perfection

The market for gadgets, products and technology can be thought of as highly analogous to nature; and the way that commercial products thrive and develop is very similar to the way that organisms thrive and evolved in the wild. Just like living things, products will live and die depending on their features and on how well they can adapt to changes in the environment. They compete for resources and they adapt to change (such as technological innovation or fashion) through iterative improvements.

This is fine for Samsung and Microsoft – companies that can afford to have their projects bomb and continue relatively unharmed – but for the small entrepreneur it makes life difficult.

In other words smart phones are the size and shape they are because that shape has been proven to be successful by previous devices, while others that have strayed too far from this recognized format (N-Gage anyone?) have suffered. Those iterations that prove to be successful stick around just like successful mutations in the wild, and those that aren’t as useful quickly die out.

The good news is that this makes for amazing products – it means that every new product builds on the last and improves as a result, while the competition ensures that those iterations come at a steady rate. The bad news though for developers is that this can be a costly and slow process – meaning that most successful breakthroughs come after only a long list of casualties have martyred themselves to pave the way.

This is fine for Samsung and Microsoft – companies that can afford to have their projects bomb and continue relatively unharmed – but for the small entrepreneur it makes life difficult. You can’t afford to release products knowing that there’s a good chance the market will reject them, but if you don’t try then you won’t cover new ground and you’ll never be a market leader. So what can you do?

Introducing: Split Testing

01 (46)

Well the good news is that you don’t have to invest millions into marketing and developing a product in order to find out if you’re making a move in the right direction, not anymore, thanks to something called ‘split testing’ or ‘A/B testing’.

The idea of split testing comes from the world of web design and takes advantage of the fluid nature of that business. Say you have a website that’s getting two hundred AdSense clicks a day and you want to increase that number, what do you do? Well once upon a time you would try adding an extra AdSense unit to your page and then pray it worked – of course recognizing the potential for it to go wrong and drive away visitors who might never come back.

But since then the way we do business has evolved and now a webmaster might do something smarter – using a PHP script to release changes to only a small portion of their user-base such as people from a certain location, and then to compare the increases or decreases in bounce rate, click through rate and other signals. Then, if the changes prove to be successful over a significant period of time, the webmaster could roll those changes out across the entire site and see them magnified.

How Does This Apply to You?

Villa del Cigliano

In the product side of manufacturing products, it’s still difficult to do split testing. Some materials can lend themselves more to minor changes, such as textiles because, generally, the minimum order quantity will be smaller. Also, if you can make most of your product out of 3D printing and the price point is high enough to cover the printing costs, this may be viable. But, overall, split testing is more useful in our line of work on the marketing side.

When mocking up packaging, you can make up two or more designs and show them to a focus group to get feedback on which one is most attractive to your market segment. Or, you can vary your pricing for a similar market segment in different distribution channels to measure the elasticity of the market for your particular product. On an even less scientific basis, you can vary your elevator pitch with different people you meet to find body language and expressions that drive your idea home in the best way.

Split testing has a range of different applications and is fantastic for testing any number of different iterations and changes. Think about ways you can apply this strategy to your current business model and you’ll come up with some good mechanism to test changes while controlling the overall downside risk. Be creative!

Split Testing and How it Can Evolve Your Product to Perfection tweet this

Leave a Reply