The Problem With Reading Site Tools
Businesses want to see the product that they have. That is fairly obvious. What may be slightly less obvious, though we probably all know it if we stop and think, is that businesses want to provide us with the tools we need to engage with them. A basic offline example is carts at the grocery store. This is provided to you simply to make shopping with that store easier and so that you can buy more stuff (as opposed to if you had to carry it all in your hands).
These same sorts of tools exist on almost all e-commerce websites in one form or another. A simple example is a search box. The option to search provides the visitor with a tool to find what they are looking for. Another example is product categories. A company doesn’t just put all of the products they have and place them into a single index page. They try to group them by how they think their (potential) customers would group them.
All of these things make it easier to engage with the business. The tools make it easier to shop or discover product. This, hopefully, will get you to buy more product which is how the business can justify the investment in whatever tool they have developed. Some, like the search box, are easy to see the value in while others may just be for fun. Either way the end goal is the same.
Clearly it costs money to develop and implement these tools, therefore the initiative has to generate money. This could be directly (sale now) or indirectly (brand goodwill for later purchase and/or consideration). The hard part is measuring the value of one of these tools retrospectively. If it was upfront, then test, test test. But, in some cases, that just isn’t possible or wasn’t done for some other reason and yet the leadership will still want some number to hang their hat on even if it is just a very rough estimate.
The problem with reading tools after the fact is that it is very hard to prove direct value. Sure you can see how much demand came through the tool. Or how many visits interacted with it and how many times it was used in total. You can look at participation metrics for the tool. You can look at data point after data point but one very tough question will remain: how much of this is incremental?. Even if you launch something and it is used by a significant portion of your visits, you will still have a hard time saying what is truly incremental.
For instance, mother’s day is coming up. Let’s say you launch a page that lists many products for mom. You may see that for those visits the % of time that a product is viewed and then purchased will decline. But don’t worry, they are just browsing for gifts. They are using the tool exactly as expected. But if this change in behavior happened on the whole site you would have an issue because people are viewing product and simply not buying. You might read that as a poorly designed product page or that your visitor can’t find the information they want.
On the flip side, you may launch some tool and find that anybody that touches the tool is like pure gold. Conversion is up 50% for visits that use the tool. Everything looks great, right? Well, what if all of those people were your best customers anyway; what if they are your most engaged and most likely to buy? Perhaps your tool is actually lowering conversion for these people, but the fact that they perform so well compared to the site as a whole causes you to miss this issue. And, at worst, to declare the new tool a success!
The solution, of course, is to test everything. That way you will have a true and clean read of the exact lift of a new tool.
What are your favorite tools that you wish all sites had?
This has been a Thought From The Cake Scraps.
Test EVERYTHING, huh? The guy in the article below doesn’t blame Google for it, exactly, but it’s an interesting point that seems like it fits with your theme in terms of the difficulty in understanding how much of an impact something has on your metrics. His specific example refers to Google’s testing on which of 41 shades of blue to use in their logo. Is a slightly greener shade of blue REALLY what’s going to be what’s causing people to click more? Maybe so.
http://stopdesign.com/archive/2009/03/20/goodbye-google.html
The author of that post links to this article giving a more in-depth explanation:
“A designer, Jamie Divine, had picked out a blue that everyone on his team liked. But a product manager tested a different color with users and found they were more likely to click on the toolbar if it was painted a greener shade.
Skip to next paragraph
As trivial as color choices might seem, clicks are a key part of Google’s revenue stream, and anything that enhances clicks means more money. Mr. Divine’s team resisted the greener hue, so Ms. Mayer split the difference by choosing a shade halfway between those of the two camps.
Her decision was diplomatic, but it also amounted to relying on her gut rather than research. Since then, she said, she has asked her team to test the 41 gradations between the competing blues to see which ones consumers might prefer.”
@Fantasy7 It is a very common struggle. The business wants data to drive decisions, designers want beauty. It is like flash-intro websites. They get oohhs and ahhs from other designers, but it is a pain for anybody that just wants to use the site.
This is not to say that we should abandon ‘gut feelings’, as they are necessary at times; there just needs to be data surrounding the decisions to prove that it is worth the money. It really is true that people click on a button more often when color X is used vs color y. The numbers don’t lie.
I want search bars that not only exist but actually work decently, which is very rarely the case. I often find it’s easier to use Google to find what I want within a site rather than using the site itself!