The Value of Good Research
Ever since I took a Statistics course in college, I have been unable to trust any statistic reported by news organizations. While I saw how they could be useful to companies and political campaigns, I learned that reporting on them within the limited space of the Old Media is always incomplete, misleading, and potentially damaging. What's more, if they were to be used as a means to guide official policy, the people relying on them would be at the mercy of the honesty of the pollster.
This potential was made evident by the recent scandal involving pollster DataUSA, whose owner admitted falsifying polling data to meet deadlines. The employees were reportedly instructed--and I swear I'm not making this up--to poll dogs and cats if they could not get all the subjects they needed.
The motivation of DataUSA to falsify data was to make their customers happy rather than their customers' best interests. When truth was needed, the polling company chose instead to provide reports that put smiles on their customers' faces, which is a serious faux pas that will result in 27-33 months of prison time for the owner and a hefty fine.
Happily, the most common mistakes we can make with statistics probably won't result in prison or fines, but that doesn't mean our responsibility to get it right (not just in intent, but also in effect) is diminished in any way.
Research is essential to what we do as marketers, and the deeper the quality of the research we do, the better we will be able to do our jobs. Surveys are simply tools we can use to get our data. When determining how stakeholder behavior will affect how a site is designed, we really have to walk a mile in their shoes... We must do what they do. This tactic can be used to either validate the data we derive from our surveys or help us design better surveys with fewer biases.
But in a way, if we're not careful, relying on research too much for every little thing can also be limiting. Many times the research is gathered to measure how a user can imagine interacting with something, so the answers we get are limited by the user's imagination. We will have to come up with new and interesting ideas all by ourselves. Research can and should guide those ideas--it may even be the genesis of them--but it cannot take the place of innovation itself. Acting as if it does will only protect the status quo. -- Cam Beck
One of my favorite books is a little gem called How To Lie With Statistics. It was written in 1954 by Darrell Huff, but it is just as relevant today as it ever was.