Anecdotal evidence over statistical evidence?
Does verifying a theory by looking at few experimental data validate our thesis? Aren’t we misrepresenting the scientific process in that situation. For example, experiments and its results that are sometimes conducted only once, and without adequate controls are generalized to make definitive claims, rather than repeating the experiment and using statistical analysis, as a scientist would do to figure out what is really true. That brings us to the question, should we change what we’re doing i.e., really show the repetitive nature of science and use statistical results as evidence for our claims. Doesn’t human experience matter in this regard?
In my opinion the answer is no. But to understand why we first have to dig into something called the helping experiment. In one of the experiments, students were invited to play a game of Monopoly. What was unknown to them is that the game was rigged in favor of one of them. And, whenever a particular player would start getting clearly ahead, he would exhibit the signs of dominant behavior. Once the game was finished, the winner would say that it was their tactics that got them to win, not that they have realized that the game was rigged. In this experiment, people who observed the students were aware whose favor the game was rigged in. And there was no control group where people played un-rigged sessions of Monopoly. And the whole subject group was homogeneous and consistent purely of young university student who were mainly men, so there was no consideration given to any other factors that could have affected the results. Yet, according to the standards of sociology, the methodology was perfectly valid and sufficient to make the conclusions that were made — that money makes people mean. This is despite the fact that there wasn’t even any money involved in the experiment, except for Monopoly money.
Another experiment was done to see whether wealthier people were more likely to break the law. To determine this, a camera was placed next to a zebra crossing. By law, pedestrians at such crossings were given priority and the drivers were supposed to stop once a pedestrian sets foot on the crossing. A researcher would start crossing the road as some car was about to drive past. And if the car failed to stop for the researcher, it’s make and model was recorded. Expensive cars were later used as a proxy indicator of the wealth of its driver. It was found that more of the expensive cars were failing to stop compared to cheaper ones. Therefore it was concluded that wealthier people are more likely to be willing to break the law.
However, once again, the methodology was anything but scientific. Double blind principle wasn’t employed, so a researcher could have influenced the outcome by either stepping on the crossing too late for the car to stop safely, or standing on it in such a way that the driver wouldn’t easily notice him, which appeared to be the case in some of the videos shown in the TED talk. Likewise, nobody considered other factors. Perhaps, it’s those people who spend more time on the road and not necessarily the wealthiest who are more likely to drive past the pedestrian. This was suggested in one of the videos by a bus failing to stop for the pedestrian as well. Finally, what if it was just an area where expensive cars significantly outnumbered inexpensive ones? What about having a control run in a poorer area to see if it’s the total proportion of drivers that are less likely to stop for the pedestrians and not just the drivers of expensive cars?
So, next time you hear some statements of fact coming from the practitioners of social sciences (psychologists, sociologists, political scientists, etc.), take it with a massive grain of salt and look up the source yourself.
An important life decision everyone takes is a choice of romantic partner. There is no proper scientific way of determining what kind of person would suit you best. You will either have to rely on your own experience or experience of other people. And scientists are not exception.
Even the scientists will follow their gut feelings to make important life decisions. Even scientists will ask opinion of their friends (read: directly ask for anecdotal evidence). So, if they are not shying away from anecdotal evidence, why should you?
And I think this highlights for us that our brains are much better at working with individual stories and, and things in detail than they are with statistical results. It isbetter to communicate science, to tell the story to show the experiment really once in a dramatic way rather than three or four times, where each new iteration. Well, each repetition just confirms the original result that you were talking about. A big caveat is that if you’re actually doing the science if you’re actually trying to establish scientific facts like that, of course, you need the repetition and the statistical analysis. So I think it really does come down to what your objectives are, but with this conclusion. I think this opens up to big potential pitfalls.
- One is that people without scientific evidence can make crafty stories that catch on and quickly become what people feel is the truth, and the other pitfall is scientists who have strong scientific evidence, who have clear statistical results, and yet they can’t communicate them to people because they don’t have a great story.
- On the other hand, you have things like climate change, which very strong statistical evidence to back them up large scale results over the globe. And yet, one cold snowy winter is so much more visceral and meaningful to individual people, then this thing which feels, you know, completely data based. And it just depends on how much you trust data, I guess, as scientists we love data. And we feel like if we’re trying to communicate to someone, we’re trying to convince someone of something, all we need to do is show more data. But what experiments, demonstrate to us with statistical certainty is that stories work much better.
- Whenever somebody claims that a particular technique based purely on anecdotal evidence is truly scientific and expects a hefty sum of money to teach you the technique, I would probably stay away. It’s likely to be a scam.
Closing thoughts:
A story is much better at influencing our decisions than hard data. For so many brand managers and brand owners the emphasis in their branding is focused on the ‘What & ‘How’ — what their products and services do and how they do it — when the most critical platform to build a brand upon is ‘Why’. The ‘why we do what we do’ proposition will always be more powerful and more engaging — and provides an opportunity to stand-out through brand story telling.
A core human need — stories make us feel like part of something bigger than ourselves.
So next time, describe the scene, show the results, and tell the story.
The key is to bring out your story in a way that draws people in.
