2017 is already shaping up to be a terrifying year for us statisticians (as it is for most of us Americans, but we’re going to focus on my math brethren for a moment).
First, “we got the election wrong,” so polling and projecting is met with more skepticism than ever. Now, the White House openly disputes statisticians on numerous issues, from climate change to inaugural crowd sizes. In a time where data is king (I’d argue it’s the whole royal court), some people are starting to distrust stats like they distrust the media.
So how can you identify the real stats from the “alternative stats”? Here are our top 5 ways:
- CHECK THE SOURCE
It sounds pretty straight forward, but the names of sources have become increasingly cryptic, and it can be difficult to spot the lobbyist from the independent institution. Bias is everywhere, but some sources are trying to disguise that bias under an official sounding name. Always Google the source and make sure they’re legitimate.
- ASK YOURSELF, “WHAT’S NOT BEING REPORTED?”
Daniel Kahneman has written at length about the human bias of WYSIATI (what you see is all there is). The notion is that we form impressions and judgments based on the information that is available to us, and do not regard missing information as evidence of something. With statistics, ask yourself what is the opposing stat to what’s being reported? For example, “the website has great engagement – 20% of users explore several pages and spend time on the site.” What about the other 80%? Do they not use the site? Are they just bouncing from the site without using it?
- TRUST EXPERTS WHO ADMIT UNCERTAINTY
Nate Silver is a legend in my book—not just for his superior website, or his longstanding track record of projecting the future, but because he understands uncertainty and embraces it. Statisticians who respect that the world is inherently unpredictable make the best predictions and dictate accurate caveats with their stats. If someone declares complete certainty (particularly when it comes to projections), be cautious in trusting their stats.
- IF IT SOUNDS TOO GOOD TO BE TRUE, IT PROBABLY IS
Nothing can be 100% successful 100% of the time, but that doesn’t stop some people from trying to act like it. You can tell someone is trying to only report successes when the key metrics change over time or if proven key metrics are being willfully ignored. Be critical of reports that chase success rather than presenting the truth.
- BE CRITICAL AND ENGAGE WITH THE STATS
Many public stats that use government figures are free data for you to look at—go for it! The more you play with data, the easier it is. Most stats are easy to replicate and some are as easy as looking at a picture. Not only will you know the truth, you’ll be able to have fruitful conversations with your analyst and keep them honest.
It’s going to be a tough year for a lot of us for numerous reasons, but if we all remember these five things, we will be able to easily navigate this world of “truthiness” and alternative facts.