Shannon Hall writes: It begins with the smallest anomaly. The first exoplanets were the slightest shifts in a star’s light. The Higgs boson was just a bump in the noise. And the Big Bang sprung from a few rapidly moving galaxies that should have been staying put. Great scientific discoveries are born from puny signals that prompt attention.

And now, another tantalizing, result is gathering steam, stirring the curiosity of physicists worldwide. It’s a bump in the data gathered by the Large Hadron Collider (LHC), the world’s most powerful particle accelerator. If the bump matures into a clearer peak during the LHC’s second run, it could indicate the existence of a new, unexpected particle that’s 2,000 times heavier than the proton. Ultimately, it could provoke a major update to our understanding of physics.

Or it could simply be a statistical fluke, doomed to disappear over time. But the bump currently has a significance level of three sigma, meaning that this little guy just might be here to stay. The rule of thumb in physics is that a one-sigma result could easily be due to random fluctuations, like the fair coin that flipped tails twice. A three-sigma result counts as an observation, worth discussing and publishing. But for physicists to proclaim a discovery, a finding that rewrites textbooks, a result has to be at the five-sigma level. At that point, the chance of the signal arising randomly is only one in a million.

There’s no knowing if the LHC researchers’ new finding is real until they gather more data. And even bigger would-be discoveries — those with five-sigma results and better — have led physicists astray before, raising hopes for new insights into the Universe before being disproved by other data. When pushing the very limits of what we can possibly measure, false positives are always a danger. Here are five examples where seemingly solid findings came undone. [Continue reading…]