That sounds like a lot of qualifiers but particle physics is conservative that way. Data means what it means and not much more. But certainly not less, and that is pretty important too.
5-sigma is a modern designation for 'discovery' but 3-sigma or even 5-sigma is not telling the whole story. A 3-sigma result means there is a 1 in 370 chance it could just be wrong - polls are 2-sigma and that 95% confidence interval is why individually they don't have much value but, as we saw in the last US presidential election, averaging a bunch of polls makes for a lot better result - every statistician who made a call the day before election got every state right just by averaging a dozen polls. 5-sigma is something else. At 5-sigma the odds of a crazy result are down to 1 in over 1.7 million. The top quark discovery by the Tevatron at Fermi National Accelerator Laboratory (FNAL) in the 1990s, and the collaboration referring to it as 'evidence' before the 5-sigma 'discovery' result, is the time when physics commonly established 5-sigma as the discovery threshold.
But 5-sigma is not some magic number, it is still just a statistic. Sometimes people don't know what they are doing, or they want to manipulate results for legitimacy, so you instead have to understand how consistent their data is with the null hypothesis, a much trickier proposition. Fortunately, particle physics is filled with cranky people who like to argue and show each other up so errors are not likely to happen and deception is impossible. The benefit for us all is that the field uses p-values correctly as a result.
So if the data is consistent with the null hypothesis, a 3-sigma result may actually be quite good, but some context remains important. Tevatron at FNAL reported a 2.5 sigma Higgs signal, for example. It clearly was right, we now know, and with enough collisions physicists knew that a Higgs signal had to have happened there even by the end of 2010 (assuming it existed - still a concern, then) but finding it was going to be difficult and what the LHC was designed to do much better - but it was not a discovery even though it was correct. On the other hand, a faster-than-light neutrinos claim was wrong even though it was a 6-sigma result. In my old area of physics, electromagnetic simulation, we were number one worldwide because people knew we weren't going to converge on an answer with a high degree of confidence, we were going to converge on the right one. 5-sigma results do not exempt anyone from systematic error. Even with 5-sigma results it helps to have confirmation and that is why the LHC runs two experiments.
CDMS has been searching for a dark matter signal since 2003 (CDMS II, before the now-running SuperCDMS, gathered the data in the paper) by using cryogenically cooled germanium and silicon crystals to identify nuclear recoils - an atom being kicked around by a WIMP is a fun thought - the idea is that though they must be invisible, because they don't interact with normal matter and can't be electromagnetically observed, if they exist they will interact with atomic nuclei and phonon detectors will sense the WIMP-nucleon collisions.
And they found bumps in the data that stood out from the background at a 3-sigma level. Interesting stuff.
Their Monte Carlo simulations found that the probability of a statistical fluctuation producing three or more similar events was 5.4% but their having a similar energy distribution is much more rare. As they wrote, "A likelihood analysis that includes the measured recoil energies of the three events gives a 0.19% probability for a model including only known background when tested against a model that also includes a WIMP contribution. This ~3-sigma confidence level does not rise to the status of a discovery, but does call for further investigation."
If what they have is what they interpret as spin-independent scattering of WIMPs, they see it being a mass about 8.6 GeV/c² and a WIMP-nucleon cross section of 1.9E-41 cm² - which disagrees with exclusion limits from the XENON collaboration. That's a cautionary note and I asked Texas A&M's Dr. Rupak Mahapatra, part of the CDMS collaboration, about what that might mean.
"The colliders normally look for higher mass WIMPs, due to threshold effects. Although, recent searches geared towards these low mass WIMPs, in events called mono-jet, LHC has probed sensitivity down to about 10^-39 cm^2, about 100 less sensitive than the observed mean cross-section," he wrote in an email.
Experimental upper limits (90% confidence level) for the WIMP-nucleon spin-independent cross section as a function of WIMP mass. We show the limit obtained from the exposure analyzed in this work alone (black dots), and combined with the CDMS II Si data set reported in (blue solid
line). Also shown are limits from the CDMS II Ge standard and low-threshold analysis (dark and light dashed red), XENON10 S2-only (light dash-dotted green), and XENON100 (dark dash-dotted green). The filled regions identify possible signal regions associated with data from Co-GeNT (magenta, 90% C.L., as interpreted by Kelso et al. including the effect of a residual surface event contamination described , DAMA/LIBRA (yellow, 99.7% C.L.), and CRESST (brown, 95.45% C.L.) experiments. 68% and 90% C.L. contours for a possible signal from these data are shown in blue and cyan, respectively. The asterisk shows the maximum likelihood point at (8.6 GeV/c2, 1:9x10-41 cm2).
Fair enough. Of course, plenty of 3-sigma results have been wrong in the past and disappeared into background when more data was gathered, which is why multiple experiments often chase the same goal. People new to physics at the time of the Higgs boson announcement were surprised two experiments were doing the same thing, it sounded like overkill, but without two the results would have been less meaningful. They got a lot of matching bumps in the LHC data that stood out from the background.
What might this mean for dark matter and supersymmetry? Not much yet, though it will energize physicists searching for a particle solution to missing mass and perhaps get Modified Newtonian Dynamics (MOND) less attention. Obviously it's a 3-sigma 'this is interesting' result, and that's encouraging. They'll continue using the germanium detectors in the Soudan experiment and see what they can find in this region. SuperCDMS is scheduled to move from Soudan mine in Minnesota to Vale Inco Mine in Sudbury, Canada, a deeper facility and that means less likelihood of interference by known background particles, which will certainly help. And other experiments are also searching for dark matter, like the Alpha Magnetic Spectrometer, which has measured 30 billion cosmic ray events so far.
Headlines like Scientific American's "Dark Matter Signals Recorded in Minnesota Mine" aren't doing physicists or readers any favors and are clearly designed to promote a magazine rather than science - because if an actual discovery occurs the public will think it has already happened a dozen times. But some more attention for particle physics may be good - a squabble over who can name the 400 trillion planets in the universe has caught the public's attention and that is probably good for space science. People can just read Science 2.0 instead of corporate science media for the straight scoop.
So tell your friends to bookmark us if they like independent media!
Preprint: Silicon Detector Results from the First Five-Tower Run of CDMS II, CDMS Collaboration, arXiv:1304.3706
Comments