WIMP - Is This 3 Sigma Signal The Road To Dark Matter?
    By Hank Campbell | April 16th 2013 09:44 AM | 28 comments | Print | E-mail | Track Comments
    About Hank

    I'm the founder of Science 2.0® and co-author of "Science Left Behind".

    A wise man once said Darwin had the greatest idea anyone...

    View Hank's Profile
    Cryogenic Dark Matter Search (CDMS) collaborators are reporting what could be a weakly interacting massive particle (WIMP) signal at the 3-sigma level. In common parlance, that is 99.7 confidence - which sounds high.  But to physicists it really means they have 3 bumps in their data that could be a WIMP, which means it might be a hint of dark matter.

    That sounds like a lot of qualifiers but particle physics is conservative that way. Data means what it means and not much more.  But certainly not less, and that is pretty important too.

    5-sigma is a modern designation for 'discovery' but 3-sigma or even 5-sigma is not telling the whole story. A 3-sigma result means there is a 1 in 370 chance it could just be wrong - polls are 2-sigma and that 95% confidence interval is why individually they don't have much value but, as we saw in the last US presidential election, averaging a bunch of polls makes for a lot better result - every statistician who made a call the day before election got every state right just by averaging a dozen polls. 5-sigma is something else. At 5-sigma the odds of a crazy result are down to 1 in over 1.7 million. The top quark discovery by the Tevatron at Fermi National Accelerator Laboratory (FNAL) in the 1990s, and the collaboration referring to it as 'evidence' before the 5-sigma 'discovery' result, is the time when physics commonly established 5-sigma as the discovery threshold. 

    But 5-sigma is not some magic number, it is still just a statistic. Sometimes people don't know what they are doing, or they want to manipulate results for legitimacy, so you instead have to understand how consistent their data is with the null hypothesis, a much trickier proposition. Fortunately, particle physics is filled with cranky people who like to argue and show each other up so errors are not likely to happen and deception is impossible. The benefit for us all is that the field uses p-values correctly as a result. 

    So if the data is consistent with the null hypothesis, a 3-sigma result may actually be quite good, but some context remains important.  Tevatron at FNAL reported a 2.5 sigma Higgs signal, for example. It clearly was right, we now know, and with enough collisions physicists knew that a Higgs signal had to have happened there even by the end of 2010 (assuming it existed - still a concern, then) but finding it was going to be difficult and what the LHC was designed to do much better - but it was not a discovery even though it was correct. On the other hand, a faster-than-light neutrinos claim was wrong even though it was a 6-sigma result. In my old area of physics, electromagnetic simulation, we were number one worldwide because people knew we weren't going to converge on an answer with a high degree of confidence, we were going to converge on the right one. 5-sigma results do not exempt anyone from systematic error.  Even with 5-sigma results it helps to have confirmation and that is why the LHC runs two experiments.

    CDMS has been searching for a dark matter signal since 2003 (CDMS II, before the now-running SuperCDMS, gathered the data in the paper) by using cryogenically cooled germanium and silicon crystals to identify nuclear recoils - an atom being kicked around by a WIMP is a fun thought - the idea is that though they must be invisible, because they don't interact with normal matter and can't be electromagnetically observed, if they exist they will interact with atomic nuclei and phonon detectors will sense the WIMP-nucleon collisions.

    And they found bumps in the data that stood out from the background at a 3-sigma level. Interesting stuff.

    Their Monte Carlo simulations found that the probability of a statistical fluctuation producing three or more similar events was 5.4% but their having a similar energy distribution is much more rare. As they wrote, "A likelihood analysis that includes the measured recoil energies of the three events gives a 0.19% probability for a model including only known background when tested against a model that also includes a WIMP contribution. This ~3-sigma confidence level does not rise to the status of a discovery, but does call for further investigation."

    If what they have is what they interpret as spin-independent scattering of WIMPs, they see it being a mass about 8.6 GeV/c² and a WIMP-nucleon cross section of 1.9E-41 cm² - which disagrees with exclusion limits from the XENON collaboration. That's a cautionary note and I asked Texas A&M's Dr. Rupak Mahapatra, part of the CDMS collaboration, about what that might mean.

    "The colliders normally look for higher mass WIMPs, due to threshold effects. Although, recent searches geared towards these low mass WIMPs, in events called mono-jet, LHC has probed sensitivity down to about 10^-39 cm^2, about 100 less sensitive than the observed mean cross-section," he wrote in an email.

    Experimental upper limits (90% confidence level) for the WIMP-nucleon spin-independent cross section as a function of WIMP mass. We show the limit obtained from the exposure analyzed in this work alone (black dots), and combined with the CDMS II Si data set reported in (blue solid 
    line). Also shown are limits from the CDMS II Ge standard and low-threshold analysis (dark and light dashed red), XENON10 S2-only (light dash-dotted green), and XENON100 (dark dash-dotted green). The filled regions identify possible signal regions associated with data from Co-GeNT (magenta, 90% C.L., as interpreted by Kelso et al. including the effect of a residual surface event contamination described , DAMA/LIBRA (yellow, 99.7% C.L.), and CRESST (brown, 95.45% C.L.) experiments. 68% and 90% C.L. contours for a possible signal from these data are shown in blue and cyan, respectively. The asterisk shows the maximum likelihood point at (8.6 GeV/c2, 1:9x10-41 cm2).

    Fair enough. Of course, plenty of 3-sigma results have been wrong in the past and disappeared into background when more data was gathered, which is why multiple experiments often chase the same goal. People new to physics at the time of the Higgs boson announcement were surprised two experiments were doing the same thing, it sounded like overkill, but without two the results would have been less meaningful. They got a lot of matching bumps in the LHC data that stood out from the background. 

    What might this mean for dark matter and supersymmetry?  Not much yet, though it will energize physicists searching for a particle solution to missing mass and perhaps get Modified Newtonian Dynamics (MOND) less attention. Obviously it's a 3-sigma 'this is interesting' result, and that's encouraging. They'll continue using the germanium detectors in the Soudan experiment and see what they can find in this region. SuperCDMS is scheduled to move from Soudan mine in Minnesota to Vale Inco Mine in Sudbury, Canada, a deeper facility and that means less likelihood of interference by known background particles, which will certainly help. And other experiments are also searching for dark matter, like the Alpha Magnetic Spectrometer, which has measured 30 billion cosmic ray events so far.
    Headlines like Scientific American's "Dark Matter Signals Recorded in Minnesota Mine" aren't doing physicists or readers any favors and are clearly designed to promote a magazine rather than science - because if an actual discovery occurs the public will think it has already happened a dozen times. But some more attention for particle physics may be good - a squabble over who can name the 400 trillion planets in the universe has caught the public's attention and that is probably good for space science. People can just read Science 2.0 instead of corporate science media for the straight scoop. 

    So tell your friends to bookmark us if they like independent media!

    Preprint: Silicon Detector Results from the First Five-Tower Run of CDMS II, CDMS Collaboration, arXiv:1304.3706


    So what do you think climate science is, maybe 1 sigma?
    Never is a long time.
    It's the wrong question. You wouldn't ask, 'is physics 2 sigma'? Theoretical physics is no sigma but that doesn't mean you can jump off a building and gravity won't kill you.
    Fair enough, though let me note, gravity is at least 6 sigma :)
    Never is a long time.
    Thanks for the good discussion. Other news articles are effectively unreadable precisely for reasons stated in the last paragraph and you link the free pre-print.

    Although I know it's a losing battle, data are plural.

    I always forget my blog posts are peer-reviewed.

     In some sentences it is going to be a disconnect: "Data means what it means" can't be 'data mean what they mean' without being weird, which means instead a rewrite to a much longer sentence and a lack of clarity. I don't get paid by the word. 

    "So if the data is consistent with the null hypothesis" is wrong, yeah.
    What is Dark Matter?
    This 4 page conceptual vixra article conjectures that, more energetic than the cosmic gamma photons, there should exist dark photons and Planck photons. Dark photons may be the particles of the elusive dark matter.

    Rick Ryals
    Sometimes an expert like Dorigo will tell you why his years of experience are screaming at his gut that a 3 sigma ain't squat... and he'll be right for that reason more often than not.

    On the other hand, a 2.5 might indicate a discovery if you find it in the same place every time you look.
    That was a very informative article with depth I could not include in mine.  :)
    Science advances as much by mistakes as by plans.
    Good article Hank. I linked to it on physicsworld, where for some strange reason they'd referred to an article by a non-expert. Hi Hontas.

    Hi John, thanks for the link. If you know anyone there, you might want to write whoever runs PW that they have an issue with their comment form. It seems to store information. When I went to the article to look the comment field was already pre-filled with your info. I tried to send you an email with a screenshot but it bounced.
    I saw pre-fill on the 16th and contacted them about it. They're on the case. I've been getting "new comment for your post" emails OK. I'll try a test reply.

    The write-up is too verbose. It would have been more interesting if it were 1/3 of its present length.

    That's why USA Today exists. People who want the meat in science have options also. Celebrate diversity.
    The write-up is too verbose. It would have been more interesting if it were 1/3 of its present length.
    1148 words is too long? It's 0.8 % the length of an an average novel. If you are reading at an average learning rate of 150 words per minute, it's about 7 and half minutes of your time. You must be very busy!

    Enrico, when a thing is well-understood and well-digested, the brevity in its presentaion comes by itself.

    when a thing is well-understood and well-digested, the brevity in its presentaion comes by itself.
    True, but the minimal length varies in relation to the topic.

    There is no way in 300 words anyone is writing anything more than journalism for 14-year-olds. "3 sigma" "dark matter" blah blah. It's not a lack of understanding that makes it a thousand words, it's that physics and the statistics and models in use, is complicated and needs an explanation.  A short statement can be a pithy metaphor, and people may see it as clever - a "Daily Show" for science is the last thing we need - but it isn't educating anyone.  People who already know the subject may say it is too long, they can also sign up here and write the perfect version. 
    Hank, one is not talking or dealing with stuff for 14-year ones. The "abstract" of a long paper has to be very short yet very expressive of its contents.

    An abstract is written for the 400 people who are going to read it, primarily experts in their fields and subscribers to a technical journal. Preprints don't even bother with an abstract because they knew even fewer people will read it.  In recent culture, a fetish about '5 sigma' has been created precisely because no one talks about what it means - experts already know and journalists just repeat a number and then say it is a 'gold standard'.  You're saying you could explain both the paper's results and what the p-values and everything else about a physics result mean in 350 words. I'd like to see that.  I predict it will read like it was written for a 14-year-old who wants to be a web designer when they grow up.
    Hank, it is relatively easy to write something in two or more pages than in less than in one page, because it needs more concentration and effort.

    "Make everything as simple as possible, but not simpler," Einstein is quoted as saying. I think by having a word-length focus and editing to fit that you are going to sacrifice understanding.  Long-form exists because some people want that. I grant you, USA Today has 60X the readership of Science 2.0, so it's not wrong to focus on brevity. I regard The Economist as one of the best publications in the world, it is as tightly-edited as can be, but people in science do not read The Economist for science.

    If you can condense everything in physics to 350 words, I applaud you. My inability to do so is a weakness the audience endures. To me, theoretical physics is complicated and needs explanation. I have to wonder how many other physicists will agree that anything in this field can be understood in 1.5 pages of a book.
    Simplicity is part of common sense. This is what I tried to put forwards. Here one half page or one third of it, is not fundamental. However, I hope our discussion was of some use. I wish you all the best in your endeavour.

    What has the XENON experiment got to do with colliders?

    If one project has a result that is excluded by another, you ask why.  I linked to their expected signal and asked a collaborator on CDMS how the public can understand what is happening. 

    You would have written an article and not mentioned other groups are searching for dark matter?  That would be public relations, not science.
    My point is that by mentioning colliders and mono-jets, Mahapatra seems to be answering a question about the LHC, rather than explaining why the XENON collaboration doesn't seem to be seeing a similar result to CDMS. I dont think XENON is an LHC experiment-- I dont think it is even associated with any LHC-like colliders. Am I wrong?

    Oh, I see. I say "LHC" rather than ATLAS and CMS because that is how people refer to the experiment in total but we are talking about detectors. WIMPs should be all around us so a Xenon detector might be able to find them passively, just like AMS might by looking at cosmic rays. Those aren't generating the collisions, they are just happening organically.

    There are a dozen dark matter experiments right now, they all do something different. But, no, Xenon 100 is in Italy, not at the LHC, and LUX is South Dakota, AMS is in space, etc.
    particle physics is filled with cranky people who like to argue and show each other up ...

    Fixed!   ;-)

    But seriously, a great explanation of what sigma means in physics, and why that is important.