Statistical hypothesis testing is quite boring if you apply it to cases where you know the answer, or where the data speak loud and clear about one hypothesis being true or false. Life at the interface between testability and untestability is much more fun.
In fact, if you have read this blog for long enough, you will have certainly observed that the hottest debates and the peaks of interest appear when we discuss stuff that is at the borderline - anomalies, such as those I discussed in my recent book; effects whose origin could both be statistical (a fluctuation) or systematic (a real effect, be it a problem with the measurement or a new physics signal).
One such case is the excited B_c meson signal produced in 2014 by the ATLAS collaboration (preprint here). They studied a sample of J/Psi meson decays to muon pairs, wherein they could identify a pretty solid signal of B_c meson decays  . This is shown in the plots below, which show separately the mass distribution of signal candidates in 7 TeV and 8 TeV LHC proton-proton collisions acquired in the 2011 and 2012 runs, respectively. The B_c meson is a particle composed of a bottom and a charge quark (both bottom-anticharm and antibottom-charm combinations count, and have opposite charge, giving rise to pions of the corresponding charge accompanying the neutral J/psi).

ATLAS took candidates within 3 standard deviations of the B_c peaks above and combined the B_c signal with two additional pions, in search of a structure in the resulting mass distributions. When you have one positive and one negative additional pion you might be looking at decays of an excited B_c meson to a B_c plus a pion pair; while when you have two additional pions of the same charge you can't expect the originator to be a well-behaved meson - mesons cannot have a +- 2 charge, as they contain only a quark-antiquark pair. 

What ATLAS observed, and published in 2014, is shown below: the Q-value distribution (a function created by considering the mass of the 4-body system minus the mass of the B_c minus twice the pion mass, which equates the free energy available in the decay of the alleged excited state) shows a peak in both 7 and 8 TeV data. The data are shown as black points with vertical uncertainty bars, and a signal fit is overlaid in red. The background-only model is shown as a dotted blue curve, and the green histogram shows the data with wrong charge pairings of the pions.

ATLAS reported the signals as incompatible with the estimated backgrounds at a level of 3.7 and 4.5 standard deviations, respectively, and quoted the combined effect as a 5.2 sigma one - so well above the "five sigma" threshold that has become an established threshold above which you are authorized to claim discovery of a new physics process. So this is the end of the story, right ? After all, nobody doubts that bottom and anticharm quarks may create bound states that possess additional binding energy from angular momentum quantum numbers. This kind of physics is observed in all lighter mesons and baryons, so the claim is not so extraordinary. 

However, a confirmation from an independent experiment is a good thing if you can afford to have it. And indeed, LHCb recently studied 2 inverse femtobarns of collisions acquired at 8 TeV to search for the same signal. LHCb can boast about a much larger bounty of B_c signal candidates in the first place, as the detector is optimized for B hadron physics. As you can see below, their B_c signal (in the same final state studied by ATLAS, the J/psi pi one) is impressive and it dwarfs the ATLAS one. Data are shown as black points, and the various contributions are shown with various colours. The red component is the B_c signal.

The news, reported two weeks ago by LHCb, is that their analysis of those B_c candidates, performed similarly to ATLAS by combining them with pairs of additional pions, does not evidence any peaking signal. You can check it for yourself in the graphs below, which show the mass distributions of the four particles (J/Psi plus three pions) in four separate subsets of data, ordered by the value of a multi-layer perceptron (MLP) used to discriminate the excited Bc signal from backgrounds (the highest signal-populated histogram should be the bottom right one, for highest MLP values). Again, the data are shown as black points with uncertainty bars, and the estimated background is a filled histogram. The excited B_c signal should appear as a bumpy structure between the red vertical lines. Nope, nothing like that is apparent, you got that right.


So what is going on here ? Is LHCb "blind" to those excited states because of some obscure kinematical effect ? One could in principle imagine that the two experiments have different efficiency for collecting the excited B_c signal in data containing a B_c decay to J/Psi plus pion. Or that the forward production of those excited states is suppressed. The LHCb preprint does explain, in the concluding section, that 

"The LHCb and ATLAS results are compatible only in case of very large (unpublished)
relative efficiency of reconstructing the B^*_c (2S)+ candidates with respect to the B_c
signals for the ATLAS measurement."

The alternative, which I cannot subscribe to but still buzzes somewhere in the back of my mind, is that ATLAS got enamoured with a large, but not unheard of, statistical fluctuation in their data. Since I have often said, in seminars around the world and in an article (linked above), that the multiple testing of hypothesis performed by the LHC experiments is bound to result in a spurious 5-sigma effect sooner or later, I cannot deny that this could be the case... 

I hope that CMS will enter the fray and look into this matter. CMS had a slightly higher momentum resolution for muons than ATLAS the last time I checked, so it should be possible to get some good information from a targeted analysis of that signal. Let us wait and see what happens, then!


Tommaso Dorigo is an experimental particle physicist who works for the INFN at the University of Padova, and collaborates with the CMS experiment at the CERN LHC. He coordinates the European network AMVA4NewPhysics as well as research in accelerator-based physics for INFN-Padova, and is an editor of the journal Reviews in Physics. In 2016 Dorigo published the book “Anomaly! Collider physics and the quest for new phenomena at Fermilab”. You can get a copy of the book on Amazon.