"Oh Wind, if Winter comes, can Spring be far behind ?"

Good old Shelley inspired me to start today's article with the above verse, taken from his magnificent "Ode to the West Wind". With the weather we are experiencing these days in Geneva and northern Italy, I found it a relieving thought...

So, winter conferences are over, and summer ones are still far away. This is therefore a nice moment to try an assessment on the quality of the results that the two competing CERN experiments have produced on the study of the Higgs boson. Why ? Because we are not going to have to change our conclusions in a short time scale caused by a result about to be published.

How to compare the results

It is actually quite complicated to have a clear picture, because of the large number of results and measurements that have been produced. This multitude is due to two factors. Or maybe three.

One - the intrinsic interest in the topic, which has polarized the attention not just outside of, but also within the collaborations.

And, of course, two - the peculiarity of the particle we are studying: the Higgs boson made us the great favour of having a mass which enables it to decay in many different observable channels; the fact that also the production mechanisms are multiple creates the possibility to search for a signal in literally dozens of different final states.

And finally one must note that the LHC experiments have a chance, because of the significant number of Higgs bosons produced and the good signal to noise ratio of some channels, to study several distinct characteristics, and not just mass and production rate: its spin and parity, most notably; as well as its couplings to fermions and bosons.

There is therefore a real jungle of measurements, and in this summary article I am conveniently using a compilation of results produced by a colleague. Any mistake is however my own responsibility.

In order to compare the accuracy of the two experiments one needs to rely on expected results rather than observed ones, because the latter are affected by statistical fluctuations. Sometimes that information is not available, e.g. when one compares measurements of properties rather than p-values or expected upper limits on signal rate.

In that case, one unavoidably is confronted with a situation where the experiment which observes the largest signal has the smallest statistical uncertainty. One can however still compare the systematic one to gauge how powerful is the analysis method and the experimental technique, and how good is the signal purity. It is hard to give guidelines here however, so conclusions are not univocal in specific cases. But let us look at the numbers and then we may get back to the issue on a case-by-case basis.

The numbers compared

Let us start with the study of Higgs decays to W boson pairs, detected by collecting samples of events containing lepton pairs (electrons and muons) and missing transverse energy from the neutrinos in the double leptonic WW-> lνlν final state. Here ATLAS reports an expected significance of 3.5 standard deviations, while CMS has an expected significance of 5.1 standard deviations. The larger a priori sensitivity of CMS is probably due to analysis choices on the lepton selection and data reduction.

On the Higgs decays to tau lepton pairs, H->ττ, ATLAS has not produced results using the whole 2011+2012 statistics, so a comparison is not too meaningful. Of course CMS has a better result here because of using more data, but it does not teach us much in this instance. The situation is similar in the H->bb decay mode, CMS again has analyzed more statistics but it would be unfair to compare results.

On the H->ZZ search, there are several things to note. The expected significance of the four-lepton signal at 125 GeV still sees CMS on top, with 5.6 standard deviations against 4.4 of ATLAS. Interesting to note that Nature (the bitch) chose otherwise, with an observed significance actually higher in ATLAS (6.6 against 4.7 sigma); but if we compare sensitivity the figure of merit is the expected one, as I noted above.

By relying on the careful examination made by my colleague, we can give a look at why CMS has a higher sensitivity. This is due to both the better mass resolution of CMS (1.2 GeV versus 1.6 GeV of ATLAS in the four-muon mode; 1.7 GeV vs 1.9 GeV of ATLAS in the two-muon-two-electron mode; and 2.0 GeV vs 2.4 GeV in the four-electron mode)  and better selection, which grants lower backgrounds in CMS (1.6+-0.6 events versus 2.1+-0.6 in ATLAS in the 4μ mode; 4.0+-1.6 in CMS versus 8.5+-0.2 in ATLAS in the 2μ2e mode; and 2.5+-1.0 versus 4.5+-0.8 in the 4-electron mode).

Given the above numbers, it is hardly surprising that also the mass measurement produced by CMS from the signal detected in the 4-lepton decay mode has smaller systematics: +-0.2 GeV, versus +0.6/-0.5 GeV in the corresponding ATLAS analysis.

Finally, let us consider the Higgs decay to diphotons. Here there is a substantial parity, with expected significances of 4.1 standard deviations in ATLAS and 4.2 in CMS. And again, as seen in the ZZ mode, there is a startling difference in the observed signals here: the observed significance in ATLAS is 7.4 standard deviations!, while in CMS we see a downward fluke yielding 3.2 standard deviations.

The mass measurements produced by the two collaborations of course are affected by the large difference in yields observed; anyway we can compare the systematic uncertainty, which is 0.6 GeV in CMS and +-0.7 in ATLAS. The statistical one has ATLAS winning by far, with +-0.2 versus +-0.5 due to the much larger signal seen there.

In general, other things being equal one could expect CMS to have a lead also in the diphoton mode if one compares the mass resolution. It is 1.77 GeV in ATLAS and 1.64 GeV in CMS; the best category of diphotons, those measured more accurately in the central detectors, still sees CMS doing better with 1.27 GeV versus 1.40 GeV of ATLAS. This notwithstanding the better pointing capabilities of ATLAS (remember, photons do not yield tracks in the central detectors, so it is sometimes hard to decide where the Higgs boson was produced along the beam axis; this translates in uncertainties in the reconstructed mass).

It is hard to compare the quality of spin-parity measurements, so I will spare myself the trouble. Instead, one may compare the combined mass measurement of ATLAS and CMS. The former measures M=125.5 +-0.2 (stat) +0.5/-0.6 (syst) GeV, while the latter measures M=125.7 +- 0.3 (stat) +- 0.3 (syst) GeV. Regardless of the larger observed signals of ATLAS, the total uncertainty on the mass is smaller in CMS (by squaring stat. and syst. uncertainties one gets +0.54/-0.63 GeV in ATLAS and +-0.42 in CMS).


So in conclusion, I would say that CMS has shown to have a better sensitivity than ATLAS on the Higgs boson; but the observed results have on the other hand favored ATLAS in some of the measurements. Perhaps the combined mass measurement is a good all-inclusive testing ground, and here we see CMS winning by a significant margin.

On the other hand, I warn the reader that the numbers should not be taken as a direct indication of differing detector performances, since the difference in chosen analysis techniques provide a good part of the differences observed in expected sensitivity.

That said, I would like to reject the easy accusation of having being partisan in writing this article. Indeed, I am a member of the CMS collaboration and have been involved in the production of the above results, so I am by no means an independent observer; however, the quoted numbers are publically accessible by anybody willing to make up their mind independently from my own conclusions... The numbers are numbers, and I am sorry if I cannot reply as Groucho "These are my numbers, and if you don't like them... I don't have others!".

As some of you will remember, I have been often criticized for what I write in this blog, but I fail to see how a true assessment of experimental reach can do any harm. The comments thread below is open to anybody willing to present their own assessment of the situation. And finally, a disclaimer:

The opinions expressed in the above article are those of the author, who is not representing anybody here and speaks as a free observer of publically accessible experimental results.