It is always awesome to observe how much information is contained in it. It is 1526 pages long, and I wonder how many typos and mistakes are contained in the data-thick pages... Probably much fewer than an ordinary book. Some of the review articles are of exceptionally good quality, because they have been passed from hand to hand in the last few dozen years, and constantly improved. If you want an example, for instance, go to the "Statistics" section - you will find a lot of new material which, along with the old one, still meets the highest standards.
My attempt today at finding a mistake, which serves also as a good example to check if we have understood how weighted averages are computed by the particle data group, rests on checking the PDG average for the Omega_b mass. You might know that this particle has been observed independently by CDF and DZERO a few years ago, in both cases with claimed significances above five standard deviations. The mass measurements, however, are in stark disagreement - CDF quotes a mass of 6054.4 +- 6.8 +- 0.9 MeV, while DZERO has it at 6165 +- 10 +- 13 MeV ! The difference between these two values is about six standard deviations away from zero, making it certain that either one or both measurements are flawed [I have written several posts on this matter, actually so many that I do not even fancy the idea of putting together links here! In any case, just google "Omega site:www.science20.com/quantum_diaries_survivor" if you are curious!].
By "flawed" above I might be referring to many different possibilities, some of which I think are quite unlikely. But let me list them below for clarity anyway:
- The two measurements are incompatible because they measure two different resonances. Nah!
- The incompatibility is due to a statistical fluctuation. Wellll, a one-in-ten-millions one ?
- The disagreement is due to one or both the two experiments heavily underestimating a systematic uncertainty. That's more like it... And if you ask me privately I will tell you which one is faulty!
Now, if you know or ever read about (and then forgot) what the "PDG method" for averaging independent measurements of particle properties, you may recall that it is a simple weighted average, the weights being as they should the inverse total variances (errors squared) of the determinations. Also, there is provision for a "Scale Factor" in the determination of the error to associate to the PDG weighted average. This scale factor is meant to multiply the ordinary error of the weighted average when the chisquared per degree of freedom of the various inputs is larger than one. Well, not surprisingly, in the case at hand the PDG scale factor turns out to be a whopping 6.22, corresponding to the six-sigma discrepancy I was mentioning above.
The PDG explains, on page 14 of its 2012 edition (sec. 5.2.2 of the Introduction -sorry I meant to link it here but today the site seems to be down, anyway you can try this link tomorrow and work your way to the relevant section) how the unconstrained averaging is performed, and how the scale factor is determined. In particular, they say that
"If the reduced chisquared, i.e. χ^2/(N-1) [here N is the number of measurements that get averaged - TD's note] is very large, we may choose not to use the average at all. Alternatively, we may quote the calculated average, but then make an educated guess of the error, a conservative estimate designed to take into account known problems with the data".But this is apparently not what they do. Instead, they do use the scale factor quoted above, although it is evidently too large for a weighted average to have any meaning! Okay, diatriba mode off - after all I did not mean to criticize this honored institution, but rather review with you how to treat the combination of independent measurements when they agree well, and when they only agree so-so.
There are other subtleties in the weighted average "A' la PDG", but in the rather straightforward case we have chosen we should just take the two numbers and use the formulas: the center value is not even affected by the scale factor, which only applies to the error estimate. So in an attempt at being scrupulous, today I am doing my homework. Here is what I find:
CDF determination: x1 = 6054.4 MeV
CDF total estimated uncertainty: δ1 = sqrt(6.8^2+0.9^2) MeV = 6.8593 MeV
CDF weight: w1 = 1/δ1^2 = 0.021254 / MeV^2
DZERO determination: x2 = 6165 MeV
DZERO total estimated uncertainty: δ2 = sqrt(10^2+13^2) = 16.4012 MeV
DZERO weight: w2 = 1/δ2^2 = 0.0037175 / MeV^2
So I get
Weighted average: x_aver = (x1*w1 + x2*w2) / (w1+w2) = 6070.86 MeV,
which is exactly what the PDG quotes (6071 MeV). And about the uncertainty:
Standard error on weighted average: δ = sqrt[1/(1/δ1^2+1/δ2^2)] = 6.328 MeV
which is what one would have to take if one were to consider "compatible" the two estimates. But they are not of course! Indeed, one must consider that the chisquare of the two determinations is
χ^2 = w1*(x1-x_aver)^2+w2*(x2-x_aver)^2 = 38.704
whose square root is, as we have already stated, 6.22. Hence the PDG wants us to multiply the error of the weighted average by 6.22, obtaining
Scaled error on weighted average: δ' = 6.328*6.22 = 39.37 MeV
I may have ran into rounding errors, but my estimate of the uncertainty is one MeV smaller than the PDG, which quotes the Omega_b mass average as 6071+-40 MeV. So maybe I can boast I've found a typo! ;-) In any case I hope this example can be useful to those of you who casually pick PDG average numbers as inputs to other quantities in calculations etc.: knowing how things are made is always a guarantee that one does not make bad use of them!