Banner
    Weighted Averages "A' La PDG"
    By Tommaso Dorigo | January 17th 2013 11:30 AM | 4 comments | Print | E-mail | Track Comments
    About Tommaso

    I am an experimental particle physicist working with the CMS experiment at CERN. In my spare time I play chess, abuse the piano, and aim my dobson...

    View Tommaso's Profile
    The new PDG - a full-size copy of the glorious "Review of Particle Properties"- is on my desk since its arrival a few weeks ago, but only today did I get some time to browse it.

    It is always awesome to observe how much information is contained in it. It is 1526 pages long, and I wonder how many typos and mistakes are contained in the data-thick pages... Probably much fewer than an ordinary book. Some of the review articles are of exceptionally good quality, because they have been passed from hand to hand in the last few dozen years, and constantly improved. If you want an example, for instance, go to the "Statistics" section - you will find a lot of new material which, along with the old one, still meets the highest standards.

    My attempt today at finding a mistake, which serves also as a good example to check if we have understood how weighted averages are computed by the particle data group, rests on checking the PDG average for the Omega_b mass. You might know that this particle has been observed independently by CDF and DZERO a few years ago, in both cases with claimed significances above five standard deviations. The mass measurements, however, are in stark disagreement - CDF quotes a mass of 6054.4 +- 6.8 +- 0.9 MeV, while DZERO has it at 6165 +- 10 +- 13 MeV ! The difference between these two values is about six standard deviations away from zero, making it certain that either one or both measurements are flawed [I have written several posts on this matter, actually so many that I do not even fancy the idea of putting together links here! In any case, just google "Omega site:www.science20.com/quantum_diaries_survivor" if you are curious!].

    By "flawed" above I might be referring to many different possibilities, some of which I think are quite unlikely. But let me list them below for clarity anyway:

    - The two measurements are incompatible because they measure two different resonances. Nah!
    - The incompatibility is due to a statistical fluctuation. Wellll, a one-in-ten-millions one ?
    - The disagreement is due to one or both the two experiments heavily underestimating a systematic uncertainty. That's more like it... And if you ask me privately I will tell you which one is faulty!

    Now, if you know or ever read about (and then forgot) what the "PDG method" for averaging independent measurements of particle properties, you may recall that it is a simple weighted average, the weights being as they should the inverse total variances (errors squared) of the determinations. Also, there is provision for a "Scale Factor" in the determination of the error to associate to the PDG weighted average. This scale factor is meant to multiply the ordinary error of the weighted average when the chisquared per degree of freedom of the various inputs is larger than one. Well, not surprisingly, in the case at hand the PDG scale factor turns out to be a whopping 6.22, corresponding to the six-sigma discrepancy I was mentioning above.

    The PDG explains, on page 14 of its 2012 edition (sec. 5.2.2 of the Introduction -sorry I meant to link it here but today the site seems to be down, anyway you can try this link tomorrow and work your way to the relevant section) how the unconstrained averaging is performed, and how the scale factor is determined. In particular, they say that

    "If the reduced chisquared, i.e. χ^2/(N-1)  [here N is the number of measurements that get averaged - TD's note] is very large, we may choose not to use the average at all. Alternatively, we may quote the calculated average, but then make an educated guess of the error, a conservative estimate designed to take into account known problems with the data".
    But this is apparently not what they do. Instead, they do use the scale factor quoted above, although it is evidently too large for a weighted average to have any meaning! Okay, diatriba mode off - after all I did not mean to criticize this honored institution, but rather review with you how to treat the combination of independent measurements when they agree well, and when they only agree so-so.

    There are other subtleties in the weighted average "A' la PDG", but in the rather straightforward case we have chosen we should just take the two numbers and use the formulas: the center value is not even affected by the scale factor, which only applies to the error estimate. So in an attempt at being scrupulous, today I am doing my homework. Here is what I find:

    CDF determination: x1 = 6054.4 MeV
    CDF total estimated uncertainty: δ1 = sqrt(6.8^2+0.9^2) MeV = 6.8593 MeV
    CDF weight: w1 = 1/δ1^2 = 0.021254 / MeV^2

    DZERO determination: x2 = 6165 MeV
    DZERO total estimated uncertainty: δ2 = sqrt(10^2+13^2) = 16.4012 MeV
    DZERO weight: w2 = 1/δ2^2 = 0.0037175 / MeV^2

    So I get

    Weighted average: x_aver = (x1*w1 + x2*w2) / (w1+w2) = 6070.86 MeV,

    which is exactly what the PDG quotes (6071 MeV). And about the uncertainty:

    Standard error on weighted average: δ = sqrt[1/(1/δ1^2+1/δ2^2)] = 6.328 MeV

    which is what one would have to take if one were to consider "compatible" the two estimates. But they are not of course! Indeed, one must consider that the chisquare of the two determinations is

    χ^2 = w1*(x1-x_aver)^2+w2*(x2-x_aver)^2 = 38.704

    whose square root is, as we have already stated, 6.22. Hence the PDG wants us to multiply the error of the weighted average by 6.22, obtaining

    Scaled error on weighted average: δ' = 6.328*6.22 = 39.37 MeV

    I may have ran into rounding errors, but my estimate of the uncertainty is one MeV smaller than the PDG, which quotes the Omega_b mass average as 6071+-40 MeV. So maybe I can boast I've found a typo! ;-)  In any case I hope this example can be useful to those of you who casually pick PDG average numbers as inputs to other quantities in calculations etc.: knowing how things are made is always a guarantee that one does not make bad use of them!

    Comments

    Sec 5.3: "The basic rule states that if the three highest order digits of the error lie between 100 and 354, we round to two significant digits. If they lie between 355 and 949, we round to one significant digit. Finally, if they lie between 950 and 999, we round up to 1000 and keep two significant digits. In all cases, the central value is given with a precision that matches that of the error."

    dorigo
    Right Andre, but then they should have quoted 6070+-40, not 6071+-40 !
    Cheers,
    T.
    Maybe they just wanted to quote a just-one-significant-digit number as the uncertainty?

    Halliday
    Tommaso:

    I don't know that "knowing how things are made is always a guarantee that one does not make bad use of them!"  (Emphasis added.)  However, I most certainly agree that "knowing how things are made is" essential in optaining the goal "that one does not make bad use of them!"

    David