The video “Why Quantum Mechanics is Weird” (25k views) won the 2005 Berkeley Video&Film Festival Best of Festival Award in Education. That is second place, behind a Grand Festival Award which went to a film on polar bears (much cuter than me talking for 27 minutes). In this blog, I will go through the math behind the video which provides a novel, entirely mathematical explanation for why causality is different between classical and quantum mechanics. Calculus done correctly in spacetime may turn out to provide the correct answer, completely philosophy-free, as it must be.

The riddle of why quantum mechanics is different from classical physics has been around since the birth of quantum mechanics in 1901. Max Planck took an inspired leap using only integers worked to resolve the black body radiation problem. Einstein applied that idea the photo-electric effect in 1905. Bohr got on the bandwagon by quantizing angular momentum as part of a model of the hydrogen atom. The math of all three of these is simple and consistent with experiments.

That is all physics needs: equations that are consistent with experiments. Well, not exactly. We also demand the back story, why it all works. That is the open issue with quantum mechanics, its interpretation. Here are two less often used quotes:

Very interesting theory - it makes no sense at all.
Groucho Marx
We have always had a great deal of difficulty understanding the world view that quantum mechanics represents. At least I do, because I'm an old enough man that I haven't got to the point that this stuff is obvious to me. Okay, I still get nervous with it.... You know how it always is, every new idea, it takes a generation or two until it becomes obvious that there's no real problem. I cannot define the real problem, therefore I suspect there's no real problem, but I'm not sure there's no real problem.
Richard Feynman

The issue remains open today. I went to the talk: "The cosmological interpretation of quantum mechanics" given by Max Tegmark at Harvard in September, 2010. The small room was packed with grad students. He started with a survey: which interpretation of quantum mechanics do you prefer? No one voted for the Copenhagen interpretation, too old school. Hugh Everett’s many worlds  got about the same number as David Bohm’s Pilot wave theory. The fact that we had a vote to me was data that we need a new, specific mathematical proposal. I voted for D, none of the above. We never vote on physics we understand. Physicists fail people who get the accepted answer wrong.

People enjoy pointing out the Einstein didn’t get quantum mechanics. He most certainly understood the details of the photo electric effect since it garnered him a Nobel Prize. He most certainly understood Planck’s work since he so admired the man. He most certainly understood the math behind Bohr’s atom which can be taught to sufficiently motivated high school students.

What did he never understand? Causality.

Classically, some event A leads to another event B that leads to a third event C. This is what happens in a Buster Keaton film even if each event looks improbable. Special relatively is a study in the subtleties of causality. Einstein was a grand master of the subject.

In quantum mechanics, all possible histories must be added up and averaged. If a certain class of unlikely events is omitted, the calculated answer is a little bit wrong. This is one of the things that eats up supercomputer time chewing through data from the LHC: there are so many things that could happen in high energy collisions, they all must be accounted for before looking for something new.

The math of quantum mechanics is flawless. (Note: slides were taken from a Boston Bar Camp talk I gave.)

One cannot set a bank of supercomputers to work on inverse femtobarns of data unless the math is the best we have constructed to date. That math works fine without understanding causality in quantum mechanics.

Here are the two largest math ideas generated over the entire history of physics:
1. Calculus
2. Spacetime

We most strongly associate these developments with the two biggest names in the history of physics, Sir Isaac Newton and Albert Einstein. 

Gottfried Leibniz got to calcualus on his own and had the same big wig. Einstein’s math teacher Hermann Minkowski had to point out the Lorentz transformations represented a rotation in spacetime.

Big ideas need big help.

What are these two big ideas? Calculus is a precise, mathematical study of change. Spacetime is a formal link made between the scalar time and a three dimensional vector space. My thesis is that if the calculus of spacetime is done correctly, then we will understand the difference between causality in classical versus quantum mechanics. Calculus when the scalar time goes to zero is not the same as calculus when a 3-vector goes to zero.

A quaternion has a slot for the scalar time, and three more for the 3-vector. Quaternions are a division algebra, meaning that addition, subtraction, multiplication and division are all well-defined. Take out a college calculus book, and write down the limit definition of a derivative:

(Thomas&Finney, 5th edition, p. 31) Oops, this expression is meaningless for quaternions. The reason is that quaternions are not a mathematical field. The key difference between a mathematical field and a division algebra is that mathematical fields must commute while division algebras may or may not. Quaternions do not commute. Should the delta x be written on the right or the left? It makes a difference for quaternions, but it does not for real and complex numbers.

One of the guiding principles in my unfunded basement research project is that real and complex numbers are subgroups of quaternions. One must be able to pick out the real and complex numbers inside of a quaternion house with all their properties and functions because they live under the same roof.

The problem with the limit definition of a derivative is the cross product. That is the only part that cares about the delta element being on the left or right. I stole the two limit trick from L’Hospital’s rule, which was actually done by one of the Bernoulli brothers. Let the problem 3-vector go to zero first, then the scalar go to zero. Here is the time-like quaternion derivative definition:

The name “time-like” comes from considering the difference between two events. If the changes between those two events are greater in time than the three directions of space combined, the events are time-like separated. One event may cause another. The two events can have a causal link. A different label I could have chosen was to call this a directional derivative along the real axis. To be a true mathematical physicist, perhaps I should call it the time-like directional derivative along the real axis, but that is wordy.

Whatever the label, the definition stays the same. Put this to use on a simple function, f=q2. This was the first time I solved a limit problem outside of MIT :-)

Even if you have never solved a limit problem, a few minutes staring at these four lines might provide a flavor of calculus. The first line is the definition with f squared as the function. The second line multiplies out all the terms. The third line applies the limit of the 3-vector going to zero. The forth has the scalar going to zero. How does the function of the q squared change, no matter what q is? This calculation says that the instantaneous change that happens as the change in time goes to zero is twice whatever q is.

What happens if the changes in time are smaller than the changes in space? The order of the two limits reverse. First time stops, then changes in the 3-vector goes to zero. Oops, that brings up the problem of the cross product. What can be done with that?

The norm of the derivative is independent of the cross product. The size of the derivative stays the same no matter where it points in space. Call this the space-like quaternion derivative:

[correction: In the comments below, David Halliday pointed out that the definition I originally used (norm, limit, limit) did not have the properties I claimed. This correction (limit, norm, limit) is like the time-like derivative definition in that by the second limit, the value of the differential is a real number.]

If the changes between two events are greater in the three directions of space combined than the changes in time, the events are spacelike separated. The two events are independent of each other. There can be no causal link between the two events. A mathematician would to call this the norm of the derivative. It provides less information, only the overall size of change, not its direction. Some information is better than nothing.

Apply the space-like quaternion derivative to the function used before, f = q2:

[correction: as noted above, the norm must be taken before the second limit process for this to work.]

The derivation changes only at the third line: who goes to zero first. A small bit of magic is required to go from line 3 to 4. What I had to realize is everything here is a norm, everything is a scalar, everything commutes with each other. The norm of  d(0,R)/d(0,R) is unity no matter if the division happens on the left or the right.

The time-like and space-like quaternion derivatives of the same function give different results (2 q versus twice the norm of q). 

It is my belief that classical physics is governed by the time-like derivative.
Here events can be be put into a totally-ordered set according to their time: this happened, then that, then that. We can make a movie of what happened - say ten frames. The changes have directions, like velocities and accelerations. Every pattern has a cause.

It is my belief that quantum mechanics is governed by the space-like derivative. Changes in time don’t happen, how odd. We cannot make a movie. Instead we take the same ten pictures, put them all together, then shine a really strong light so we can see the superposition of the ten frames. That is what the wave function looks like. 

No one gets to see the complete wave function. Instead, by making a measurement, one plucks out one frame at a time which looks clear as day unlike the fuzzy but complete wave function.
 No event causes another one. All are independent. All we can know is the average.

The standard approach to quantum mechanics uses complex numbers. Real numbers do not suffice since they cannot account for all the wonderful interference seen in quantum mechanics. Complex numbers are a mathematical field, meaning they commute. There is no need for the two limit definition. Some might consider that an advantage. I think it is a liability because studying change correctly in spacetime justifies the deep difference between change in classical versus quantum mechanics.

There is a reason I don’t climb on top of a soapbox and bang a kettle to promote this novel explanation of the difference between classical and quantum physics. At this time, there is no testable difference between my proposal and what people use to do day-to-day calculations in quantum mechanics. Since I don’t have a specific experiment to discuss, I used the term belief.

I feel confident that someone with a strong opinion will jump on the comments soapbox and bang their kettle loudly. I honestly don’t want to fight with such people as it disturbs my tranquility. Unlike for Groucho, quantum mechanics makes sense to me. There is an absurd amount of work to do, but the ground feels solid, rooted in what could be the most fundamental definition of mathematical physics, the spacetime derivative.


Snarky puzzle: Complex analysis is a gold mind of important ideas. Quaternion analysis is a barren moonscape. The best work to date from a guy in the 1930’s was unable to show that q2 is analytic in q, dead end city. Grab a polynomial by the horns. Show that q*2 is analytic in q*. Bonus points if you do your proof five ways.

Next week: Presenting "Constructing a Consistent Linear Theory for Gravity: GR's Gravitomagnetism versus Hypercomplex G" at the 14th Eastern Gravity Meeting, Friday, June3, 4:30pm Jadwin 407, Princeton University [No, I was not an invited speaker, I volunteered, and they put me last.] I will present a variation of that talk here.