Gauge symmetry is a clear, direct idea in EM, as I will detail. This property is essential, [correction: due to redundancy found in the 4-potential description of light and not any issue of the speed of travel for the wave]. Any proposal for gravity must also have [gauge symmetry]. I will show how the Lagrangians I have discussed over the last few months do not have this property, therefore they are wrong. I have altered three titles to say "RETRACTION", including a brief explanation at the start. [Note: the corresponding YouTube video titles have also been marked.] A different Lagrangian I am toying with might have a chance, but that will be discussed at another time.

Click or skip this reading of the blog.

Gauge symmetry is like all other symmetries: find changes that are both real and don't matter. Rotate a circle and the circle looks the same (the group S1 for those in the know). In EM, one works with derivatives of potentials, here written using quaternion products instead of tensor notation:

$\nabla \times A = (\frac{\partial \phi}{\partial t} - \nabla \cdot A, \frac{\partial \vec{A}}{\partial t} + \vec{\nabla} \phi + \vec{\nabla} \times \vec{A})$
The vector part is the difference of the magnetic and electric fields. What about the first term? Note: I avoided calling it the scalar term as I am prone to do because people might think it was invariant under a Lorentz transformation which is not the case. The first term does not play a part in the EM Lagrangian, so subtract it away:

$(\nabla \times A - (\nabla \times A)^*)/2 = (0, \frac{\partial \vec{A}}{\partial t} + \vec{\nabla} \phi + \vec{\nabla} \times \vec{A})$

The same subtraction happens in the tensor notation. In a path integral approach to quantum mechanics, the field strength tensor goes upstairs into the exponent of an exponential as part of the process to form the action. Take the product of these fields with the field strength formed by reversing the order of the derivative and the potential to generate the Lorentz invariant magnetic field squared minus the electric field squared. The exponential of the field strength tensor would have a norm of one, since this is a property of quaternions whose first term is zero in the exponent of an exponential. Unit quaternions have SU(2) symmetry, and U(1) is a subgroup of that symmetry. I don't feel on solid ground, reaching up to an exponential cloud as it were, so just file this one under a tentative observation related to U(1) symmetry.

Devise something that can be added to the 4-potential, but will not change these fields. The addition makes the operation a transformation, while the lack of change makes it a symmetry. A primary target is the curl. A gradient is like radial change, while curl is perpendicular change. The curl of a gradient of a scalar function is always zero. Try the transformation:

$(\phi, \vec{A}) \rightarrow (\phi', \vec{A'}) = (\phi, \vec{A}) + (0, \vec{\nabla} f)$

where f is an arbitrary scalar field. Plug this into the definition of the magnetic field:

$\inline \vec{B} \rightarrow \vec{B'} = \vec{\nabla} \times \vec{A'} = \vec{\nabla} \times \vec{A} + \vec{\nabla} \times \vec{\nabla} f = \vec{B}$

The job is half complete. See how this effects the other two terms that constitute the electric field:

$\inline -\vec{E} \rightarrow -\vec{E'} = \frac{\partial \vec{A'}}{\partial t} + \vec{\nabla} \phi = \frac{\partial \vec{A}}{\partial t} + \frac{\partial \vec{\nabla}f }{\partial t} + \vec{\nabla} \phi \ne -\vec{E}$

That is a no go. Nothing has been done with the phi. Pick the signs wisely:

$\inline (\phi, \vec{A}) \rightarrow (\phi', \vec{A'}) = (\phi, \vec{A}) + (-\frac{\partial f}{\partial t}, \vec{\nabla} f)$

The new phi transformation will not effect the magnetic field. See the effect on the electric field:

$\inline -\vec{E} \rightarrow -\vec{E'} = \frac{\partial \vec{A'}}{\partial t} + \vec{\nabla} \phi' = \frac{\partial \vec{A}}{\partial t} + \frac{\partial \vec{\nabla}f }{\partial t} + \vec{\nabla} \phi - \vec{\nabla} \frac{\partial f}{\partial t}= -\vec{E}$

That works.

Here is the Lagrangian for EM:

$\mathcal{L}_{EM}&=\mathcal{L}_{matter} + \mathcal{L}_{interaction} + \mathcal{L}_{fields} \\ &=\sum_{particles}(-\frac{m c^2}{\gamma} - (J \times A + (J \times A)^*)/2) \\ &\quad-\frac{1}{8}\int(\nabla \times A - (\nabla \times A)^*)\times(A \times \nabla - (A \times \nabla)^*) \\ &\quad \quad+ (\nabla \times A - (\nabla \times A)^*)\times(A \times \nabla - (A \times \nabla)^*)^*)~dx~dy~dz \\ &\approx\sum_{particles}(\frac{1}{2} m v^2 - q \phi + q V \cdot A) -\frac{1}{2}\int(B^2-E^2)~dx~dy~dz$

The only places derivatives of the potential appear are in the field terms. One uses the Lagrangian to get force equations, field equations, and the stress-energy tensor.

What does one do with this newly proved gauge symmetry? One can pick a gauge. Here are three popular examples:

Coulomb gauge: $\inline \nabla \cdot A = 0$
Static gauge: $\inline \frac{\partial \phi}{\partial t} = 0$
Lorenz gauge: $\inline \frac{\partial \phi}{\partial t} + \nabla \cdot A= 0$

Pick any of these gauges, and that will not change the EM Lagrangian at all. Picking a gauge does change how equations that result from varying the action are written.

To say a Lagrangian is invariant under a gauge transformation has three attributes: the tentative observation regarding the subtraction, examples of gauge choices that do not alter the Lagrangian, and most importantly, a proof of symmetry under a gauge transformation. Some will argue that only the third issue is of consequence. I mention the first two because the Lagrangians I have worked with did have those properties. It indicates I was not totally negligent to the issue of gauge symmetry. Two out of three is bad when the third one is the most important.

Start down the hypercomplex gravity dead end as before with the fields defined for the hypercomplex product:

$\nabla^* \boxtimes A = (\frac{\partial \phi}{\partial t} - \nabla \cdot A, \frac{\partial \vec{A}}{\partial t} - \vec{\nabla} \phi - \vec{\nabla} \otimes \vec{A})$
Toss out the first term:

$\nabla^* \boxtimes A - (\nabla^* \boxtimes A)^* = (0, \frac{\partial \vec{A}}{\partial t} - \vec{\nabla} \phi - \vec{\nabla} \otimes \vec{A})$

Propose a similar gauge transformation, taking note of the change in signs in the first two terms:
$(\phi, \vec{A}) \rightarrow (\phi', \vec{A'}) = (\phi, \vec{A}) + (\frac{\partial f}{\partial t}, \vec{\nabla} f)$
See how the first two terms do:

$\frac{\partial \vec{A}}{\partial t} - \vec{\nabla} \phi \rightarrow \frac{\partial \vec{A'}}{\partial t} - \vec{\nabla} \phi' = \frac{\partial \vec{A}}{\partial t} + \frac{\partial \vec{\nabla}f }{\partial t} + \vec{\nabla} \phi - \vec{\nabla} \frac{\partial f}{\partial t}= \vec{e}$
That works. On to the final term:

$\vec{\nabla} \otimes \vec{A} \rightarrow \vec{\nabla} \otimes \vec{A'} = \vec{\nabla} \otimes \vec{A} + \vec{\nabla} \otimes \vec{\nabla} f \ne \vec{b}$

That doesn't.

Big picture people spotted this long ago (the start of October). I am the lab tech sort. I spent four or five hours this weekend trying a wide variety of gauge transformations. I now have a few pages of futility, a good Zen practice. There are so many masters to satisfy: field equations, force equations, solutions to both field and force equations, and of course gauge symmetries. I pondered the lessons swallowed trying to staple a force equation to my previous field theory efforts. I have no problem admitting I became a desperate man in the comments section, willing to give up a rule of calculus to make things work.

And I have one other mandate. The Maxwell equations are drop dead gorgeous. Should equations for gravity stand beside them, they must also have that quality. While the hypercomplex gravity equations got close, a flaw is a flaw is a flaw by any other name.

One of the lessons learned flopping around like a fish on the ground was the rules for vector calculus are so wired into my mind's eye, I was not able to see the Z2xZ2 symmetry powering the hypercomplex products, with the Klein 4-group behind the curtain. Playing with conjugates that were Z2-like appeared to be productive. There might be some there there. That will be left for another week.

Doug

Snarky puzzle: The oh-so-familiar conjugate operator is a tool of the quaternion Wall Street elite hiding in a velvet vector dress. The rectangular box world of Z2xZ2 prefers to divide the world in two. Doodle with these:
Let the Z2 conjugate *i, *j, and *k do flip two signs:

$\\ (\phi, Ax, Ay, Az)^{*i} \equiv (\phi, Ax, -Ay, -Az) \\ (\phi, Ax, Ay, Az)^{*j} \equiv (\phi, -Ax, Ay, -Az) \\ (\phi, Ax, Ay, Az)^{*k} \equiv (\phi, -Ax, -Ay, Az)$

Form the following products:

$\\ (\rho/\sqrt{3}, Jx, Jy, Jz) \boxtimes (\phi/\sqrt{3}, Ax, Ay, Az)^{*i}\\ (\rho/\sqrt{3}, Jx, Jy, Jz) \boxtimes (\phi/\sqrt{3}, Ax, Ay, Az)^{*j}\\ (\rho/\sqrt{3}, Jx, Jy, Jz) \boxtimes (\phi/\sqrt{3}, Ax, Ay, Az)^{*k}$
Just for fun, take the sum.

Google+ hangout: 11:00-11:45pm Eastern time, Tuesday-Friday. http://gplus.to/sweetser
This could be an efficient way to exchange a few ideas. If you have a question or two, hangout.

Next Monday/Tuesday: A Lagrangian between Newton and Maxwell