The experiment aims at searching for muon decays to electrons in the field of heavy nuclei (tungsten or aluminum, as far as I understand from a quick browsing). Leaving aside the nucleus, I would bet that if you have a basic knowledge of the standard model of fundamental interactions, gathered by reading standard undergraduate books such as the Perkins or the Burcham-Jobes, you might be tempted to say that muon conversions to electrons are a very exotic kind of non-standard physics to search for. In fact, they entail the violation of the rule of lepton number conservation, which has been one of the best-verified rules of fundamental physics in the XXth century.
In reality, we know that lepton flavour violation must occur: this bit of knowledge came from the groundbreaking results of the Super-Kamiokande experiment in Japan, which in 1998 proved the neutrino oscillation phenomenon. Both cosmic-ray neutrinos and neutrinos from the sun were observed to change flavour, solving an astrophysical puzzle which had lasted for over twenty years (the solar neutrino deficit problem).
When neutrinos change flavour, they violate lepton flavour conservation all right. This of course need not be a proof that charged leptons (the electron, the muon, and the tau) also turn one into another: flavour violation could well be a mechanism specific of neutrinos. However, Occam's razor -a principle usually preventing the construction of unnecessary hypotheses, and as such a great tool for the sceptical mind- can be used this time to support the notion that there is nothing special with charged leptons: being electrically charged is arguably a less fundamental property of being a lepton. So if neutrinos do it, why not electrons and muons ?
If lepton flavour is violated also by charged leptons, we may expect decays such as the one of a muon to an electron plus a photon, or even direct conversion of a muon into an electron in the field of a nucleus; in both cases, no neutrinos are emitted, so energy conservation can be checked with precision if one is capable of detecting and measuring the final state bodies. Given the ease of producing intense beams of muons at today's accelerator facilities, and the rather striking signature of the mentioned decays, it is not surprising to learn that the rate of muon decays to electrons has been already constrained to be smaller than 2.4 x 10^-12 (two point four trillionths), e.g. by the mu-e-gamma experiment, or down to 7x10^-13 (seven tenths of a trillionth) by the SINDRUM II experiment.
While the electron plus photon decay used by MEG exploits the back-to-back, energy-balancing decay into a photon and an electron (see figure, right), Mu2E like SINDRUM II searches for an electron with energy almost exactly equal to the muon rest mass (the heavy nucleus taking off only a very small fraction of the released energy, to conserve momentum!). The expected single-event sensitivity, given the total statistics envisioned in the report, corresponds to 5x10^-17, probing four orders of magnitude deeper into extremely rare decays. This is quite a lot of unexplored land, where many new physics scenarios could pitch in, turning on the searched reaction.
In the graph on the left you can see the kind of signal expected by SINDRUM II: a tiny peak of muon conversions to electrons at the energy corresponding to the muon rest mass. The three histograms with different shades of grey show the data at different levels of selection. The darker one is after the removal of the nasty cosmic-ray background: a signal corresponding to some decays per trillion muons is shown by the white dots.
The Mu2E experiment will cost about 250 million dollars, and is planning to operate in year 2020 at Fermilab. A sketch of the experimental area is shown in the graph on the right, where you can see on the left the interaction point where pions are copiously produced; muons produced from the decay of pions are then directed to the detector on the right, where decay electrons can be accurately measured. The figure does not include the cosmic ray shield, which is a must in this kind of experiments.
I believe that the price tag of the experiment, given that it is one which puts all the eggs in one basket, being as it is a very focused search on a narrow hypothesis, is rather steep. However, what I think is nice about this project is that if we found some kind of Supersymmetry at the LHC we would benefit a lot from measuring the rate of muon decays to electrons with Mu2E. In the graph below, also taken from the conceptual report, you can see that much of the parameter space accessible to LHC could in principle allow coherent muon decays to electrons at rates measurable by Mu2E. So the complementarity of this experiment to others is certainly there. Also worth mentioning is the complementarity of the physics responsible for coherent muon-to-electron decays with the one investigated by experiments searching for the muon-to-electron-gamma decay: there are two different parts of an extended Lagrangian which enable the two processes, so one really needs to study both processes. More on this topic in Section 3.2 of the report.
In the figure you can see as red and green crosses the predicted conversion rates of muons into electrons as a function of the universal mass M_1/2, for two distinct scenarios of supersymmetry, and at tan(β)=10. The points "illuminate" the parameter space allegedly accessible to the LHC. I am not sure what assumptions are used to constrain the space accessible to the LHC, but the report cites a very thick paper written by LHC theorists and experimentalists, so it is probably okay.