The characteristics of Λ, the cosmological constant, resemble those of the painter Caravaggio: a dubious origin, a career spotted by scandal, and yet the creator of a magnificent world. Indeed, many astronomers think that Λ is the cause of the expansion and evolution of our Universe.

The cosmological constant occurs on the left-hand side of the Einstein equation1

where Rij is the Ricci tensor, gij the metric tensor, G the gravitational constant, Tij the stress–energy tensor and c the speed of light. Translated from maths into English, this expression tells us that the structure and the evolution of spacetime are determined by the distribution of mass, energy and momentum.

Einstein's remarkable formula is constructed as follows2. The speed of light, c, is an invariant, so the Universe is invariant under Lorentz symmetry. This implies that c is also the largest possible speed. Therefore, a global symmetry cannot exist — only local symmetries are possible in a relativistic world. The metric tensor, gij, describes the geometric properties of spacetime; the left-hand side of the Einstein equation is a function of gij and its derivatives and is completely determined by the properties of the local Lorentz symmetry.

Einstein introduced the third term on the left, Λgij, to obtain a static solution for the Universe (using a lower-case lambda3). This term conveys that space itself has a fixed energy density, just by virtue of its being there. The more space there is, the bigger the contribution of Λ to the expansion of space. Hence the De Sitter model for a, the scale factor of a universe without matter, expanding exponentially: . It was 'quoted' in a 1930 cartoon (pictured, courtesy of Leiden Observatory) with the caption: “But who inflates the ball? What causes the Universe to expand, or swell? That's what lambda does — a different answer cannot be given.”

In recent decades, the De Sitter model universe has been resurrected in the concept of cosmic inflation, in which the value of Λ is time-dependent and craftily adjusted through the introduction of hypothetical 'inflaton' fields. These must be fine-tuned with extreme care to reproduce a universe with roughly the properties that are observed in the actual Universe.

The empty-space energy density represented by the Λ-term resembles the zero-point energy of the vacuum in quantum field theory. It is therefore natural to ask if they are equal, even if only approximately. The answer is: not at all. Quantum chromodynamics requires Λ ≈ 10118 times the value implied by astronomical observations, which is about 10−9 J m−3. If local SU(3) quantum symmetry is a good description of the colour force, then our Universe ought to have blown up in a De Sitter explosion on a timescale of 10−34 seconds or so.

For practical purposes, Λ is negligible. But from the Casimir effect we know that quantum vacuum energy exists, so Λ is not exactly zero. Somehow its influence on the large-scale Universe must be reduced with a precision of one part in 10118.

To a mathematician, the placement of the terms in the Einstein equation does not matter. But to a physicist, it does. Putting Λ on the right expresses, essentially, that the energy density of the spatial vacuum is somehow related to the presence of particles, and the structure of the quantum vacuum. A future theory might replace that right-hand side by a suitably constructed 'improved standard model' with a corresponding equation of state. Conversely, one could place the Tij term on the left-hand side. Such a move would symbolize the point of view that particles are a form of spacetime structure, possibly in a space with more than three dimensions, or with a complicated topological structure.

But the most worrisome part of that equation is the equals sign itself. The structure and the evolution of spacetime are determined by the distribution of mass, energy and momentum. How so? We need a dynamical mechanism for the interaction between spacetime and matter, instead of that equals sign on which Einstein centred his equation. Such a mechanism should remove those 118 powers of ten, similar to the way in which quantization removed the ultraviolet catastrophe of blackbody radiation.

There are probably as many theories about Λ as there are researchers4. In the absence of facts, these are all opinions, no matter how abstract the algebra in which they are expressed. Other than possibly the presence of 'dark stuff' (we do not even know whether it is matter) there is no observational evidence that gives us a reason to take sides. Maybe the gravitational-lens effect is not achromatic. Maybe the cosmic redshift holds a surprise. Whoever comes up with the answer will be partying in Stockholm shortly after.