Einstein thought this was all a bit much, believing it to be evidence of major problems with the theory, as many critics still suspect today. Quantum enthusiasts point to the theory's extraordinary success in explaining the behaviour of atoms, electrons and other quantum systems. They insist we have to accept the theory as it is, however strange it may seem.

But what if there were a way to reconcile these two opposing views, by showing how quantum theory might emerge from a deeper level of non-weird physics?

If you listen to physicist Tim Palmer, it begins to sound plausible. What has been missing, he argues, are some key ideas from an area of science that most quantum physicists have ignored: the science of fractals, those intricate patterns found in everything from fractured surfaces to oceanic flows (see What is a fractal?).

Take the mathematics of fractals into account, says Palmer, and the long-standing puzzles of quantum theory may be much easier to understand. They might even dissolve away.

It is an argument that is drawing attention from physicists around the world. "His approach is very interesting and refreshingly different," says physicist Robert Spekkens of the Perimeter Institute for Theoretical Physics in Waterloo, Canada. "He's not just trying to reinterpret the usual quantum formalism, but actually to derive it from something deeper."

That Palmer is making this argument may seem a little odd, given that he is a climate scientist working at the European Centre for Medium-Range Weather Forecasting in Reading, UK. It makes more sense when you learn that Palmer studied general relativity at the University of Oxford, working under the same PhD adviser as Stephen Hawking.

So while Palmer has spent the last 20 years establishing a reputation as a leading mathematical climatologist, he has also continued to explore the mysteries of his first interest, quantum theory (see "Quantum ambitions").

"It has taken 20 years of thinking," says Palmer, "but I do think that most of the paradoxes of quantum theory may well have a simple and comprehensible resolution."

Arguments over quantum theory have raged since the 1920s, starting with a series of famous exchanges between Einstein and the Danish physicist Niels Bohr.

Bohr and his supporters believed that the theory's successful description of atoms and radiation meant you should abandon old philosophical concepts, such as the idea that objects have definite properties even when no one is there to measure them.

Einstein and his followers countered that such radicalism was wildly premature. They argued that much of the quantum weirdness was nothing more than a lack of adequate knowledge. Find a quantum system's "hidden variables", Einstein suspected, and quantum theory might make common sense, a view that quantum enthusiasts thought was ultra conservative and out of touch. The argument rages to this day.

Fractals unite

Palmer believes his work shows it is possible that Einstein and Bohr may have been emphasising different aspects of the same subtle physics. "My hypothesis is motivated by two concepts that wouldn't have been known to the founding fathers of quantum theory," he says: black holes and fractals.

Palmer's ideas begin with gravity. The force that makes apples fall and holds planets in their orbit is also the only fundamental physical process capable of destroying information. It works like this: the hot gas and plasma making up a star contain an enormous amount of information locked in the atomic states of a huge number of particles. If the star collapses under its own gravity to form a black hole, most of the atoms are sucked in, resulting in almost all of that detailed information vanishing. Instead, the black hole can be described completely using just three quantities - its mass, angular momentum and electric charge.

Many physicists accept this view, but Palmer thinks they haven't pursued its implications far enough. As a system loses information, the number of states you need to describe it diminishes. Wait long enough and you will find that the system reaches a point where no more states can be lost. In mathematical terms, this special subset of states is known as an invariant set. Once a state lies in this subset, it stays in it forever.

A simple way of thinking about it is to imagine a swinging pendulum that slows down due to friction before eventually coming to a complete standstill. Here the invariant set is the one that describes the pendulum at rest.

Because black holes destroy information, Palmer suggests that the universe has an invariant set too, though it is far more complicated than the pendulum.

Complex systems are affected by chaos, which means that their behaviour can be influenced greatly by tiny changes. According to mathematics, the invariant set of a chaotic system is a fractal.

Fractal invariant sets have unusual geometric properties. If you plotted one on a map it would trace out the same intricate structure as a coastline. Zoom in on it and you would find more and more detail, with the patterns looking similar to the original unzoomed image.

Gravity and mathematics alone, Palmer suggests, imply that the invariant set of the universe should have a similarly intricate structure, and that the universe is trapped forever in this subset of all possible states. This might help to explain why the universe at the quantum level seems so bizarre.

For example, it may point to a natural explanation for one of the biggest puzzles of quantum physics, what physicists refer to as its "contextuality". Quantum theory seems to insist that particles do not have any properties before they are measured. Instead, the very act of measurement brings their properties into being. Or, put another way, quantum systems have meaning only in the context of the particular experiments performed on them.

Ever since Einstein, many physicists have hoped that a new approach might go beyond quantum theory and find a way to restore belief in objective and independent properties. But in 1967, mathematicians Simon Kochen and Ernst Specker published a theorem showing that this dream, if possible, cannot be done in quite the way physicists would like.

Central to Kochen and Specker's theorem is a thought experiment. Say you choose to measure different properties of a quantum system, such as the position or velocity of a quantum particle. Each time you do so, you will find that your measurements agree with the predictions of quantum theory. Kochen and Specker showed that it is impossible to conceive a hypothesis that can make the same successful predictions as quantum theory if the particles have pre-existing properties, as would be the case in classical physics.

This result has driven many physicists to reach a startling conclusion about how to interpret quantum theory. Either you have to abandon the existence of any kind of objective reality, instead believing that objects have no properties until they are measured, or you have to accept that distant parts of the universe share a spooky connection that allows them to share information even when the distance and timing means that no signal could have passed between them without travelling faster than light.

Palmer's idea suggests a third possibility - that the kinds of experiments considered by Kochen and Specker are simply impossible to get answers from and hence irrelevant.

The key is the invariant set. According to Palmer's hypothesis, the invariant set contains all the physically realistic states of the universe. So any state that isn't part of the invariant set cannot physically exist.

Suppose you perform the Kochen-Specker thought experiment and measure the position of an electron. Then you ask what you would have found if you repeated the experiment, only this time measuring the electron's velocity instead.

According to Palmer, when you repeat the experiment you are testing a hypothetical universe that is identical to the real one except that the position-measuring equipment is replaced with velocity-measuring equipment.

This is where the fractal nature of the invariant set matters. Consider a place of interest you want to visit along a coastline. If you get the coordinates even slightly wrong you could end up in the sea rather than where you want to be. In the same way, if the hypothetical universe does not lie on the fractal, then that universe is not in the invariant set and so it cannot physically exist.

Due to the spare and wispy nature of fractals, even subtle changes in the hypothetical universes could cause them to fall outside the invariant set. In this way, Spekkens says, Palmer's hypothesis may help to make some sense of quantum contextuality.

"I think his approach is really interesting and novel," says Spekkens. "Other physicists have shown how you can find a way out of the Kochen-Specker problem, but this work actually provides a mechanism to explain the theorem."

Following on from this, Palmer believes that many other features of quantum theory also fall into place. For example, quantum theory is famous for making only statistical predictions - it can only tell you the probability of finding an electron with its quantum-mechanical spin pointing up.

This arises naturally, suggests Palmer, because quantum theory is blind to the intricate fractal structure of the invariant set. Just as our eyes cannot discern the smallest details in fractal patterns, quantum theory only sees "coarse grain approximations", as if it is looking through fuzzy spectacles.

Other physicists seem inspired by the novelty of Palmer's approach. "What makes this really interesting is that it gets away from the usual debates over multiple universes and hidden variables and so on," says Bob Coecke, a physicist at the University of Oxford. "It suggests there might be an underlying physical geometry that physics has just missed, which is radical and very positive."

Coecke points out that very few scientists working on fundamental physics have explored how fractals might be incorporated into the theory, even though they are commonplace in other parts of physics.

Palmer is hoping that will change. In a paper submitted to the journal Proceedings of the Royal Society A, he shows how the basic idea can account for quantum uncertainty, contextuality and other quantum puzzles (www.arxiv.org/abs/0812.1148).

Many details still need to be fleshed out, says Coecke. "Palmer manages to explain some quantum phenomena," he says, "but he hasn't yet derived the whole rigid structure of the theory. This is really necessary."

Palmer accepts the criticism and is hopeful that he will be able to improve his theory over time. In the best of worlds, he thinks his framework may provide a way to finally reunite the warring parties of Einstein's and Bohr's followers.

After all, the theory backs Einstein's view that quantum theory really is incomplete. It is, Palmer says, blind to the fractal structure of the invariant set. If it wasn't, it would see that the world is not only deterministic, but it never exhibits any spooky effects.

On the other hand, it also agrees with the view of Bohr and his followers: the properties of individual quantum systems are not independent of the entire world, especially the experiments we humans use to explore them. We are stuck with the disturbing fact that how we measure always influences what we find.

For now, quantum theory remains mysterious but its air of mystique may not last forever.

Mark Buchanan is a writer based in Cambridge, UK

---