I recently read a Machine Learning (ML) paper titled, Wave Physics as an Analog Recurrent Neural Network (arXiv). The meat of the paper might be tedious for anyone who is not interested in the topic of ML, so I will skip it. Instead, I will provide an intuitive explanation of the result that is demonstrated in the paper.
To facilitate my explanation, consider the following images:
These images are the result of a wave-physics simulation run by the authors of the paper. Here's a rough description of what the images show. Imagine we are looking at a top-down view of a sound-proof room. In this sound-proof room are a speaker (the bright spot on the left-hand side of each image) and three microphones (the three dots on the right-hand side of each image). Between the speaker and the microphones is a porous membrane -- think of it like plastic, foam or some other solid that has been laser-cut into a very specific pattern (the contours traced by the white lines in the center of each image). The sound waves cannot pass directly from the speaker to the microphones but must, instead, reflect and reverberate off the complex interior surfaces of the membrane between the speaker and the microphones. By construction, the microphones do not need to be able to record a full spectrum of audio, they only need to be able to detect the presence or absence of sound power.
The speaker can play one of three vowel sounds -- the ay in "cape", the ee in "keep" or the a in "cap". The shape of the membrane has been learned by a novel type of neural network called a "neural ordinary differential equation" or "neural ODE". The NN has learned this shape by being trained with many different examples of the ay, ee and a vowels, respectively. The goal of the NN is to learn a membrane shape that will "steer" the sound power from the speaker to a specific microphone for a specific vowel. The simulations above show that the same membrane shape can steer many examples of ay, ee and a vowels that the NN never heard during training to the correct microphone.
This result is remarkable for a few reasons. First, the membrane is completely passive, that is, the membrane does not require any power to perform its job of correctly classifying vowels. The sound power is provided by the input itself (the sound waves from the speaker). Second, the membrane performs the classification task via simple wave propagation (as described by the classical wave equation). Third, the system is analog (continuous). This is in contrast to most neural networks used in machine learning today which are digital (discrete).
This result shows that wave physics -- a completely continuous domain -- is capable of performing machine learning tasks typically performed by discrete processors. While the example simulation performed in the paper is relatively simple, the techniques used can be generalized to a significant percentage of ML systems -- recurrent neural networks (RNNs) are one of the most common neural network architectures. The practical implications of this paper are that (a) we may be able to save enormous amounts of power in machine learning systems (factors of a million or more, depending on the application) and (b) we may be able to speed up complex tasks that incur significant processing delay in digital systems since a wave-physics system evolves at the speed of whatever wave is propagating through it (the speed of sound, speed of light, etc.) In other words, the processing time for a wave-physics system is just the time it requires for the input wave to travel through the system from input to output.
At this point, I am going to part ways with machine learning and the above paper. What is most remarkable to me about this paper's result is not discussed in the paper itself. This is my thesis: that the simulation performed in the paper above demonstrates a microcosm of physical wave-particle duality. On the left-hand side is the continuous (wave) domain. On the right-hand side is the discrete (particle) domain. (Note that I have slipped in two identities, here: (1) The identity between waves and continuous mathematics. (2) The identity between particles and discrete mathematics.)
It might not be obvious how this demonstrates the equivalence of waves and particles. Wave physics is time-reversible, that is, waves propagate according to exactly the same rules when moving forward or backward in time[1]. Given no other information, you cannot tell whether a movie depicting smooth wave phenomena is playing forward or backward. For this reason, we can see that if a sound impulse were to be produced at one of the places in the room where a microphone has been positioned, the sound that would come out on the left-hand side of the room would be something roughly like the vowel which corresponds to that microphone. Imagine striking a steel rod with a hammer at precisely the point where the microphone for the vowel ay is supposed to stand. Someone standing on the other side of the room - exactly where the speaker is supposed to stand, should hear the vowel ay, or something very close to it. The same goes for each of the other vowels. Since we can focus waves to discrete points (forward propagation through the membrane), and since we can produce waves from discrete points (backward propagation through the membrane), we can see that the shape of the membrane somehow "encodes" a duality, that is, an equivalence transform between these types of waves and a set of discrete points.
The existence of the equivalence I am pointing out is not a new insight. Information theory gives a unified framework for both continuous and discrete domains. There are well-defined transforms between the two domains. In digital-signal processing (DSP), these transforms are implemented as devices called ADCs (analog-to-digital converters) and DACs (digital-to-analog converters). But this new paper gives us a demonstration of an equivalence transform between continuous and discrete domains that can be built (or simulated) using only wave physics and which is entirely passive (requires no active power input). This makes the transform itself describable without having to posit two separate domains from the outset and without having to employ a clock to synchronize a sampling process. If we begin with waves-only physics, we can build a system in which there are waves and equivalent particles (which are discrete elements) in a very natural way. What makes it more "natural" than, say, ADCs and DACs is that it does not require sampling or discrete processing, and it does not require any input power.
To recap, my thesis is this: All discrete and continuous phenomena are related by some (possibly extremely complex) equivalence transform and this transform can, in principle, be "learned" by a sufficiently powerful neural net, perhaps implemented as a neural-ODE, as in the linked paper. This relation is sufficiently general that it includes physically discrete phenomena (such as particles) and physically continuous phenomena (such as waves). Thus, wave-particle duality is merely a byproduct of the existence of such equivalence transforms.
We might still wonder why a particular system is behaving in a wave-like or particle-like manner. How does Nature choose on which side of the membrane (equivalence transform) to stand? This question is very similar to the design problems faced by signal processing engineers. In some cases, a signal can be processed very power-efficiently and with high fidelity by a simple analog (continuous) circuit. In other cases, implementing a desired signal transform with analog circuitry would require a very complex, noisy circuit with poor fidelity and poor power-efficiency. In these cases, the signal-processing engineer will prefer to implement the transform in a digital signal processor (DSP).
It is not difficult to imagine that Nature is making a similar choice when choosing to maintain quantum state in either a continuous (analog) or discrete (digital) form. We observe that Nature is everywhere economical in her use of resources. So, she chooses that form which is most economical in respect to the well-known conservation principles (conservation of mass, energy, angular momentum, and so on). Like the design choices of a signal-processing engineer, Nature's choices are not arbitrary.
I find this this deep connection between information theory and Nature to be remarkable. In fact, there are many such connections and I hope to cover some of them in future posts. A consistently information-theoretic view of physics eliminates almost all of the paradoxes that arise from thinking about Nature using purely mechanical intuition and metaphors (for example, the billiard-ball model or wave-model). For example, the Heisenberg uncertainty principle has deep connections to time-frequency analysis and signal processing theory.
By thinking more deeply about this subject, I think we will find that endangered classical theories may eventually be preserved from extinction. Stephen Hawkings's work on black hole radiation, for example, has deep connections to information theory. Resolving the tension between the black-hole -- a "pure classical" object -- and quantum uncertainty may require modifications to quantum theory to limit the universality of quantum theory. Of course, theoretical work is already ongoing in this area (string theory, M-theory, topological quantum physics, etc.) But it is possible that the answer lies in going back to what came before, that is, in reinvigorating classical theories of physics.
[1] There is a caveat, here -- the waves must be mathematically "smooth", meaning, they cannot have discontinuities. Imagine a sheet waving gently in the breeze versus a sheet that has been ripped by gale force winds. The waves on the gently waving sheet are time-reversible, the waves on a sheet being ripped by forceful winds are not.
