While reading the paper “Learning Nonlinear Operators via DeepONet Based on the Universal Approximation Theorem of Operators” (Lu et al., Nature Machine Intelligence, 2021), I gained a preliminary understanding of DeepONet. Below is a summary of my learning process and reflections.
What is DeepONet?
DeepONet (Deep Operator Network) is a neural network architecture based on the universal approximation theorem for operators. It is designed to learn nonlinear continuous operators — mappings from input functions to output functions. It consists of two subnetworks: a Branch Network, which processes the discrete values of the input function at fixed sensor locations, and a Trunk Network, which processes the domain coordinates of the output function. The outputs of these two networks are combined through an inner product to approximate the target operator.
Theoretical Basis
The theoretical foundation of DeepONet comes from the general approximation theorem for operators proposed in 1995: a neural network with one hidden layer can approximate any nonlinear continuous operator (a continuous mapping from one function space to another) with arbitrary precision. For any continuous nonlinear operator G and any precision ε > 0, there exist positive integers p, m, n and real parameters such that, for all input functions u and output coordinates y, G(u)(y) can be approximated by a sum of p terms, each the product of a coefficient bₖ(u) depending on the input function and a basis function τₖ(y) associated with the output coordinates.
The summation over p terms represents a linear combination of basis functions — similar to the number of neurons in a hidden layer. A larger p generally provides stronger approximation capacity.
The Branch Network — Processing the Input Function
The branch network encodes the input function u into coefficients {b₁, b₂, …, bₚ}. The function u is sampled at m fixed sensor points, producing discrete input data; each coefficient is computed through a nonlinear activation (e.g. Tanh or Sigmoid) applied to a weighted sum of those sampled values, plus a bias term. This represents the “stacked” structure of the theoretical DeepONet.
The Trunk Network — Processing the Output Coordinates
The trunk network provides the basis functions with respect to the output coordinates y, each computed via a nonlinear activation applied to a weighted combination of the coordinates plus a bias term. These outputs combine with the branch coefficients through an inner product to produce the final operator output.
From Stacked to Non-Stacked DeepONet
The original, stacked form of DeepONet directly follows the theorem: each coefficient is computed by an independent branch sub-network. While mathematically sound, this structure is inefficient and parameter-heavy.
To improve efficiency, researchers introduced the non-stacked DeepONet, which merges all branch sub-networks into a single, deeper network that outputs the full coefficient vector at once. This enables parameter sharing, reduces redundancy, and improves generalization — achieving lower training error, better stability, and faster convergence while using fewer parameters. It has become the de facto standard in research and applications, transforming the branch network from a “committee” of independent voters into a “brain” that learns collaboratively.
Applications and Current Research
Most research has applied DeepONet to elliptic PDEs such as Poisson and Darcy flow equations. Its application to parabolic PDEs (time-dependent systems) remains relatively under-explored. Classical numerical methods — finite difference, finite element, and spectral methods — work well but face challenges for high-dimensional or parametric problems, complex geometries, and fast computation under varying conditions (the “curse of dimensionality”). DeepONet offers a new paradigm: it learns the entire solution operator across a family of equations, enabling rapid inference without re-solving each configuration.
At NEDMIL, my current work focuses on applying DeepONet to parabolic PDEs, assessing whether this framework can provide an efficient, generalizable solution for complex dynamic systems.
Summary
DeepONet is a neural network architecture designed to learn nonlinear continuous operators, bridging mathematical theory and deep learning. Its twin structure — the branch and trunk networks — makes it a powerful framework for learning mappings between functional spaces. The transition from the theoretical stacked form to the efficient non-stacked architecture marks its evolution from mathematical proof to practical implementation, positioning DeepONet as a cornerstone of modern scientific machine learning.
Reference: Lu, L., Jin, P., Pang, G., Zhang, Z., & Karniadakis, G. E. (2021). Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3), 218–229.
