Universal Differential Equations

Resources

• Rackauckas, Christopher, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, and Alan Edelman. 2021. “Universal Differential Equations for Scientific Machine Learning.” arXiv. http://arxiv.org/abs/2001.04385.
• Silvestri, Mattia, Federico Baldo, Eleonora Misino, and Michele Lombardi. 2023. “An Analysis of Universal Differential Equations for Data-Driven Discovery of Ordinary Differential Equations.” arXiv. https://doi.org/10.48550/arXiv.2306.10335.

Related

Main Idea

We can represent a general class of equations through

where $W(t)$ is a Weiner process and $\alpha (t)$ is a delay function (like $t-\tau$). This setup allows encoding partially known information into the problem structure, while $U_{\theta }$ remains as an unknown. Taking $U_{\theta }$ to just depend on $u$ as $N{N}_{\theta }(u)$ and $$\mathcal{N}[u(t),\cdot, \cdot, NN_\theta (u)] = u_t - NN_\theta (u),$$
we recover a Neural ODE.

Other choices of $\alpha (t)$ and $W(t)$ lead to Delay Differential Equations and Stochastic Differential Equations.

For Partial Differential Equations, we presumably need $u(x,t)$ and to allow for the operators depending on $u$ to take $u_x,u_{xx},$ and so on. A similar choice recovers Deep Hidden Physics Models within the framework of Partial Differential Equation Discovery.