Sequencing Initial Conditions in Physics-Informed Neural Networks


  • Saman Hooshyar Department of Computer Science, University of Illinois Chicago, Chicago, IL, 60607, USA
  • Arash Elahi Department of Chemical Engineering, University of Illinois Chicago, Chicago, IL, 60607, USA



Scientific machine learning PINN, soft-regularization, multiphysics modeling, chemical engineering PDEs


The scientific machine learning (SciML) field has introduced a new class of models called physics-informed neural networks (PINNs). These models incorporate domain-specific knowledge as soft constraints on a loss function and use machine learning techniques to train the model. Although PINN models have shown promising results for simple problems, they are prone to failure when moderate level of complexities are added to the problems. We demonstrate that the existing baseline models, in particular PINN and evolutionary sampling (Evo), are unable to capture the solution to differential equations with convection, reaction, and diffusion operators when the imposed initial condition is non-trivial. We then propose a promising solution to address these types of failure modes. This approach involves coupling Curriculum learning with the baseline models, where the network first trains on PDEs with simple initial conditions and is progressively exposed to more complex initial conditions. Our results show that we can reduce the error by 1 – 2 orders of magnitude with our proposed method compared to regular PINN and Evo.


Acharya N. et al. Instability analysis of poiseuille flow between two parallel walls partially obstructed by porous surfaces. In APS Division of Fluid Dynamics Meeting Abstracts, 2019, pp. NP05-140.

Ainsworth M., Dong J., 2021. Galerkin neural networks: a framework for approximating variational equations with error control. SIAM J. Sci. Comput., 43, A2474-A2501.

Aygün A. Physics informed neural networks for computational fluid dynamics. Middle East Technical University, Master's thesis, 2023.

Batra R. et al. Emerging Materials Intelligence Ecosystems Propelled by Machine Learning. Nature Reviews. Materials, 2021, 6(8), 655-678.

Beck C., Hutzenthaler M., Jentzen A., Kuckuck B.J., 2020. An overview on deep learning-based approximation methods for partial differential equations.

Bengio Y. et al. Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 41-48.

Bharadwaja B.V.S.S. et al. Physics-Informed Machine Learning and Uncertainty Quantification for Mechanics of Heterogeneous Materials. Integrating Materials and Manufacturing Innovation, 2022, 11(4), 607-627.

Cai S. et al. Physics-informed neural networks (PINNs) for fluid mechanics: a review. Acta Mechanica Sinica/Lixue Xuebao, 2021, 37(12), p. 1727-1738.

Cai S., Wang Z., Wang S., Perdikaris P., Karniadakis G. E., 2021. "Physics-Informed Neural Networks for Heat Transfer Problems." ASME. J. Heat Transfer, June 2021; 143(6): 060801.

Chen Y. et al. Physics-informed neural networks for inverse problems in nano-optics and metamaterials. Optics Express, 2020, 28(8), 11618-11633.

Chervonyi Y. et al. Semi-analytical industrial cooling system model for reinforcement learning. arXiv preprint arXiv:2207.13131, 2022.

Cooley M. et al. Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases. OpenReview, 2023,

Daw A. et al. Mitigating propagation failures in PINNs using evolutionary sampling. arXiv preprint, 2022, arXiv:.

Dekhovich A. et al. iPINNs: Incremental learning for Physics-informed neural networks. arXiv preprint arXiv:2304.04854, 2023.

Droniou J., Le N., 2020. The gradient discretization method for slow and fast diffusion porous media equations. SIAM Journal on Numerical Analysis, 58(3), pp: 1965-1992.

Du H. et al. Modeling density-driven flow in porous media by physics-informed neural networks for CO2 sequestration. Computers and Geotechnics, 2023, 159, 105433.

Du H., Zhao Z., Cheng H. Modeling Density-Driven Flow in Porous Media by Physics-Informed Neural Networks for CO2 Sequestration. Computers and Geotechnics, 2023, 159, 105433.

Edwards C. Neural networks learn to speed up simulations. Communications of the ACM, 2022, 65(5), 27-29.

Elahi A. et al. Temperature-Transferable Coarse-Grained Model for Poly(propylene oxide) to Study Thermo-Responsive Behavior of Triblock Copolymers. J. Phys. Chem. B, 2022, 126, p. 292-307.

Elahi A., Chaudhuri S. Computational Fluid Dynamics Modeling of the Filtration of 2D Materials Using Hollow Fiber Membranes. ChemEngineering, 2023, 7, 108.

Geneva N., Zabaras N. Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks. Journal of Computational Physics, 2020, 403, 109056.

Guo J. et al. Pre-training strategy for solving evolution equations based on physics-informed neural networks. arXiv preprint, 2022, arXiv:2212.00798.

Gusmão G. S., Medford A. J., 2023. Maximum-likelihood Estimators in Physics-Informed Neural Networks for High-dimensional Inverse Problems. arXiv preprint arXiv:2304.05991.

Hooshyar S. et al. Energy budget analysis of plane poiseuille-couette flow over a permeable surface. Bulletin of the American Physical Society, 2020.

Hooshyar S. et al. The impact of imposed Couette flow on the stability of pressure-driven flows over porous surfaces. Journal of Engineering Mathematics, 2022, 132(1).

Hussaini M., Zang T.A., 1987. Spectral methods in fluid dynamics. Annual Review of Fluid Mechanics, 19(1), pp: 339-367.

Jagtap A.D. et al. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 2020, 365, 113028.

Jin X. et al. NSFNNets (Navier-Stokes Flow Nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. Journal of Computational Physics, 2021, 426, 109951.

Karniadakis G.E., Kevrekidis I.G., Lu L., Perdikaris P., Wang S., Yang L., 2021. Physics-informed machine learning. Nature Reviews Physics, pp: 422-440.

Krishnapriyan A. et al. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 2021, 34, 26548-26560.

Mao Z. et al. Physics-informed neural networks for high-speed flows. Computer Methods in Applied Mechanics and Engineering, 2020, 360, 112789.

Mishra S., Molinaro R. Physics informed neural networks for simulating radiative transfer. Journal of Quantitative Spectroscopy and Radiative Transfer, 2021, 270, 107705.

Münzer M., Bard C. A Curriculum-Training-Based Strategy for Distributing Collocation Points during Physics-Informed Neural Network Training. arXiv preprint, 2022, arXiv:2211.11396.

Ngo S.I., Lim Y. Forward Physics-Informed Neural Networks Suitable for Multiple Operating Conditions of Catalytic CO2 Methanation Isothermal Fixed-Bed. IFAC-PapersOnLine, 2022, 55(7), 429-434.

Ngo S.I., Lim Y. Modeling density-driven flow in porous media by physics-informed neural networks for CO2 sequestration. Computer Aided Chemical Engineering, 2022, pp. 1675-1680.

Ngo S.I., Lim Y. Solution and Parameter Identification of a Fixed-Bed Reactor Model for Catalytic CO2 Methanation using Physics-Informed Neural Networks. Catalysts, 2021, 11(11), 1304.

Pang G. et al. fPINNs: Fractional Physics-Informed Neural Networks. SIAM Journal on Scientific Computing, 2019, 41(4), A2603-A2626.

Raissi M., Perdikaris P., Karniadakis G.E., 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, pp: 686-707.

Sak H. et al. Modeling dependence dynamics of air pollution: Pollution risk simulation and prediction of PM2.5 levels. arXiv preprint, 2016, arXiv:1602.05349.

Shi S., Liu D., Huo Z. Simulation of flow field in silicon single-crystal growth using physics-informed neural network with spatial information. Physics of Fluids, 2022, 34(11), 113610.

Shiratori S. et al. Spatio-temporal thickness variation and transient Marangoni number in striations during spin coating. International Journal of Heat and Mass Transfer, 2020, 154, 119678.

Shiratori S., Kubokawa T. Double-peaked edge-bead in drying film of solvent-resin mixtures. Physics of Fluids, 2015, 27, 102105.

Stegmeir A., Ross A., Body T., Francisquez M., Zholobenko W., Coster D., Kang S., 2019. Global turbulence simulations of the tokamak edge region with GRILLIX. Physics of Plasmas, 26(5), 052517.

Subramanian S. et al. Adaptive Self-supervision Algorithms for Physics-informed Neural Networks. arXiv preprint, 2022, arXiv:2207.04084.

Subramanian S. et al. Adaptive Self-supervision Algorithms for Physics-informed Neural Networks. arXiv preprint, 2022, arXiv:2207.04084.

Wang S. et al. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint, 2022, arXiv:2203.07404.

Wang S., Teng Y., Perdikaris P. Understanding and mitigating gradient pathologies in physics-informed neural networks. arXiv preprint, 2020, arXiv:2001.04536.

Yin M. et al. Non-invasive inference of thrombus material properties with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 2021, 375, 113603.

Zhu Q., Liu Z., Yan J. Machine Learning for Metal Additive Manufacturing: Predicting Temperature and Melt Pool Fluid Dynamics using Physics-Informed Neural Networks. Computational Mechanics, 2021, 67(2), 619-635.

Zienkiewicz O., Taylor R., Zhu J., 2005. The finite element method: its basis and fundamentals. Elsevier.



DOI: 10.56946/jce.v3i1.345

How to Cite

Hooshyar, S., & Elahi, A. (2024). Sequencing Initial Conditions in Physics-Informed Neural Networks. Journal of Chemistry and Environment, 3(1), 98–108.