Institute for Computational and Experimental Research in Mathematics (ICERM)

Abstract

All engineering companies want to build high-quality products as efficiently and quickly as possible. Physics-based simulation or CAE can provide answers virtually and faster without building prototypes or performing tests. However, in a recent survey among 163 engineering leaders conducted by the market research institute Forrester*, 55% of leaders said that they found CAE too inaccurate. Despite years of continuous progress on CAE methods, there are still many problems for which solving PDEs does not provide accurate insights – either because the underlying physics is too complex, or the boundary conditions are insufficiently known. To give an example, calibrating a P2D model for battery simulation requires so much data of a real battery cell prototype, that the battery must already have been manufactured and have undergone a large number of tests before the model becomes useful. In a case like this, where the physics is intractable, data-driven simulation can be a faster and easier way to model and optimise system performs. Intractable physics problems are everywhere in engineering, from materials science to dynamic system behaviour. This talk will look at various industrial case studies in which data-driven modelling was used to accelerate product development, including the Porsche Jota race team who used data-driven models for tire degradation and won Le Mans, Honeywell who found that data-driven methods can beat CFD in ultra-sound flow measurement, Kautex Textron who developed a more accurate method to model fuel sloshing noise, as well as BMW group who managed to predict the result of crash tests without CAE. The talk will end by focusing on battery simulation. In the author’s opinion, this is the most impactful area of engineering where machine learning can help model intractable physics, as the electrochemical relationships of batteries are notoriously hard to model and require a huge of amount of time-consuming testing. *Commissioned by Monolith

• Video

Abstract

High-fidelity simulations, crucial in science and engineering, are traditionally time-consuming and computationally demanding, hindering rapid iteration in various applications like design analysis and optimization. NVIDIA Modulus, a cutting-edge scientific machine learning platform, revolutionizes these processes by creating surrogate models that significantly outpace conventional methods in speed, maintaining high simulation accuracy. NVIDIA Modulus's surrogate models cater to diverse applications, such as weather prediction, power plant modeling, cardiovascular simulation, additive manufacturing, and aerodynamic evaluation. Modulus is accessible as open-source software under the Apache 2.0 license, fostering the expansion of the scientific machine learning community.

Abstract

Akselos provides Digital Twins of industrial equipment in a range of industries, such energy, renewables, mining, and aerospace. The Akselos platform is based on RB-FEA, which is a unique combination of the Reduced Basis method for fast reduced order modeling of parametrized PDEs, with a domain decomposition framework that enables large-scale component-based analysis. RB-FEA shares many similarities to supervised learning approaches, in which "full order" solutions are used as the "supervisor" during RB-FEA training. In this presentation we will discuss the similarities and differences between RB-FEA and other ML methods, and demonstrate applications of the methodology to industrial Digital Twins.

• Video

Abstract

In this talk, I will introduce a semi-supervised Machine Learning (ML) approach to model PDEs across a variety of engineering applications. The local, low-dimensional ML-Solver solves PDEs on multi-resolution meshes which are significantly coarser than the computational meshes used in typical engineering simulations. A coarse ML-Solver element, known as a subdomain, is representative of hundreds or even thousands of computational elements. The PDE solutions as well as other PDE conditions such as geometry, source terms and BCs are represented by n-dimensional latent vectors on each subdomain. The transformations between latent vectors and solution or condition fields on computational elements within a subdomain are learnt using field neural networks. Additionally, spatial, and temporal flux relationships between neighboring subdomains are learnt in the latent space using flux conservation neural networks and time integration neural networks. All the NNs are trained using data generated from engineering simulations. This talk will delve further into the nuts and bolts of the ML-Solver and demonstrate it across a variety of engineering use cases from fluids, thermal and mechanical applications. Finally, this talk will also demonstrate a use case of the ML-Solver that shows the potential of development towards a foundation model which can be used across a wide range of applications with consistent physics.

• Video

Abstract

From reduced order models to PINNs and neural operators, the exponential growth of neural network techniques has shifted industry toward a new paradigm: scientific machine learning (SciML). However, industrial use cases are becoming increasingly complex while demanding faster, near real-time turnaround, necessitating the adaption of SciML to drastically more difficult problems. At Siemens Technology, we are researching the latest machine learning advances to develop solutions for our businesses and products. In this talk, we present an overview of interesting applications of SciML for Siemens, and our recent work on accelerating digital twins for a variety of use cases, including inverse heat exchanger control, quasi real-time prediction of airbag deployment, and neural design optimization in large design spaces. Through these examples, we also share some challenges in applying SciML to industrial applications.

• Video

Abstract

This talk will begin with a brief snapshot of two case studies of the use of SciML in an industrial context, followed by a vision for computational decision-support and digital twins. Some recent developments in SciML will be reviewed, and an attempt will be made at providing a structured perspective on the current, somewhat chaotic landscape of modeling approaches. Following this, I will discuss various demands placed on models in scalable industrial applications, and opportunities for integration of structure within machine learning models will be highlighted. Particularly, I will touch upon two aspects: a) Introduction of physics constraints and adaptivity in reduced order models to greatly enhance predictive accuracy in complex problems, and b) Use of conditional parameterization to enforce mathematical structure and to sensitize models to secondary inputs.

• Video

Abstract

In this talk, we discuss the development of physically-constrained machine learning (ML) models that incorporate techniques from scientific computing for learning dynamical and physical systems with applications in epidemiology and fluid dynamics. We first study the lack of generalization of black-box deep learning models for ODEs with applications to COVID-19 forecasting and the need for incorporation of advanced numerical integration schemes. We then focus on learning a physical model that satisfies conservation laws which are ubiquitous in science and engineering problems ranging from heat transfer to fluid flow. Violation of these well-known physical laws can lead to nonphysical solutions. To address this issue, we propose a framework, which constrains a pre-trained black-box ML model to satisfy conservation by enforcing the integral form from finite volume methods. We provide a detailed analysis of our method on learning with the Generalized Porous Medium Equation (GPME), a widely-applicable parameterized family of PDEs that illustrates the qualitative properties of both easier and harder PDEs. Our model maintains probabilistic uncertainty quantification (UQ), and deals well with shocks and heteroscedasticities. As a result, it achieves superior predictive performance on downstream tasks, e.g., shock location detection. Lastly, we study how to hard-constrain Neural Operator solutions to PDEs to satisfy the physical constraint of boundary conditions on a wide range of problems including Burgers’ and the Navier-Stokes’ equations. Our model improves the accuracy at the boundary and better guides the learning process on the interior of the domain. In summary, we demonstrate that carefully and properly enforcing physical constraints using techniques from numerical analysis results in better model accuracy and generalization in scientific applications.

• Video

Abstract

Predicting real-time complex dynamics in physical systems governed by partial differential equations (PDEs) is challenging due to the computational expense of conventional numerical simulations. Neural operators offer a promising solution by approximating mappings between infinite-dimensional Banach spaces. However, their performance diminishes with increasing system size and complexity when trained on labeled observational data from high-fidelity simulators. In this presentation, I introduce a novel method for learning neural operators in latent spaces, enabling real-time predictions for systems with highly nonlinear and multiscale dynamics on high-dimensional domains. Leveraging the DeepONet architecture, the approach operates on a low-dimensional latent space encoding the dynamics of interest to efficiently and accurately approximate underlying operators. The efficacy of this approach is demonstrated across various real-world problems such as material fracture, stiff chemical kinetics, and global-scale climate modeling. Our method surpasses state-of-the-art techniques in prediction accuracy while significantly enhancing computational efficiency, thus enabling operator learning on previously infeasible scales.

• Video
• Slides

Abstract

Simulations and physical experiments are often resource intensive, and may take days or weeks to obtain estimates of a particular design under consideration. Bayesian optimization is a popular approach to solving such problems. I will discuss recent advances in Bayesian optimization and open source software developed at Meta to solve complex scientific and engineering problems, including methods for high-dimensional optimization, combinatorial, multi and many-objective optimization, and targeting long-term outcomes with short-term experiments. I will present these methods in the context of applications to inverse design of optical components, concrete formulations, and machine learning systems.

• Video

Abstract

The maintenance and sustainability profile of machines in industrial applications is of critical importance to their field performance. In many instances machine failure modes are dominated by physics and chemistry-based processes such as fatigue, wear, and corrosion. Moreover, these processes typically have measured data for analytics algorithms to predict these failure events. Physics informed machine learning (PIML) techniques seem ideal for these types of problems given that physics can compensate for the lack of data and the fact the physical-chemical processes dominate these failure mechanics. In this paper, we discuss the development of an engine PIML thermo-mechanical (TMF) PIML model and its corresponding validation with test and field data. This paper also discusses extension of these models to wear and oil health applications.

Abstract

Advances in Scientific Machine Learning (SciML) are poised to have profound impact within engineering domains that deal with industrial control and process systems (ICPS). Such systems are characterized by the operation of interconnected physical equipment; complex control and data acquisition systems; and sensor data that is nonlinear, noisy, and multi-dimensional. Consequently, maintaining reliable and predictable operation of an ICPS is challenging and often relies on robust advisory systems and expert operator decisions. SciML methods thus offer unique opportunities to develop new emulation and analysis tools that can ultimately enhance and redefine ICPS operation and decision making activities. This talk presents our SciML developments and experiences in using data-driven surrogates and causal discovery methods within the ICPS paradigm. We first discuss perspectives on the use of dynamic surrogate architectures to develop real-time ICPS emulators and how they facilitate effective simulation life-cycle development versus traditional emulator approaches. We further demonstrate how dynamic surrogate models naturally enable state-of-the-art numerical optimization and model predictive control capabilities. Next, we present challenges and opportunities toward utilizing causal discovery methods in the context of monitoring and fault analysis. We discuss how causal discovery methods provide a natural framework to both assist operators in understanding complex system behavior and to help diagnose root causes of faults without requiring a pre-existing historian of faulty operating conditions. We lastly use a benchmark chemical process dataset (the Tennessee Eastman Process) to demonstrate our results.

• Video
• Slides

Abstract

Modern machine learning has shown remarkable promise in multiple applications. However, brute force use of neural networks, even when they have huge numbers of trainable parameters, can fail to provide highly accurate predictions for problems in the physical sciences. We present a collection of ideas about how enforcing physics, exploiting multifidelity knowledge and the kernel representation of neural networks can lead to significant increase in efficiency and/or accuracy. Various examples are used to illustrate the ideas.

• Video

Abstract

Physical simulations are performed at multiple scales covering diverse physical domains. Macro scale continuum simulations at part and assembly level, usually employing traditional numerical techniques such as 3D Finite Elements or Finite Volumes, are widely used for physics-based product design because of their high predictiveness value. However, more often than not these simulations are compute-intensive and require minutes/hours/days to execute. Starting from these computationally expensive 3D models, surrogate models massively reduce execution times to seconds (or interactive times), allowing a high number of parameter evaluations in the design space. However, these surrogate models are often of lower fidelity and only provide limited information through a few scaler KPIs (e.g., max stress, max pressure, max intrusion, etc.). To retain the richness of 3D simulation results while significantly reducing executing time, this paper presents a Neural Networks-based approach with the ultimate aim to enable 3D quasi-interactive design exploration . Multiphysics-multiscale traditional simulations [1] are used as the starting point: a) FEA analyses of structural statics, dynamics, manufacturing, packaging and safety, and b) CFD analyses– are used as Design of Experiments (DOEs) to generate the parametric design data. The data is processed and used to train fast executing neural networks as 3D Surrogates. Neural network algorithms and architectures (deep feed forward networks, recurrent and recursive nets, etc.) [2] are chosen depending on the nature of the physics. The trained neural network models can be deployed in a collaborative design environment for interactive design explorations. The proposed approach extends traditional model surrogates to cover both transient physical responses and 3D fields, which could enable an information rich much more productive environment for product design.

Abstract

A goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth-independence and autonomy of mission operations. Here we present an overview of AI/ML architecture support deep space mission goals, developed with leaders in the field. First, we focus on the fundamental biological research that supports our understanding of physiological responses to spaceflight, and we describe current efforts to support AI/ML research including data standardization and data engineering through FAIR (findable, accessible, interoperable, reusable) databases and the generation of AI-ready datasets for reuse and analysis. We also discuss robust remote data management frameworks for research data as well as environmental and health data that are generated during deep space missions. We highlight several research projects that leverage data standardization and management for fundamental biological discovery to uncover the complex effects of space travel on living systems. Next, we provide an overview of cutting-edge AI/ML approaches that can be integrated to support remote monitoring and analysis during deep space missions, including generative models and large language models to learn the underlying biomedical patterns and predict outcomes or answer questions during offworld medical scenarios. We also describe current AI/ML methods to support this research and monitoring through automated cloud-based labs which enable limited human intervention and closed-loop experimentation in remote settings. These labs could support mission autonomy by analyzing environmental data streams, and would be facilitated through in situ analytics capabilities to avoid sending large raw data files through low bandwidth communications. Finally, we describe a solution for integrated, real-time mission biomonitoring across hierarchical levels from continuous environmental monitoring, to wearables and point-of-care devices, to molecular and physiological monitoring. We introduce a precision space health system that will ensure that the future of space health is predictive, preventative, participatory and personalized.

• Video

Abstract

Environmental monitoring – traditionally relied on collecting point samples – are undergoing transformational changes with new technologies such as remote sensing, in situ sensors and various imaging techniques at different scales. At the same time, environmental simulation capabilities are advancing rapidly, predicting environmental flow and contaminant transport in complex systems and quantifying its uncertainty. However, there are still significant challenges to integrate these multi-type multiscale datasets with model simulations. In particular, these datasets are often indirectly correlated with the variables of interest, and have different scales and accuracies. Simulation results are often not perfect due to natural heterogeneities or fine-scale processes not captured in conceptual models. The Advanced Long-term Environmental Monitoring Systems (ALTEMIS) project aims to establish the new paradigm of long-term monitoring of soil and groundwater by integrating these new technologies through machine learning (ML). This talk highlights the two new developments, involving groundwater flow and contaminant transport simulations. First, I will talk about an emulator based on the Fourier neural operator, considering the uncertainty in subsurface parameters and climate forcing. This emulator aims to enable the off-line assessment of future climate change impacts on residual contaminants. Second, I will introduce a Bayesian hierarchical approach coupled with Gaussian process models to integrate in situ sensor data, groundwater sampling data and ensemble simulations. It enables us to infuse physics –such as flow direction and contaminant mobility – into the spatiotemporal characterization of contaminant plumes. Lastly, I will discuss the pathway to actual deployment with the understanding of environmental regulations and site needs, as well as the feedback mechanism to advance deployable technologies

Abstract

Large Language Models (LLMs) have rapidly emerged as transformative tools in the landscape of scientific inquiry, heralding a new era of interdisciplinary innovation. Our presentation at MIT will embark on a journey through the evolution and significance of LLMs, starting with a brief overview to lay the foundational understanding necessary for appreciating their impact. We will then navigate towards the specific application of LLMs within the realm of Scientific Machine Learning (SciML), illuminating how machine learning techniques are revolutionizing scientific research by examining the practical approaches and methodologies adopted by researchers to address complex SciML challenges. A focal point of our discussion will be the role of simulations in Mechanical Engineering, where we will showcase examples and scenarios that highlight the superior solutions enabled by machine learning. This exploration will not only provide a comprehensive insight into the current state of play but also set the stage for considering LLMs as dynamic agents of change, surveying the latest and greatest research articles employing those agents. We will then try to break the ceiling of 'agent'/'assistant' by discussing General AI (GenAI) and its burgeoning potential to further empower SciML applications directly, and not as orchestrators. Concluding our session, we will venture into speculative territory, pondering the future landscape of the industry as it is shaped by the guiding hand of LLMs in steering research and development towards uncharted territories of knowledge and innovation.

• Video Available Soon