Loading...
Organizing Committee
University of Illinois Urbana-Champaign
ETH Zürich
King Abdullah University of Science and Technology
RIKEN Center for Computational Science
Argonne National Laboratory
Flatiron Institute, Simons Foundation
Abstract
Demands of resolution and fidelity have driven the performance of simulations of first-principles mathematical models in science and engineering from MegaFlop/s to ExaFlop/s and their datasets from MegaBytes to ExaBytes (12 orders of magnitude) over the past four decades. Over the most recent decade, the application of machine learning to science and engineering systems has created a similar demand for performance and associated storage. Lower levels of the software stack created for simulation have proved immediately useful for machine learning at scale. However, higher levels of the simulation software have not yet fulfilled their potential to lift the dominant algorithms for machine learning and inference today above relatively “brute force” implementations, resulting in massive costs for facilities and energy that slow progress and restrict access to the research frontier for many.
At the same time, hardware optimized for machine learning applications possesses yet unrealized potential for traditional simulation. Increasingly, the same science or engineering application is more effectively addressable by simulation and learning working in tandem than by either alone, resulting in a confluence of two formerly distinct research communities. This “hot topics” weekend aims to capitalize by highlighting opportunities for cross-fertilization in both directions.
At the same time, hardware optimized for machine learning applications possesses yet unrealized potential for traditional simulation. Increasingly, the same science or engineering application is more effectively addressable by simulation and learning working in tandem than by either alone, resulting in a confluence of two formerly distinct research communities. This “hot topics” weekend aims to capitalize by highlighting opportunities for cross-fertilization in both directions.
Courtesy of the National Center for Supercomputing Applications (NCSA) and the Board of Trustees of the University of Illinois.
Loading...
Loading participant list in background...
Workshop Schedule
Saturday, September 13, 2025
8:30 - 8:50 AM EDT
Check In
11th Floor Collaborative Space
8:50 - 9:00 AM EDT
Welcome
11th Floor Lecture Hall
Brendan Hassett, ICERM/Brown University
9:00 - 9:45 AM EDT
From Large Language Models to Reasoning Language Models - Three Eras in The Age of Computation
11th Floor Lecture Hall
Speaker
Torsten Hoefler, ETH Zürich
Session Chair
David Keyes, King Abdullah University of Science and Technology
Abstract
We explore the fascinating evolution of Large Language Models (LLMs) and their transformative journey through the lenses of computation and optimization. We begin by tracing the origins of LLMs, highlighting how advances in computation and optimization were pivotal in their development. We then delve into the key optimizations that have achieved a staggering 1,000x cost reduction, making LLMs widely accessible even on portable devices. Moving forward, we address the limitations of human-generated data and introduce the concept of constructive hallucination in LLMs. This technique allows for the generation of new hypotheses and their validation through reasoning chains, pushing the boundaries of knowledge creation. Next, we provide an overview of the technology fundamentals and early successes of reasoning models, such as OpenAI's o1 and o3 preview. These models, while significantly enhancing computational capabilities, also exponentially increase computational demands. Finally, we conclude by presenting our ambitious Ultra Ethernet effort, which aims to establish the interconnect standard for future AI workloads. This initiative is crucial in meeting the growing demands at the system level, ensuring seamless and efficient operation in the age of reasoning models.
10:00 - 10:30 AM EDT
Coffee Break
11th Floor Collaborative Space
10:30 - 11:15 AM EDT
Supercomputer architectures, existing and imagined
11th Floor Lecture Hall
Speaker
Richard Vuduc, Georgia Institute of Technology
Session Chair
David Keyes, King Abdullah University of Science and Technology
Abstract
One point of intersection between Bill Gropp and me is the dear, late, and great Hormozd Gahvari, who was a classmate of mine at Berkeley before leaving to do his Ph.D. at UIUC under Bill. In 2010, Hormozd and Bill published a paper at IPDPS, one that I continually return to as I reflect on prior supercomputer architectures and wonder about their future. ("Wonder" here meaning in the admittedly cartoon-ish way an algorithmic person, as opposed to a systems person, does.) In this talk, I'll reflect on their work and trace how its ideas shaped my own research and views on the machines we have built, the ones we seem likely to build, and what we leave "on the table" when we pursue designs that favor one class of systems over others.
11:30 AM - 12:15 PM EDT
Building a Gaussian Process Statistical and Quantitative Learning Framework for Scientific Applications
11th Floor Lecture Hall
Speaker
Xiaoye Li, Lawrence Berkeley National Laboratory
Session Chair
David Keyes, King Abdullah University of Science and Technology
Abstract
High Performance Computing application codes often contain many tuning parameters and require extensive computational resources. It is desirable to have a machine leaning framework to automate the parameter optimization process with minimal code executions on the actual HPC systems. Another desideratum is a trustworthy digital twin for a physical system, by which we want to quantify the uncertainties of the simulation model with respect to the physical phenomena. For these optimization and uncertainty quantification purposes, a Bayesian statistical learning framework is a powerful tool that can treat the application as a black-box function and use Gaussian Process regression to compute the mean function and the variance in distribution. To this end, we have been developing a public domain software framework called GPTune. For parameter tuning we implemented a number of learning methods including multi-task learning, transfer learning, multi-objective tuning, and multi-fidelity tuning. For uncertainty quantification, we have been developing novel kernels to make them scientific-domain aware, fast linear algebra algorithms to enable large scale GP with millions of data points, and novel Gaussian Process to handle uncertainties in both input data and modeling. We illustrate the versatility of the GPTune software when it is applied to HPC codes ranging from mathematical libraries to complex simulation codes, as well as large-scale scientific apparatuses and instruments.
12:25 - 12:30 PM EDT
Group Photo (Immediately After Talk)
11th Floor Lecture Hall
12:30 - 2:30 PM EDT
Lunch/Free Time
2:30 - 3:15 PM EDT
Using Acceleration as a Service: Rethinking HPC Infrastrucutre in the AI Era
11th Floor Collaborative Space
Speaker
Thomas Schulthess, ETH Zürich
Session Chair
Torsten Hoefler, ETH Zürich
Abstract
Scientific computing has undergone a fundamental transformation over the past decade. GPU-accelerated computing, which in 2015 was still at the fringes of supercomputing, has become the norm today. The recent advances — indeed, the revolution — in AI, together with the broader digital transformation of science, have placed HPC at the very center of scientific discovery. No AI-for-Science project can succeed without it.
This tremendous opportunity, however, also poses a threat to HPC centers that fail to adapt to the needs of scientists. The success of AI in science depends not only on access to sufficient compute power but equally on data — and data requires curation and domain-centric workflows. Traditional HPC operational models are ill-suited for this.
In a recent publication [1], Hoefler et al. introduced the concept of Acceleration as a Service (XaaS), a step in the right direction that leverages container-based deployment of HPC workloads. In this presentation, I will go a step further and demonstrate how supercomputing infrastructures can evolve into service-oriented architectures. Drawing on principles from cloud computing and exploiting features of modern high-performance networks, we can simultaneously serve operational weather prediction, climate simulation, AI, traditional HPC, and large-scale experimental and observational instruments — all through specialized, elastic platforms that do not compromise scalability or performance.
[1] T. Hoefler et al., “XaaS: Acceleration as a Service,” arXiv preprint, arXiv:2401.04552, Jan. 2024.
3:30 - 4:00 PM EDT
Coffee Break
11th Floor Collaborative Space
4:00 - 4:45 PM EDT
Machine Learning & HPC Enabled Prediction in Tokamaks
11th Floor Lecture Hall
Speaker
William Tang, Princeton University
Session Chair
Torsten Hoefler, ETH Zürich
Abstract
We focus on the scientific and engineering advances being driven by HPC together with advanced statistical methods featuring artificial intelligence/deep learning/machine learning (AI/DL/ML). Accelerated progress in delivering scientific discovery via predictions with these methods guided by Validation, Verification & Uncertainty Quantification (VVUQ) are enabling data-driven discoveries essential for realizing the grand challenge potential of fusion energy. The best validated fusion devices are tokamaks, which are vulnerable to large scale plasma instabilities called “disruptions” that can stop the reaction and damage the device. Reliably predicting and avoiding these damaging events is essential, and recent advances feature successful capabilities involving the deployment of recurrent and convolutional neural networks in Princeton's Deep Learning Code – FRNN – for carrying out efficient transfer learning across very different tokamak devices. Essential validation vs. a huge data base is documented and highlights the applicability of FRNN to both the large JET and smaller DIIID. It is further shown that this AI/DL capability can provide not only the “disruption score,” as an indicator of the probability of an imminent disruption but also a “sensitivity score” in real-time to indicate the underlying reasons for the predicted event. Consequently, this deep learning capability provides the detailed information for a plasma control system (PCS) that can help improve disruption avoidance in real-time and to thereby optimize plasma performance. Application of such AI/DL methods for real-time prediction and control has been further advanced with the innovative introduction of a surrogate model simulator (SGTC). These capabilities are now leading to exciting avenues for moving from passive prediction to active control and ultimately, to the optimization of the design for a first-of-a-kind fusion pilot plant.
5:00 - 6:30 PM EDT
Reception
11th Floor Collaborative Space
Request Reimbursement
This section is for general purposes only and does not indicate that all attendees receive funding. Please refer to your personalized invitation to review your offer.
- ORCID iD
- As this program is funded by the National Science Foundation (NSF), ICERM is required to collect your ORCID iD if you are receiving funding to attend this program. Be sure to add your ORCID iD to your Cube profile as soon as possible to avoid delaying your reimbursement.
- Acceptable Costs
- 1 roundtrip between your home institute and ICERM
- Flights in economy class to either Providence airport (PVD) or Boston airport (BOS)
- Ground Transportation to and from airports and ICERM.
- Unacceptable Costs
- Flights on U.K. airlines
- Seats in economy plus, business class, or first class
- Change ticket fees of any kind
- Multi-use bus passes
- Meals or incidentals
- Advance Approval Required
- Personal car travel to ICERM from outside New England
- Multiple-destination plane ticket; does not include layovers to reach ICERM
- Arriving or departing from ICERM more than a day before or day after the program
- Multiple trips to ICERM
- Rental car to/from ICERM
- Arriving or departing from airport other than PVD/BOS or home institution’s local airport
- 2 one-way plane tickets to create a roundtrip (often purchased from Expedia, Orbitz, etc.)
- Travel Maximum Contributions
- New England: $350
- Other contiguous US: $850
- Asia & Oceania: $2,000
- All other locations: $1,500
- Note these rates were updated in Spring 2023 and supersede any prior invitation rates.
- Reimbursement Requests
Request Reimbursement with Cube
Refer to the back of your ID badge for more information. Checklists are available at the front desk and in the Reimbursement section of Cube.
- Reimbursement Tips
- Scanned original receipts are required for all expenses
- Airfare receipt must show full itinerary and payment
- ICERM does not offer per diem or meal reimbursement
- Allowable mileage is reimbursed at prevailing IRS Business Rate and trip documented via Google Maps PDF
- Keep all documentation until you receive your reimbursement!
- Reimbursement Timing
6–8 weeks after all documentation is sent to ICERM. All reimbursements are reviewed by multiple Brown University offices.
- Reimbursement Deadline
Submissions must be received within 30 days of ICERM departure to avoid applicable taxes. No submissions are accepted after six months.