1 Logistics

This is:

Earth and Planetary Sciences 226: Building a Machine-Learning Climate Emulator.

Instructor:

Eli Tziperman, office hours: please see the Canvas course web page.

TF:

TBA. Office Hours: Canvas course webpage.

Day, time & location:

Tuesday, Thursday, 10:30–11:45, Geological Museum, 24 Oxford St, third floor, room 375.

Section/HW help session:

time and location: see the Canvas course web page.

Requirements:

participation (20%), in-class teaching (30%), a final course presentation (30%), and course reports (20%). Each student will teach a class or more on the method covered by their team. The student lectures will cover a conceptual explanation of the method followed by a detailed technical explanation, a guided reading of the principal papers, an explanation of the needed code, and hands-on exercises for the other teams that will serve as both in-class and HW assignments. The lectures and suggested exercises offered by the members of each team will be coordinated among the students and with the instructor so that they cover the needed material.
$\bullet$ Deliverables and Milestones: (A) Student lectures: A 1st group lecture on the algorithm. 2nd group lecture a few weeks later: coding the algorithm. 3rd group lecture: preliminary results. Final presentation. (B) Reports: 1-page intermediate reports on characterizing ENSO in observations and CESM and on the results of training/validation. The final 2-page report.

Recommended preparation:

Python programming experience. Prior machine-learning coursework is not expected.

2 Course description

Machine-learning (ML) emulators can learn from observations or complex models and then simulate or predict a given system with exceptional efficiency. In this project-based graduate course, students will build an ML emulator of El Niño events by training on climate model simulations and gain experience lecturing about machine learning. The introductory lectures will introduce basic ENSO dynamics and some machine learning ideas, including supervised learning, neural networks, backpropagation, overfitting, validation, etc. Each student team will then attempt a different approach to building the emulator and will provide a few lectures to the class on its approach. These approaches will include convolutional encoder–decoder networks (U-Net), transformers, Fourier neural operators, and graph neural networks, all introduced as both deterministic and stochastic/generative emulation methods. Students are encouraged to also apply the emulation methods to their own research as an optional class assignment.

3 Detailed syllabus

1.

Machine learning basics: [downloads] I. Supervised learning and neural networks. Regression and classification, perceptrons, neural networks, loss functions (cross entropy for classification, $L^1$ or $L^2$ for regression); evaluation metrics: RMSE and MAE (Mean Absolute Error); overfitting and training/validation/test data splitting. II. Backpropagation, convolution, autoencoders, and emulators. Readings: APM120 class notes section 7.4.

2.

ENSO dynamics. [downloads] Observed ENSO characteristics, diversity (central versus eastern Pacific events, amplitude asymmetry, etc.), and impacts/teleconnections. ENSO dynamics: Bjerknes feedback, delayed and recharge oscillators. Observed diagnoses: NINO3.4 time series, PDF, power spectra, and composites. Readings: recharge oscillator (Jin, 1997), ENSO diversity (Capotondi, Wittenberg, et al., 2021), and course slides.

3.

CESM and training data. [downloads] Assessing ENSO simulation quality by comparing to observations; selecting variables for emulator training, including domain, and temporal and spatial resolution. Readings: on CESM (Danabasoglu et al., 2020) and its ENSO simulation (Capotondi, Deser, et al., 2020).

4.

Method A: U-Net. [downloads] Encoders, decoders, reconstruction-based autoencoder training, skip connections, and U-Net. Readings: an introduction to autoencoders (Learn AI, 2023), the U-Net architecture (Ito Aramendia, 2024), course slides, and U-Net applications to global ocean emulation with Samudra (a ConvNeXt U-Net, Dheeshjith et al., 2025) and seasonal Arctic sea-ice forecasting (Andersson et al., 2021).

5.

Method B: Transformers. [downloads] Transformers: tokens and features, learned positional and fixed time encoding, self-attention, multi-head attention, transformer blocks, and overall transformer architecture. Readings: an introduction to Transformers (Turner, 2026), spatiotemporal Transformers (Lee, 2021), and their application to ENSO (Zhou et al., 2023).

6.

Method C: Fourier Neural Operators (FNO). [downloads] An FNO layer: (i) lifting (encoding); (ii) operator layer: FFT-based global convolution and local bypass term; (iii) projection (decoding). Issues: level of Fourier truncation, non-periodic domain and zero padding, and an expected correlation between kept large scales and neglected small scales. Readings: an introduction to FNO (Hora et al., 2026) and the AI2 Climate Emulator (ACE, Watt-Meyer et al., 2023).

7.

Method D: graph neural networks. [downloads] Nodes, edges, and embedding physical variables into learned feature vectors. Input: matrices $𝖠$ ($N\times N$) with node connectivity information (e.g., grid point adjacency) and $𝖷$ ($N\times d$) with features at each node (e.g., temperature, thermocline depth, and wind at each grid point). Preprocessing layers, message-passing layers (learned exchange and aggregation of information between neighboring nodes), and postprocessing layers. Readings: an introduction to GNNs (Tanis et al., 2024) and their application to global weather forecasting with DeepMind’s GraphCast (Lam et al., 2023).

8.

Stochastic and generative emulators. [downloads] Loss of variability and blurred predictions under mean-squared-error training, introducing latent random input fields ($z,z^{\prime }$) and stochastic time stepping, Continuous Ranked Probability Score (CRPS)/energy score losses that balance prediction accuracy and ensemble dispersion, e.g., $\mathcal{L}=\frac{1}{2}(\parallel X_n(X_{n-1},z)-y\parallel +\parallel X_n(X_{n-1},z^{\prime })-y\parallel )-\frac{1}{2}\parallel X_n(X_{n-1},z)-X_n(X_{n-1},z^{\prime })\parallel$. Readings: an introduction to stochastic ensemble forecasting and CRPS training (Boettner, 2026) and an application to coupled climate emulation and ENSO variability (Wu et al., 2026).

9.

More: evaluation, transfer learning, extreme events, and intercomparison. [downloads] Evaluating the emulator outcome against validation/test data, both CESM and observations: Comparing the emulator’s NINO3.4 short-term ($\sim$6 months) prediction skill to persistence and a simple AR(1) or AR(2) Markov model. Comparing long-term statistical characteristics using spatial and temporal PCA patterns, NINO3.4 spectra, PDFs, and tail statistics, etc. Transfer learning: fine-tuning the weights for some layers (often later ones) or all layers using observational training data, possibly with a smaller learning rate. Extreme events: Can the combination of stochastic and generative training and transfer learning that trains on large El Niño events lead to the simulation of Super(!) El Niño events? Will their dynamics be credible? Intercomparison: diagnosing how and why different emulator methods disagree. Readings: Introduction to Transfer Learning (Torralba et al., 2024).

10.

Final presentations.

4 Development Guidance

Here are expected milestones for the emulator development.

1.

Characterize ENSO variability in the training data: Plot a Hovmoller of the SST, then plot the time series, spectrum, and PDF for the NINO3.4 index.

2.

Follow the code template to develop the deterministic emulator.

3.

Plot loss vs. epoch for the training and validation data sets; choose the best model.

4.

Test the model predictability for one initial month that is at a peak of the training NINO3.4 time series by running 4-year predictions. Plot the NINO3.4 time series of the emulator and training data on the same axes. You are looking for a version that shows a reasonable predictability skill for 6–12 months.

5.

Find and try at least two ways to change the model configuration to build a version with a fourth of your original number of parameters, and at least two others with four times the number of parameters. Show the training results for these cases. Keep experimenting with model size to find the model with the smallest number of parameters that performs satisfactorily.

6.

Using the emulator to learn about the physics:

(a)

Test the prediction as a function of month by starting 10 predictions at each month over 10 years, a total of 100 emulation runs per month. Calculate the correlation vs. prediction month averaged over all ensemble members and years. Plot predictability skill vs. month. Do you see a “spring time predictability barrier”?

(b)

What fields are required for a successful prediction? Try removing one of the three training fields at a time, and then two at a time to find the minimal input that is required to make a successful 6-month prediction of the NINO3.4 index. What do the results tell you about the physics of El Niño?

7.

Try to strengthen the prediction skill by predicting more than one month in advance, using all predicted months in the loss function.

8.

Develop the stochastic version of the emulator.

(a)

Find the optimal model and run multiple ensemble members, each for 30 years from the same initial conditions. Produce multiple NINO3.4 time series plots starting from the initial conditions. Do they look like the training data?

(b)

Characterize these longer-term emulations in the same way the training data were: Hovmoller contour plots, NINO3.4 spectrum, PDF, etc.

(c)

Consider adding a weak damping term to the emulator if the long-term stochastic emulation does not vary around zero anomalies.

(d)

If the characterization results are not satisfactory, consider training the stochastic emulator for predictions longer than one month.

9.

Time permitting: transfer learning using observations/ocean reanalysis.

Academic Integrity and Collaboration Policy: We strongly encourage you to discuss and work on homework problems with other students and with the teaching staff. However, after discussions with peers or consulting AI tools, you need to work through the problems yourself, and any answers you submit for evaluation should be the result of your own efforts, reflect your understanding, and be written in your words. In the case of assignments requiring programming, you need to be able to explain your code line by line. You must appropriately cite any books, articles, websites, lectures, AI tools, etc. used and explain how and why you used each such source.

Course materials are the property of the instructional staff or other copyright holders and are provided for your personal use. You may not distribute them or post them on websites without the permission of the course instructor.

References

Andersson, Tom R., J. Scott Hosking, María Pérez-Ortiz, et al. (2021). “Seasonal Arctic Sea Ice Forecasting with Probabilistic Deep Learning”. In: Nature Communications 12, p. 5124. doi: 10.1038/s41467-021-25257-4.

Boettner, Chris (June 2, 2026). From Deterministic Weather Forecasts To Ensembles. Blog post. url: https://chrisboettner.github.io/blog/posts/2026-06-02-aifs_crps_blog_post.html (visited on 09/12/2026).

Capotondi, Antonietta, Clara Deser, Adam S. Phillips, Yuko Okumura, and Sarah M. Larson (2020). “ENSO and Pacific Decadal Variability in the Community Earth System Model Version 2”. In: Journal of Advances in Modeling Earth Systems 12, e2019MS002022. doi: 10.1029/2019MS002022.

Capotondi, Antonietta, Andrew T Wittenberg, Jong-Seong Kug, Ken Takahashi, and Michael J McPhaden (2021). “ENSO diversity”. In: El Niño Southern Oscillation in a changing climate, pp. 65–86.

Danabasoglu, G., J.-F. Lamarque, J. Bacmeister, D. A. Bailey, A. K. DuVivier, J. Edwards, L. K. Emmons, J. Fasullo, R. Garcia, A. Gettelman, C. Hannay, M. M. Holland, W. G. Large, P. H. Lauritzen, D. M. Lawrence, J. T. M. Lenaerts, K. Lindsay, W. H. Lipscomb, M. J. Mills, R. Neale, K. W. Oleson, B. Otto-Bliesner, A. S. Phillips, W. Sacks, S. Tilmes, L. van Kampenhout, M. Vertenstein, A. Bertini, J. Dennis, C. Deser, C. Fischer, B. Fox-Kemper, J. E. Kay, D. Kinnison, P. J. Kushner, V. E. Larson, M. C. Long, S. Mickelson, J. K. Moore, E. Nienhouse, L. Polvani, P. J. Rasch, and W. G. Strand (2020). “The Community Earth System Model version 2 (CESM2)”. In: Journal of Advances in Modeling Earth Systems 12.2, e2019MS001916.

Dheeshjith, Surya, Adam Subel, Alistair Adcroft, Julius Busecke, Carlos Fernandez-Granda, Shubham Gupta, and Laure Zanna (2025). “Samudra: An AI Global Ocean Emulator for Climate”. In: Geophysical Research Letters 52.10, e2024GL114318. doi: 10.1029/2024GL114318.

Hora, Gurpreet Singh, Prakhar Kapoor, and Albert Matveev (Apr. 23, 2026). How a Fourier Neural Operator Learns to Solve PDEs—and Where It Falls Short. PhysicsX. url: https://www.physicsx.ai/newsroom/how-a-fourier-neural-operator-learns-to-solve-pdes----and-where-it-falls-short (visited on 09/04/2026).

Ito Aramendia, Alejandro (Feb. 1, 2024). The U-Net: A Complete Guide. 7 min read. Medium. url: https://medium.com/@alejandro.itoaramendia/decoding-the-u-net-a-complete-guide-810b1c6d56d8 (visited on 09/02/2026).

Jin, F.-F. (1997). “An Equatorial ocean recharge paradigm for ENSO. Part I: conceptual model”. In: Journal of the Atmospheric Sciences 54, pp. 811–829.

Lam, Remi, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia (2023). “Learning skillful medium-range global weather forecasting”. In: Science 382.6677, pp. 1416–1421. doi: 10.1126/science.adi2336. url: https://doi.org/10.1126/science.adi2336.

Learn AI (Mar. 25, 2023). Autoencoders. 12 min read. Medium. url: https://medium.com/@divakar1591/autoencoders-6fab1a9a9f9c (visited on 09/02/2026).

Lee, Wonseok (Oct. 12, 2021). TimeSformer: Is Space–Time Attention All You Need for Video Understanding? 13 min read. Lunit Team Blog on Medium. url: https://medium.com/lunit/timesformer-is-space-time-attention-all-you-need-for-video-understanding-5668e84162f4 (visited on 08/30/2026).

Tanis, James H., Chris Giannella, and Adrian V. Mariano (2024). Introduction to Graph Neural Networks: A Starting Point for Machine Learning Engineers. doi: 10.48550/arXiv.2412.19419. arXiv: 2412.19419v1 [cs.LG]. url: https://arxiv.org/abs/2412.19419v1.

Torralba, Antonio, Phillip Isola, and William T. Freeman (2024). “Transfer Learning and Adaptation”. In: Foundations of Computer Vision. MIT Press. Chap. 37. url: https://visionbook.mit.edu/transfer_learning.html.

Turner, Richard E. (2026). An Introduction to Transformers. doi: 10.48550/arXiv.2304.10557. arXiv: 2304.10557 [cs.LG]. url: https://arxiv.org/abs/2304.10557.

Watt-Meyer, Oliver, Gideon Dresdner, Jeremy McGibbon, Spencer K. Clark, James Duncan, Brian Henn, Matthew E. Peters, Noah D. Brenowitz, Karthik Kashinath, Michael S. Pritchard, Boris Bonev, and Christopher S. Bretherton (2023). “ACE: A Fast, Skillful Learned Global Atmospheric Model for Climate Prediction”. In: NeurIPS 2023 Workshop on Tackling Climate Change with Machine Learning. arXiv: 2310.02074 [physics.ao-ph]. url: https://www.climatechange.ai/papers/neurips2023/14.

Wu, Elynn, James P. C. Duncan, Troy Arcomano, Jeremy McGibbon, Oliver Watt-Meyer, Christopher S. Bretherton, Naser Mahfouz, Claudia Tebaldi, Luke Van Roekel, Andrew Roberts, Wuyin Lin, Finn Rebassoo, Jean-Christophe Golaz, and Peter M. Caldwell (2026). Stochastic Emulation of a Fully Coupled Preindustrial E3SMv3 Simulation. doi: 10.48550/arXiv.2608.10277. arXiv: 2608.10277 [physics.ao-ph]. url: https://arxiv.org/abs/2608.10277.

Zhou, Lu and Rong-Hua Zhang (2023). “A Self-Attention-Based Neural Network for Three-Dimensional Multivariate Modeling and Its Skillful ENSO Predictions”. In: Science Advances 9, eadf2827. doi: 10.1126/sciadv.adf2827.