Rebuilding a brand-new PDE surrogate paper, from scratch
GeoLAMP is a 2026 paper on predicting a physics simulation frame by frame with a causal transformer, the same underlying idea behind large language models, adapted to spatial fields instead of text. We reimplemented it from scratch and trained it on a transient heat-conduction problem of our own: a plate with a randomized notch cut into one edge, heated from below, predicting how the temperature front bends around the notch over time.
A transformer that predicts fields instead of words
LLMs predict the next chunk of text from everything written so far, using causal self-attention: a token only ever sees what came before it. This architecture applies the same idea to a temperature field. Given a few frames of simulated history, a causal transformer predicts the next 8 frames directly, block by block, never looking ahead.
The paper's original training method, flow matching, learns to transport random noise into the correct answer. That's built for problems with real randomness in the outcome. Heat conduction doesn't have that: given the same starting condition, there's one correct answer, not a distribution to sample from. Training the same transformer to predict that answer directly, instead of learning to sample it, converged faster and predicted more accurately.
Generalizing to a shape it never saw
An OpenFOAM sweep of 300 randomly generated notch geometries (position, depth, and width all varied) trains the model. One geometry is held out completely: never trained on, never seen until this comparison. The prediction tracks the real solve's shape and gradient closely, and stays accurate across the full 40-step rollout instead of drifting further off the longer it runs, the same stability property the paper reports for its own, much larger version.
Where this is honest about its limits
This is a small, cheap 2D conduction problem, chosen deliberately for a CPU-only, few-day build. The model is small for the same reason. The prediction is close to the real solve in shape and gradient, visibly softer at the sharpest feature, and a direct CFD solve is still faster than this surrogate for a problem this size. Surrogate models earn their speed on solves that are expensive to begin with; this one wasn't.
What this demonstrates is the same recipe used elsewhere on this site for surrogate modeling: CFD generates the training data, a model learns to predict the outcome directly, and the result gets checked against a held-out real solve. The point of this particular build was showing that a newly published architecture can be understood, rebuilt, and debugged quickly, not that it should already replace a CFD solve at this scale.
Architecture: Wang, Xu & Mukerji, “Geometry-aware Latent Autoregressive Generative Model for PDEs in Complex Domains,” arXiv:2609.00297 (2026).