The challenge
Model forecasts are only useful in an operational dashboard if they run on schedule and their output arrives in a usable form. WRF-Chem and HYSPLIT runs on a shared HPC cluster involve batch jobs, large multidimensional outputs and runtime failures that have to be diagnosed and fixed quickly to avoid gaps in the forecast record.
Approach
I worked on the Linux HPC cluster running the WRF-Chem atmospheric model and the HYSPLIT dust forecast. I supported model execution and batch job submission, monitored runs and troubleshot runtime issues.
I processed model output and related atmospheric and emission datasets (NetCDF, HDF, GRIB) with Python (NumPy, Pandas, xarray, Matplotlib) for evaluation and for display alongside ground and satellite observations in the dashboards.
I coordinated with the modelling scientists and international technical partners on model workflows, using Git-based collaborative development.
What was delivered
- Execution, monitoring and runtime troubleshooting support for WRF-Chem simulations on HPC
- Operational support for the HYSPLIT dust forecast
- Python processing of model output for integration into the air quality dashboards
- Integration of emission datasets into modelling and air quality workflows
Stack
- WRF-Chem
- HYSPLIT
- HPC / Linux batch jobs
- Python (NumPy, Pandas, xarray, Matplotlib)
- NetCDF / HDF / GRIB
- THREDDS Data Server
- Git
Where it was used
Loading map…