AQ data pipelines, QA/QC and team capacity building

The problem

Station records with gaps, unit mix-ups and sensor drift, satellite granules in HDF and NetCDF that nobody has time to grid, and emission inventories in formats that do not match the model domain. These problems slow every report, paper and policy brief. Often the bigger gap is that the team has no shared, scripted way to handle these data, so each analysis starts again from scratch.

What you get

• Python QA/QC pipeline for station data, with documented flagging rules and gap handling
• Optional low-cost sensor correction workflow against co-located reference monitors
• Satellite product processing to a common grid and time step for the study area
• Emission inventory data conversion and regridding to a model or analysis domain
• Analysis-ready datasets in NetCDF/CSV with metadata
• Hands-on training sessions or workshop materials (Jupyter notebooks) for staff

How it works

1. 1

   Inventory

   List every dataset with its source, format, resolution, units and known issues, and agree the target outputs and how they will be used.

2. 2

   Build

   Write modular, version-controlled Python (Pandas, xarray, GDAL) processing and QA/QC code with clear configuration files.

3. 3

   Validate

   Review flagged data and intermediate plots with the client's scientists before finalising rules.

4. 4

   Transfer

   Run training on the client's own data so the team can rerun and extend the workflow.

Tools

• Python (Pandas, xarray, NumPy, Matplotlib)
• GDAL
• CDO
• NCO
• Jupyter
• Google Earth Engine
• PostgreSQL/PostGIS
• Git/GitHub