Enabling Scalable Insitu Geophysical Data Analysis on the NCI HPC platform

As geophysical datasets are ever increasing in volume and complexity, and open source geophysical software are becoming more diverse and intricate, geophysicists are being challenged to transparently share and reproduce their processing workflows from their Raw Field Data (RFD) through to their Analysis Ready Data (ARD) and Interpretation Ready Data (IRD) products.

  

The AuScope/NCI/TERN/ARDC funded National High-resolution Geophysics Reference Collections for 2030 Computation Project is working towards:

1.Making minimally processed, high-resolution versions of multiple geophysical data types accessible on HPC;

2.Updating RFD, ARD, IRD and associated metadata to community agreed modern high performant data formats so that they are suitable for programmatic access on next-generation scalable computation; and   

3.Developing community driven managed geophysical and data science software environments that allow users the ability to fluently scale their data analysis workflows to NCI’s HPC Gadi system using CPUs and GPUs.
 

This project will lay the foundations for rapid and reproducible, interdisciplinary, in situ data analysis of geophysical datasets on next generation HPC-cloud systems. This presentation was given as part of the 2023 January Earth Science Information Partners (ESIP) Meeting held virtually Jan 23-27, 2023.