Geoweaver Tutorial: Resilient Workflow Management in One Place
posterposted on 15.07.2021, 21:01 by Ziheng SunZiheng Sun, Annie BurgessAnnie Burgess, Nicoleta Cristea, Daniel Q. Tong, Marshall MaMarshall Ma, Ahmed Alnaim
The automation of full stack workflow has become viral since the Earth data volume expontionally increases and the complexity of Earth system models and algorithms gets more difficult to manage and faciliate. The latest development in AI/ML technique brings a lot of new opportunities to significantly improve the accuracy, increase the model resilience and intelligence, and reduce the overall cost. However, managing and automating AI experiments is a grand challenge for the entire Earth science community. Geoweaver is a software developed to tackle this problem. We will demonstrate how to use Geoweaver to create AI workflow in one place and run the processes on various distributed platforms, separate code from computing resources for resilience, record the provenance of every workflow execution, and share and reuse workflows to boost knowledge accumulation and discovery. This poster was presented at the 2021 Earth Science Information Partners (ESIP) Summer Meeting held virtually in July 2021.