Hybrid Cloud Data Storage

A hybrid approach to adding cloud storage to an existing on-premises data processing workflow with less effort will be described. We used a Python program to efficiently transfer data files from local directories to an AWS S3 bucket. Concepts of managing a manifest of files already processed to prevent sending duplicates as well as a description of filtering files will be covered. The end result is a collection of Crowdsourced Bathymetry data accessible for public use. This poster was presented at the 2021 Earth Science Information Partners (ESIP) Winter Meeting held virtually in January 2021.