Geoweaver: Affordable Workflow Tool for Earth AI Experiments

Earth scientists spend numerous hours on exploring new technologies to learn new knowledge and find critical information from the obtained Earth observations. There are many duplicated efforts happening in every corner of the universities and research institutes, and it is common to find people struggling to solve the technical problems that others might already meet and solve. However, due to disorganization and less sharable format, all these efforts vanished over time and even the authors cannot reproduce them after a period of time. It is not a new problem and there are solutions already (e.g., using professional workflow management software), but requiring users to invest even more time to learn and use them. The barrier and cost are too high to justify their benefits. In light of this problem, we developed a scientist-affordable workflow tool to organize their AI/ML workflows and record all the source code changes and logs of every model run. It is a utility tool which doesn't require much time to learn and they can take the Geoweaver workflow package and run it by themselves even without Geoweaver installed. The entry of barrier is lowered to the level that any people with Python scripting experience can easily use it. The workflow automation, provenance recording, history exportation, are completely under the hood and make scientists focus on the problem solving and information extraction over worrying about their technical debt and potential loss of their experiment history or source code. Using Geoweaver is a long term investment and will make every scientist' work fresh preserved and understandable even years after the original experiments. This presentation was given at the 2022 NASA ESDSWG Meeting (April 19-21, 2022).