SciPipe-Turning Scientific Workflows into Computer Programs

← Back to publications

Published: 2019-05-06

Formatted citation

Lampa S, Dahlö M, Alvarsson J, Spjuth O. SciPipe-Turning Scientific Workflows into Computer Programs.
IEEE Computing in Science & Engineering. 21, 3, 109-113. (2019). DOI: 10.1109/MCSE.2019.2907814

Abstract

By providing increased robustness to failure and reproducibility of results, workflows are an established workhorse of computer-aided science as well as an increasingly important part also in data pipelines in industry. For such complex and dynamic needs, we argue for the benefits of workflows implemented as computer programs, using workflow tools implemented as programming libraries. Furthermore, by using flow-based programming principles, a declarative while programmatic API can be presented, while still hiding a lot of low-level details under the hood. Being able to provide workflows in the form of compiled executable files has benefits both for the ease of deployment and future reproducibility. Workflow tool implementations are at risk for bit rot or incompatible changes well before the mainstream programming languages -or processor architectures -are.