Scientific Workflow Systems: a way to relieve the programmers?

Last week we talked about how important database administrators and programmers are to making data sets usable by scientists. It's a big enough job to preserve and ensure access to data. Who is responsible for performing the complex manipulations that makes the data useful? Does it require subject specialty? Do scientists themselves have to become programmers? I started wondering if there is a way to automate some of the most routine things that scientists might do to data in database. I discovered a new term, scientific workflow, which means breaking down a scientific problem into a series of steps. I think it works like this: take raw data from source, perform X process, take that data, perform Y process, take that data, perform Z process... etc, until you reach the desired output. Basically, a workflow is a way to describe a set of computer processes so that they are understandable by humans. It turns out there are such things as scientific workflow systems (an example is Kepler) that allow scientists to design, implement, reuse, evolve, archive, and share these workflows without knowing much about computer programming. Scientists can select computational tasks from drop down menus and the workflow is represented in graphical form. The application interprets the scientists' workflow into a programming language (in the case of Kepler, java).

A phylogenetics workflow implemented in the Kepler system.

There has even been some work on finding ways to make workflows and workflow systems standardized and efficient. Researchers at UC Davis have devised a system “to make it easier for scientists to design workflows, to clearly show how workflow products were derived, to automatically optimize the performance of workflow execution, and otherwise make scientific workflow automation both accessible and practical for scientists” (McPhillips, Bowers, Zinn, and Ludascher, “Scientific workflow design for mere mortals.” Future Generation Computer Systems, 25:1, May 2009, 541-551. doi:10.1016/j.future.2008.06.013). They argue that scientific workflow systems should provide: well-formedness, clarity, predictability, recordability, reportability, reusability, scientific data modeling, and automatic optimization. They propose the “collection oriented modeling and design” (COMAD) framework. Just as a workflow intends to make what was once computer code understandable by humans, COMAD attempts to make a framework even more transparent and reusable. COMAD is pretty complex to understand without ever using a scientific workflow system, and COMAD isn’t the only framework being proposed. But, it’s heartening that people are trying to go beyond solving problems locally and are attempting to create standardized ways of modeling data and data processes. In addition, if workflows can be shared, or if parts of them can be shared, then the task of data manipulation might become easier.