Soma-workflow: An unified and simple interface to parallel computing resource
Soma-workflow provides a homogeneous and single way to submit and monitor
parallel computing to various computing resources (laptops, workstation farms,
clusters...).
The submission, control and monitoring of jobs and workflows is done via a
simple Python application programming interface (API) and/or a graphical user
interface (GUI).
Soma-workflow uses existing distributed resource management systems (DRMS) (such
as Grid Engine, Condor, Torque/PBS, LSF..) installed on the computing resources
to execute the jobs.
Soma-workflow is an open source Python application.
The sources can be found here and be downloaded here.
Main Features
- Unified interface for many resources:
- Submission of jobs or workflows with an unique interface to various
computing resources possibly managed by various systems (such as Grid
Engine, Condor, Torque/PBS, LSF..)
- Python API and Graphical User Interface:
- The GUI provides an easy and quick way of monitoring workflows on various
resources.
- Workflow engine:
- Soma-workflow enables the user to build his workflow (a set of jobs and
dependencies between jobs) and handles the execution of jobs on computing
resource according to the jobs dependencies.
- Remote access to computing resources
- The communication with a remote computing resource is done transparently for
the user. The communication between the client application and the remote
computing resource is done within a ssh port forwarding tunnel.
- Disconnections:
- The client/server architecture enables the user to close the client
application at any time. The workflows and jobs execution are not stopped.
The user can open a client at any time to check the status of his
work.
- File transfers:
- If the user’s machine and the computing resource do not have a shared file
system, soma-workflow can manage file transfers between the client and the
computing resource. This feature is available but needs to be optimized to
speed up the transfers.
- Simple one process mode:
- If you are not interested by the remote access, file transfer and disconnection
features, Soma-workflow can be used as a simple and regular one process
application (see Installation and configuration: Light mode).
