Incubating

Apache DolphinScheduler

A distributed and easy-to-extend visual workflow scheduler system

A distributed and easy-to-extend visual workflow scheduler system

Dedicated to solving the complex task dependencies in data processing, making the scheduler system out of the box for data processing. Its main objectives are as follows:

Why DolphinScheduler

  • High Reliability

    Decentralized multi-master and multi-worker, HA is supported by itself, overload processing

  • User-Friendly

    All process definition operations are visualized, Visualization process defines key information at a glance, One-click deployment

  • Rich Scenarios

    Support pause, recover operation. Support multi-tenant. Support more task types e.g., spark, hive, mr, shell, python, sub_process

  • High Expansibility

    Support custom task types, Distributed scheduling, and the overall scheduling capability will increase linearly with the scale of the cluster