Here're 3 ways to quickly install DolphinScheduler
In this way, you need to install docker-compose as a prerequisite, please install it yourself according to the rich docker-compose installation guidance on the Internet
Please download the source code package apache-dolphinscheduler-incubating-1.3.5-src.zip, download address: download
$ unzip apache-dolphinscheduler-incubating-1.3.5-src.zip
$ cd apache-dolphinscheduler-incubating-1.3.5-src-release/docker/docker-swarm
$ docker pull dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.5
$ docker tag apache/dolphinscheduler:1.3.5 apache/dolphinscheduler:latest
$ docker-compose up -d
The PostgreSQL (with username root
, password root
and database dolphinscheduler
) and ZooKeeper services will start by default
Visit the Web UI: http://localhost:12345/dolphinscheduler (The local address is http://localhost:12345/dolphinscheduler)
The default username is admin
and the default password is dolphinscheduler123
Please refer to the Quick Start
in the chapter Quick Start to explore how to use DolphinScheduler
In this way, you need to install docker as a prerequisite, please install it yourself according to the rich docker installation guidance on the Internet
dolphinscheduler
sql/dolphinscheduler-postgre.sql
to create tables and initial dataWe have already uploaded user-oriented DolphinScheduler image to the Docker repository so that you can pull the image from the docker repository:
docker pull dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.5
$ docker run -d --name dolphinscheduler \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
-p 12345:12345 \
apache/dolphinscheduler:1.3.5 all
Note: database username test and password test need to be replaced with your actual PostgreSQL username and password, 192.168.x.x need to be replaced with your relate PostgreSQL and ZooKeeper host IP
Same as above
The following services are automatically started when the container starts:
MasterServer ----- master service
WorkerServer ----- worker service
LoggerServer ----- logger service
ApiApplicationServer ----- api service
AlertServer ----- alert service
If you just want to run part of the services in the DolphinScheduler
You can start some services in DolphinScheduler by running the following commands.
$ docker run -d --name dolphinscheduler-master \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
apache/dolphinscheduler:1.3.5 master-server
$ docker run -d --name dolphinscheduler-worker \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
apache/dolphinscheduler:1.3.5 worker-server
$ docker run -d --name dolphinscheduler-api \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
-p 12345:12345 \
apache/dolphinscheduler:1.3.5 api-server
$ docker run -d --name dolphinscheduler-alert \
-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e DATABASE_DATABASE="dolphinscheduler" \
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
apache/dolphinscheduler:1.3.5 alert-server
Note: You must be specify DATABASE_HOST
, DATABASE_PORT
, DATABASE_DATABASE
, DATABASE_USERNAME
, DATABASE_PASSWORD
, ZOOKEEPER_QUORUM
when start a standalone dolphinscheduler server.
The Docker container is configured through environment variables, and the DolphinScheduler Docker Environment Variables lists the configurable environment variables of the DolphinScheduler and their default values
Especially, it can be configured through the environment variable configuration file docker-compose.yml
and docker-stack.yml
in Docker Compose and Docker Swarm separately
Start, restart, stop or list containers:
docker-compose start
docker-compose restart
docker-compose stop
docker-compose ps
Stop containers and remove containers, networks:
docker-compose down
Stop containers and remove containers, networks and volumes:
docker-compose down -v
List all running containers:
docker ps
docker ps --format "{{.Names}}" # only print names
View the logs of a container named dolphinscheduler-api:
docker logs dolphinscheduler-api
docker logs -f dolphinscheduler-api # follow log output
docker logs --tail 10 dolphinscheduler-api # show last 10 lines from the end of the logs
Important: Please delete the container_name
and ports
field of dolphinscheduler-master
and dolphinscheduler-worker
in docker-compose.yml
Scale master to 2 instances:
docker-compose up -d --scale dolphinscheduler-master=2 dolphinscheduler-master
Scale worker to 3 instances:
docker-compose up -d --scale dolphinscheduler-worker=3 dolphinscheduler-worker
Assuming that the Docker Swarm cluster has been created (If there is no Docker Swarm cluster, please refer to create-swarm)
Start a stack named dolphinscheduler:
docker stack deploy -c docker-stack.yml dolphinscheduler
List the services in the stack named dolphinscheduler:
docker stack services dolphinscheduler
Stop and remove the stack named dolphinscheduler:
docker stack rm dolphinscheduler
Remove the volumes of the stack named dolphinscheduler:
docker volume rm -f $(docker volume ls --format "{{.Name}}" | grep -e "^dolphinscheduler")
Important: Please delete the ports
field of dolphinscheduler-master
and dolphinscheduler-worker
in docker-stack.yml
Scale master of the stack named dolphinscheduler to 2 instances:
docker service scale dolphinscheduler_dolphinscheduler-master=2
Scale worker of the stack named dolphinscheduler to 3 instances:
docker service scale dolphinscheduler_dolphinscheduler-worker=3
In Unix-Like, execute in Terminal:
$ bash ./docker/build/hooks/build
In Windows, execute in cmd or PowerShell:
C:\dolphinscheduler-src>.\docker\build\hooks\build.bat
Please read ./docker/build/hooks/build
./docker/build/hooks/build.bat
script files if you don't understand
Please download the binary distribution package apache-dolphinscheduler-incubating-1.3.5-dolphinscheduler-bin.tar.gz, download address: download. And put apache-dolphinscheduler-incubating-1.3.5-dolphinscheduler-bin.tar.gz into the apache-dolphinscheduler-incubating-1.3.5-src-release/docker/build
directory, execute in Terminal or PowerShell:
$ cd apache-dolphinscheduler-incubating-1.3.5-src-release/docker/build
$ docker build --build-arg VERSION=1.3.5 -t apache/dolphinscheduler:1.3.5 .
PowerShell should use
cd apache-dolphinscheduler-incubating-1.3.5-src-release/docker/build
If you would like to do additional initialization in an image derived from this one, add one or more environment variables under /root/start-init-conf.sh
, and modify template files in /opt/dolphinscheduler/conf/*.tpl
.
For example, to add an environment variable SECURITY_AUTHENTICATION_TYPE
in /root/start-init-conf.sh
:
export SECURITY_AUTHENTICATION_TYPE=PASSWORD
and to modify application-api.properties.tpl
template file, add the SECURITY_AUTHENTICATION_TYPE
:
security.authentication.type=${SECURITY_AUTHENTICATION_TYPE}
/root/start-init-conf.sh
will dynamically generate config file:
echo "generate dolphinscheduler config"
ls ${DOLPHINSCHEDULER_HOME}/conf/ | grep ".tpl" | while read line; do
eval "cat << EOF
$(cat ${DOLPHINSCHEDULER_HOME}/conf/${line})
EOF
" > ${DOLPHINSCHEDULER_HOME}/conf/${line%.*}
done
Because of the commercial license, we cannot directly use the driver and client of MySQL.
If you want to use MySQL, you can build a new image based on the
apache/dolphinscheduler
image as follows.
Download the MySQL driver mysql-connector-java-5.1.49.jar (require >=5.1.47
)
Create a new Dockerfile
to add MySQL driver and client:
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.5
COPY mysql-connector-java-5.1.49.jar /opt/dolphinscheduler/lib
RUN apk add --update --no-cache mysql-client
docker build -t apache/dolphinscheduler:mysql .
image
fields to apache/dolphinscheduler:mysql
in docker-compose.yml
If you want to deploy dolphinscheduler on Docker Swarm, you need modify
docker-stack.yml
Comment the dolphinscheduler-postgresql
block in docker-compose.yml
Add dolphinscheduler-mysql
service in docker-compose.yml
(Optional, you can directly use a external MySQL database)
Modify all DATABASE environment variables in docker-compose.yml
DATABASE_TYPE: mysql
DATABASE_DRIVER: com.mysql.jdbc.Driver
DATABASE_HOST: dolphinscheduler-mysql
DATABASE_PORT: 3306
DATABASE_USERNAME: root
DATABASE_PASSWORD: root
DATABASE_DATABASE: dolphinscheduler
DATABASE_PARAMS: useUnicode=true&characterEncoding=UTF-8
If you have added
dolphinscheduler-mysql
service indocker-compose.yml
, just setDATABASE_HOST
todolphinscheduler-mysql
Datasource manage
?Because of the commercial license, we cannot directly use the driver of MySQL.
If you want to add MySQL datasource, you can build a new image based on the
apache/dolphinscheduler
image as follows.
Download the MySQL driver mysql-connector-java-5.1.49.jar (require >=5.1.47
)
Create a new Dockerfile
to add MySQL driver:
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.5
COPY mysql-connector-java-5.1.49.jar /opt/dolphinscheduler/lib
docker build -t apache/dolphinscheduler:mysql-driver .
image
fields to apache/dolphinscheduler:mysql-driver
in docker-compose.yml
If you want to deploy dolphinscheduler on Docker Swarm, you need modify
docker-stack.yml
Run a dolphinscheduler (See How to use this docker image)
Add a MySQL datasource in Datasource manage
Datasource manage
?Because of the commercial license, we cannot directly use the driver of Oracle.
If you want to add Oracle datasource, you can build a new image based on the
apache/dolphinscheduler
image as follows.
Download the Oracle driver ojdbc8.jar (such as ojdbc8-19.9.0.0.jar
)
Create a new Dockerfile
to add Oracle driver:
FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.5
COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
docker build -t apache/dolphinscheduler:oracle-driver .
image
fields to apache/dolphinscheduler:oracle-driver
in docker-compose.yml
If you want to deploy dolphinscheduler on Docker Swarm, you need modify
docker-stack.yml
Run a dolphinscheduler (See How to use this docker image)
Add a Oracle datasource in Datasource manage