There are two deployment modes for the backend:
Download the latest version of the installation package, download address: download, download apache-dolphinscheduler-incubating-x.x.x-dolphinscheduler-backend-bin.tar.gz
Note: DolphinScheduler itself does not rely on Hadoop, Hive, Spark, PostgreSQL, but only calls their Client to run the corresponding tasks.
sudo-u {linux-user}
, so deployment users need sudo privileges and are confidential.vi /etc/sudoers
# For example, the deployment user is an dolphinscheduler account
dolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
# And you need to comment out the Default requiretty line
#Default requiretty
Configure SSH secret-free login on deployment machines and other installation machines. If you want to install dolphinscheduler on deployment machines, you need to configure native password-free login itself.
Create databases and accounts
Execute the following command to create database and account
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}';
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY '{password}';
flush privileges;
creates tables and imports basic data Modify the following attributes in ./conf/application-dao.properties
spring.datasource.url
spring.datasource.username
spring.datasource.password
Execute scripts for creating tables and importing basic data
sh ./script/create-dolphinscheduler.sh
instruction of dolphinscheduler-backend directory
bin : Basic service startup script
DISCLAIMER-WIP : DISCLAIMER-WIP
conf : Project Profile
lib : The project relies on jar packages, including individual module jars and third-party jars
LICENSE : LICENSE
licenses : licenses
NOTICE : NOTICE
script : Cluster Start, Stop and Service Monitor Start and Stop scripts
sql : The project relies on SQL files
install.sh : One-click deployment script
Modify permissions (please modify the 'deployUser' to the corresponding deployment user) so that the deployment user has operational privileges on the dolphinscheduler-backend directory
sudo chown -R deployUser:deployUser dolphinscheduler-backend
Modify the .dolphinscheduler_env.sh
environment variable in the conf/env/directory
Modify deployment parameters (depending on your server and business situation):
Modify the parameters in install.sh
to replace the values required by your business
If you use hdfs-related functions, you need to copyhdfs-site.xml and core-site.xml to the conf directory
Either of the following two methods can be deployed,binary file deployment is recommended, and experienced partners can use source deployment as well.
Install ZooKeeper tools
pip install kazoo
Switch to deployment user, one-click deployment
sh install.sh
Use the jps
command to check if the services are started (jps
comes from Java JDK
)
MasterServer ----- Master Service
WorkerServer ----- Worker Service
LoggerServer ----- Logger Service
ApiApplicationServer ----- API Service
AlertServer ----- Alert Service
If all services are normal, the automatic deployment is successful
After successful deployment, the log can be viewed and stored in a specified folder.
logs/
├── dolphinscheduler-alert-server.log
├── dolphinscheduler-master-server.log
|—— dolphinscheduler-worker-server.log
|—— dolphinscheduler-api-server.log
|—— dolphinscheduler-logger-server.log
After downloading the release version of the source package, uncompress it into the root directory
Build a tar package
Execute the compilation command:
mvn -U clean package -Prelease -Dmaven.test.skip=true
View directory
After normal compilation, apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-backend-bin.tar.gz
is generated in the ./dolphinscheduler-dist/dolphinscheduler-backend/target
directory
OR build a rpm package
The rpm package can be installed on the Linux platform using the rpm command or yum. The rpm package can be used to help Dolphinscheduler better integrate with other management tools, such as ambari, cloudera manager.
Execute the compilation command:
mvn -U clean package -Prpmbuild -Dmaven.test.skip=true
View directory
After normal compilation, apache-dolphinscheduler-incubating-${latest.release.version}-1.noarch.rpm
is generated in the ./dolphinscheduler-dist/target/rpm/apache-dolphinscheduler-incubating/RPMS/noarch/
directory
Decompress the compiled tar.gz package or use the rpm command to install (the rpm installation method will install dolphinscheduler in the /opt/soft directory) . The dolphinscheduler directory structure is like this:
../
├── bin
├── conf
|── DISCLAIMER
|—— install.sh
|—— lib
|—— LICENSE
|—— licenses
|—— NOTICE
|—— script
|—— sql
Install ZooKeeper tools
pip install kazoo
Switch to deployment user, one-click deployment
sh install.sh
stop all services in the cluster
sh ./bin/stop-all.sh
start all services in the cluster
sh ./bin/start-all.sh
start and stop one master server
sh ./bin/dolphinscheduler-daemon.sh start master-server
sh ./bin/dolphinscheduler-daemon.sh stop master-server
sh ./bin/dolphinscheduler-daemon.sh start worker-server
sh ./bin/dolphinscheduler-daemon.sh stop worker-server
sh ./bin/dolphinscheduler-daemon.sh start api-server
sh ./bin/dolphinscheduler-daemon.sh stop api-server
sh ./bin/dolphinscheduler-daemon.sh start logger-server
sh ./bin/dolphinscheduler-daemon.sh stop logger-server
sh ./bin/dolphinscheduler-daemon.sh start alert-server
sh ./bin/dolphinscheduler-daemon.sh stop alert-server
Modify the following properties in ./conf/application-dao.properties
```
spring.datasource.url
spring.datasource.username
spring.datasource.password
```
The database can be upgraded automatically by executing the following command:
sh ./script/upgrade-dolphinscheduler.sh