Setting Up and Deploying Apache SeaTunnel: A Step-by-Step Guide

With the continuous advancement of big data technology, Apache SeaTunnel has become the focus of many developers and enterprises. Today, we are pleased to announce that Apache SeaTunnel Web functionality has been officially released, bringing unprecedented ease of use and efficiency.

Apache SeaTunnel Web is a new component under the SeaTunnel project, designed for users who want to manage and monitor their big data stream processing tasks through a graphical interface. No longer needing complex command line operations, developers and data engineers can now easily configure, submit and manage their tasks on a user-friendly interface.

Preparing the Apache SeaTunnel environment

Because SeaTunnel Web uses the SeaTunnel Java Client to submit jobs, running SeaTunnel Web requires preparing a SeaTunnel Zeta Engine service first.

Based on the usage requirements of SeaTunnel Zeta Engine, the SeaTunnel Client node that submits the job must have the same operating system and installation directory structure as the SeaTunnel Server node that runs the job. Therefore, if you want to run SeaTunnel Web in IDEA, you must install and run a SeaTunnel Zeta Engine Server on the same machine as the IDEA.

Don’t worry, the next steps will tell you how to correctly install SeaTunnel Zeta Engine Server in different situations.

Run SeaTunnel Web In Server

To run SeaTunnel Web on the server, you need to first have a SeaTunnel Zeta Engine Server environment. If you do not already have one, you can refer to the following steps for deployment.

Deploy SeaTunnel Zeta Engine Server In Server Node

You have two ways to get the SeaTunnel installer package. Build from source code or download from the SeaTunnel website.

Note: The SeaTunnel version used here is only for writing this document to show you the process used, and does not necessarily represent the correct version. SeaTunnel Web and SeaTunnel Engine have strict version dependencies, and you can confirm the specific version mapping through xxx

Support SeaTunnel Version

• SeaTunnel 2.3.3 Only

Build from source code:

1. Get the source package from Apache SeaTunnel or GitHub Repository.
2. Build installer package using this maven command:

./mvnw -U -T 1C clean install -DskipTests -D"maven.test.skip"=true -D"maven.javadoc.skip"=true -D"checkstyle.skip"=true -D"license.skipAddThirdParty"

1. Locate the installer package in ${Your_code_dir}/seatunnel-dist/target, For example: apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz.

Download installer package:

1. Download from Apache SeaTunnel Download Page.
2. Follow the steps from the Deployment Guide to download and install connector plugins.
3. After the previous step, run:

tar -zcvf apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz apache-seatunnel-2.3.3-SNAPSHOT

Deploy SeaTunnel Zeta Server:

1. Obtain the installer package apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz.
2. Copy it to your server node and deploy as per instructions on the Deployment Page.

Deploy SeaTunnel Zeta Client In SeaTunnel Web Run Node:

1. Copy apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz to the SeaTunnel Web node and unzip it in the same path as the SeaTunnel Zeta Server node.
2. Set up SEATUNNEL_HOME and configure hazelcast-client.yaml as per this guide.
3. Run the job:

$SEATUNNEL_HOME/bin/seatunnel.sh --config $SEATUNNEL_HOME/config/v2.batch.config.template

Download and Install SeaTunnel Web

1. Download SeaTunnel web from Apache SeaTunnel.
2. Transfer the file apache-seatunnel-web-bin-${project.version}.tar.gz to your server node and unzip:

tar -zxvf apache-seatunnel-web-bin-${project.version}.tar.gz

Init database

1. Edit the configuration in apache-seatunnel-web-bin-${project.version}/script/seatunnel_server_env.sh.

export HOSTNAME="localhost"
export PORT="3306"
export USERNAME="root"
export PASSWORD="123456"

Initialize the database:

sh apache-seatunnel-web-bin-${project.version}/script/init_sql.sh

Config application and Run SeaTunnel Web Backend Server

1. Edit database info in apache-seatunnel-web-bin-${project.version}/conf/application.yml.
2. Copy the necessary configuration files.

Start SeaTunnel Web

cd apache-seatunnel-web-${project.version}
sh bin/seatunnel-backend-daemon.sh start

Now, access the web interface at http://127.0.0.1:8801/ui/. The default credentials are admin/admin.