Open in app

Sign in

Write

Sign in

Apache SeaTunnel 2.3.3+Web 1.0.0 version installation and deployment

Apache SeaTunnel
13 min readMay 6, 2024

--

Apache SeaTunnel is a distributed, high-performance, and scalable data integration platform designed for both real-time and offline data processing, supporting data migration and transformation across multiple data sources. The packages Apache-seatunnel-web-1.0.0-bin.tar.gz and apache-seatunnel-2.3.3-bin.tar.gz represent the installation package for the Apache SeaTunnel Web interface and the core engine of Apache SeaTunnel, respectively.

  • apache-seatunnel-web-1.0.0-bin.tar.gz: This is the installation package for Apache SeaTunnel Web, providing a user-friendly web interface that allows users to manage and monitor SeaTunnel tasks more conveniently. This includes creating, editing, and running jobs. This interface enables users to configure and manage without direct interaction with the command line.
  • apache-seatunnel-2.3.3-bin.tar.gz: This is the binary distribution of the Apache SeaTunnel core engine, containing the core libraries needed to execute data extraction, transformation, and loading. It is the component that actually carries out data processing tasks and interacts with various data sources and destinations, such as databases, file systems, and message queues.

When deploying Apache SeaTunnel, it is generally necessary to first install and configure the core engine, and then decide whether to use the Web interface for management based on your needs. If you only need to run and manage tasks from the command line, installing the core engine alone is sufficient. If you desire a graphical task management interface, installing Apache SeaTunnel Web is also required.

Download links for the installation and source packages: https://seatunnel.apache.org/download/

Core Engine Deployment

Uploading and Extracting the Installation Package: tar xf apache-seatunnel-2.3.3-bin.tar.gz

Automated Installation

Automatically install the plugin (full installation, which is quite slow — it didn’t finish until after work, and it was only completed the next morning).

[opensource@bigdata02 bin]$ pwd
/home/opensource/app/apache-seatunnel-2.3.3/bin
[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ ./install-plugin.sh

Service Verification

Execute a built-in task for verification: sh seatunnel.sh --config ../config/v2.batch.config.template -m local

[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

Start the Service

Start the service and run it in the background for web connection: nohup sh seatunnel-cluster.sh 2>&1 &

[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

Manual Installation

If you do not wish to install everything, modify the configuration: vi config/plugin_config

If online installation is not possible, manual installation is the only option. Maven repository download address: https://repo.maven.apache.org/maven2/org/apache/seatunnel/

For jars automatically downloaded, they have been packaged and uploaded to CSDN for free download, just place them in connectors/seatunnel.

Download address: https://download.csdn.net/download/qq_36434219/88609069

CSDN resources cannot exceed 1000M, hence files are in .tar.xz format (unzipping may be slow, about ten minutes).

Compression commands

tar -cf seatunnel_connectors_2.3.3.tar seatunnel
tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel
tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel

Corresponding decompression commands:

tar -xf seatunnel_connectors_2.3.3.tar 
tar -xzf seatunnel_connectors_2.3.3.tar.gz 
tar -xJf seatunnel_connectors_2.3.3.tar.xz

Web Interface Deployment

Upload and extract the web deployment package: tar xf apache-seatunnel-web-1.0.0-bin.tar.gz

Database Initialization

Initialize with the SQL script script/seatunnel_server_mysql.sql

SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for role
-- ----------------------------
DROP TABLE IF EXISTS `role`;
CREATE TABLE `role`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `type` int(2) NOT NULL,
  `role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of role
-- ----------------------------
INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');
INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');

-- ----------------------------
-- Table structure for role_user_relation
-- ----------------------------
DROP TABLE IF EXISTS `role_user_relation`;
CREATE TABLE `role_user_relation`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `role_id` int(20) NOT NULL,
  `user_id` int(20) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_datasource
-- ----------------------------
DROP TABLE IF EXISTS `t_st_datasource`;
CREATE TABLE `t_st_datasource`  (
  `id` bigint(20) NOT NULL,
  `datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',
  `datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_definition
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_definition`;
CREATE TABLE `t_st_job_definition`  (
  `id` bigint(20) NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `name`(`name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance`;
CREATE TABLE `t_st_job_instance`  (
  `id` bigint(20) NOT NULL,
  `job_define_id` bigint(20) NOT NULL,
  `job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  `end_time` timestamp(3) NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance_history
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance_history`;
CREATE TABLE `t_st_job_instance_history`  (
  `id` bigint(20) NOT NULL,
  `dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_line
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_line`;
CREATE TABLE `t_st_job_line`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_line_version_index`(`version_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_metrics
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_metrics`;
CREATE TABLE `t_st_job_metrics`  (
  `id` bigint(20) NOT NULL,
  `job_instance_id` bigint(20) NOT NULL,
  `pipeline_id` int(20) NOT NULL,
  `read_row_count` bigint(20) NOT NULL,
  `write_row_count` bigint(20) NOT NULL,
  `source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `read_qps` bigint(20) NULL DEFAULT NULL,
  `write_qps` bigint(20) NULL DEFAULT NULL,
  `record_delay` bigint(20) NULL DEFAULT NULL,
  `status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_task
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_task`;
CREATE TABLE `t_st_job_task`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `datasource_id` bigint(20) NULL DEFAULT NULL,
  `datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_version
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_version`;
CREATE TABLE `t_st_job_version`  (
  `id` bigint(20) NOT NULL,
  `job_id` bigint(20) NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_virtual_table
-- ----------------------------
DROP TABLE IF EXISTS `t_st_virtual_table`;
CREATE TABLE `t_st_virtual_table`  (
  `id` bigint(20) NOT NULL,
  `datasource_id` bigint(20) NOT NULL,
  `virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `status` tinyint(4) NOT NULL,
  `type` tinyint(4) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user_login_log
-- ----------------------------
DROP TABLE IF EXISTS `user_login_log`;
CREATE TABLE `user_login_log`  (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `token_status` tinyint(1) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of user_login_log
-- ----------------------------

INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);

SET FOREIGN_KEY_CHECKS = 1;

Modify Configuration Files

Edit the database information: vi conf/application.yml

Copy the configuration files from the engine service to the web configuration directory.

[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/
[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/

Download Data Source JARs

If using MySQL as the metadata database, place the corresponding driver package mysql-connector-java-8.0.25.jar in the libs directory.

Download other data source resource packages, first download the source code, then execute the download_datasource.sh script (local installation of git required).

Source code address:https://github.com/apache/seatunnel-web

Modify relevant configurations in download_datasource.sh

Execute download_datasource.sh

Begin downloading jars

Once downloaded, upload to the libs directory.

Start the Service

sh bin/seatunnel-backend-daemon.sh start

Web Interface Overview

Login

Login URL: http://xxxx:8801/ui/#/login; User credentials: admin/admin

Other

Issue Record

Service startup failure: Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
 in 'reader', line 30, column 15:
        password: &R7#J*vDVWEuT48m
                  ^
unexpected character found *(42)
 in 'reader', line 30, column 20:
        password: &R7#J*vDVWEuT48m
                       ^

        at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)
        at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
        at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)
        at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
        at org.yaml.snakeyaml.composer.Composer.composeScalarN

Solution: Add double quotes around the password &R7#J*vDVWEuT48m

Original article link: https://blog.csdn.net/qq_36434219/article/details/134857560

--

--

Apache SeaTunnel

Written by Apache SeaTunnel

282 Followers

The next-generation high-performance, distributed, massive data integration tool.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams