Enhancing Your Development Workflow with Apache SeaTunnel 2.3.4: A Deep Dive into Local Debugging

Apache SeaTunnel
2 min readMar 25, 2024

--

While developing an SM2 encryption component using Apache SeaTunnel, I noticed the community’s articles on local debugging of SeaTunnel were overly simplistic and lacked explanations for many scenarios. Thus, I’ve compiled this document based on the issues I encountered, hoping it will aid others. Check out the official SeaTunnel local debugging documentation for version 2.3.4 and upgrade your skills!

The engine used is Flink (no need to download, as SeaTunnel includes the dependencies), with input and output via mysql to mysql.

Tool Overview

  1. IDEA 2023.3.2
  2. MySQL 8.0.36
  3. JDK 1.8
  4. SeaTunnel 2.3.4-release

Preparation Before Development

  1. Pull the code from GitHub:

GitHub URL: https://github.com/apache/seatunnel/

2. Compile in the Console

Compilation command:

./mvnw clean install

For compiling version 2.3.3 of Apache SeaTunnel, refer to Da Fei Ge’s Blog 4.2.1.

3. Database and Table Creation

(Note: Do not use MySQL system databases like information_schema, mysql, performance_schema, sys. Create a new database instead, as SeaTunnel excludes these system databases, a feature added in 2.3.4 that wasn’t present in 2.3.3).

4. Adding Configuration Files

Add the configuration file you need to run in the resource/examples directory of the seatunnel-flink-connector-v2-example project and modify the file path in SeaTunnelApiExample.java to point to your file.

Example test.json (SeaTunnel supports running configuration files in both json and config formats).

5. Maven Addition

Add the following pom dependency to the seatunnel-flink-connector-v2-example project and refresh Maven. Without these dependencies, errors will occur. Add dependencies based on your connection method; I used connector-jdbc. If your source or sink uses HTTP connections, you’ll need to add connect-http dependencies. Check the seatunnel-connectors-v2 project for details.

For MySQL, include the MySQL driver dependency:

Local Debugging

1. Debug Run

Debug run the main class under SeaTunnelApiExample.java, and you can breakpoint where needed for debugging.

2. Execution Result

--

--

Apache SeaTunnel

The next-generation high-performance, distributed, massive data integration tool.