Enhancing Your Development Workflow with Apache SeaTunnel 2.3.4: A Deep Dive into Local Debugging
While developing an SM2 encryption component using Apache SeaTunnel, I noticed the community’s articles on local debugging of SeaTunnel were overly simplistic and lacked explanations for many scenarios. Thus, I’ve compiled this document based on the issues I encountered, hoping it will aid others. Check out the official SeaTunnel local debugging documentation for version 2.3.4 and upgrade your skills!
The engine used is Flink (no need to download, as SeaTunnel includes the dependencies), with input and output via mysql to mysql.
Tool Overview
- IDEA 2023.3.2
- MySQL 8.0.36
- JDK 1.8
- SeaTunnel 2.3.4-release
Preparation Before Development
- Pull the code from GitHub:
GitHub URL: https://github.com/apache/seatunnel/
2. Compile in the Console
Compilation command:
./mvnw clean install
For compiling version 2.3.3 of Apache SeaTunnel, refer to Da Fei Ge’s Blog 4.2.1.
3. Database and Table Creation
(Note: Do not use MySQL system databases like information_schema, mysql, performance_schema, sys. Create a new database instead, as SeaTunnel excludes these system databases, a feature added in 2.3.4 that wasn’t present in 2.3.3).
4. Adding Configuration Files
Add the configuration file you need to run in the resource/examples directory of the seatunnel-flink-connector-v2-example project and modify the file path in SeaTunnelApiExample.java to point to your file.
Example test.json (SeaTunnel supports running configuration files in both json and config formats).
5. Maven Addition
Add the following pom dependency to the seatunnel-flink-connector-v2-example project and refresh Maven. Without these dependencies, errors will occur. Add dependencies based on your connection method; I used connector-jdbc. If your source or sink uses HTTP connections, you’ll need to add connect-http dependencies. Check the seatunnel-connectors-v2 project for details.
For MySQL, include the MySQL driver dependency:
Local Debugging
1. Debug Run
Debug run the main class under SeaTunnelApiExample.java, and you can breakpoint where needed for debugging.
