package com.test.flink.kafkaSource import java.util.Properties import org.apache.flink.api.common.serialization.SimpleStringSchema; import org.apache.flink.streaming.api.datastream.DataStreamSource; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.streaming....
importorg.apache.flink.streaming.api.functions.source.RichSourceFunction;importjava.sql.Connection;importjava.sql.DriverManager;importjava.sql.ResultSet;importjava.sql.Statement;publicclassMySQLSourceextendsRichSourceFunction<User>{privatefinalStringjdbcUrl;privatefinalStringusername;privatefinalStringpassword;private...
import java.sql.Connection; import java.sql.DriverManager; import java.sql.PreparedStatement; import java.sql.ResultSet; /** * @Description TODO * @Author zytshijack * @Date 2019-05-14 08:32 * @Version 1.0 */ public class SourceFromMySQL extends RichSourceFunction<Student> { //Student就是...
在DynamicTableSource 中添加一个 TableScan 的 source @OverridepublicScanRuntimeProvidergetScanRuntimeProvider(ScanContext runtimeProviderContext){finalSourceFunction<RowData> sourceFunction =newMysqlSource(producedDataType, options);returnSourceFunctionProvider.of(sourceFunction,false); } MysqlSource 主要的逻辑...
importorg.apache.flink.api.connector.source.SourceSplit;importjava.util.UUID;publicclassMySplitimplementsSourceSplit{@OverridepublicStringsplitId(){returnUUID.randomUUID().toString().replace("-",""); } } 状态 // 无状态,占位publicclassNoOpEnumState{ ...
自定义Flink Source,案例分别实现了继承于SourceFunction的四个案例,三个完全自定义的Source, 另外一个Source为常见的MySQL,通过这几个案例,启发我们进行实际案例的Source研发 代码版本 Flink : 1.10.0 Scala : 2.12.6 官网部分说明 这个是关于Interface中Souce中的信息以及链接,关于SourceFunction的说明,基本使用到的是...
为了方便管理,FlinkSQL方式也是用Java代码写 1、创建database tEnv.executeSql("CREATE DATABASE IF NOT EXISTS cloud_test"); tEnv.executeSql("CREATE DATABASE IF NOT EXISTS league_test"); 2、创建source表 注意类型是'connector' = 'mysql-cdc'。
还有一种方法是继承RichSourceFunction,重写里面的方法,具体如下: 读取mysql的类: package com.my.flink.utils.streaming.mysql; import org.apache.flink.api.java.tuple.Tuple2; import org.apache.flink.configuration.Configuration; import org.apache.flink.streaming.api.functions.source.RichSourceFunction; ...
当MySQL表中存在多个主键时，Flink CDC的MySQL CDC Source会抛出以下异常信息：Copyjava.lang.Illegal...
source: type: mysql name: MySQL Source hostname: localhost port: 3306 username: <username> password: <password> tables: adb.\.*, bdb.user_table_[0-9]+, [app|web].order_\.* server-id: 5401-5404 sink: type: xxx 配置项 参数 说明 是否必填 数据类型 默认值 备注 type 数据源类型。 是...