2019-06-17 09:05:11,184 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore 2019-06-17 09:05:11,184 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job CarTopSpeedWindowingExample (7752ea7b0e7303c780de9d86a5ded3fa) switched from state ...
247 INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at z05c05217.sqa.zth.tbsite.net/11.163.188.29:80502019-06-18 09:21:59,428 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class...
2019-06-17 09:05:11,184 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No master state to restore 2019-06-17 09:05:11,184 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job CarTopSpeedWindowingExample (7752ea7b0e7303c780de9d86a5ded3fa) switched from state ...
19/06/01 14:52:07 INFO CheckpointCoordinator: No master state to restore 19/06/01 14:52:07 INFO ExecutionGraph: Job com.zzy.bigdata.flink.streaming.StreamDemoKafka2Mysql (a7188181ec45ab397d21bb1f928c7b89) switched from state CREATED to RUNNING. 19/06/01 14:52:07 INFO ExecutionGraph:...
ded3fa.2019-06-1709:05:11,184INFOorg.apache.flink.runtime.checkpoint.CheckpointCoordinator -Nomaster statetorestore2019-06-1709:05:11,184INFOorg.apache.flink.runtime.executiongraph.ExecutionGraph - Job CarTopSpeedWindowingExample (7752ea7b0e7303c780de9d86a5ded3fa) switchedfromstate RUNNINGto...
a5ded3fa.2019-06-1709:05:11,184INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator-Nomaster statetorestore2019-06-1709:05:11,184INFO org.apache.flink.runtime.executiongraph.ExecutionGraph-Job CarTopSpeedWindowingExample (7752ea7b0e7303c780de9d86a5ded3fa) switchedfromstateRUNNINGto...
默认的超时时间是 10min,如果 state 规模大,则需要合理配置。最坏情况是分布式地创建速度大于单点(job master 端)的删除速度,导致整体存储集群可用空间压力较大。建议当检查点频繁因为超时而失败时,增大超时时间。 存储 消息中间件 NoSQL 大数据 Kafka API 数据库 流计算 容器 ...
master e) zoo.cfg server.1=master:2888:3888 五.验证:本地模式 参考:https://ci.apache.org/projects/flink/flink-docs-release-1.11/try-flink/local_installation.html 1. 启动服务 start-cluster.sh start-cluster 2.测试 mkdir $FLINK_HOME/examples/output ...
当Flink触发stateful functon的一次checkpoint时,snapshotState()方法会被调用。方法接收两个参数,checkpointId为唯一的单调递增的检查点Id,timestamp为当master机器开始做检查点操作时的墙上时钟(机器时间)。方法必须返回序列化好的状态对象的列表。 当宕机程序从检查点或者保存点恢复时会调用restoreState()方法。restoreSt...
那么如果你的 operator state 中的 list 长度达到一定规模时,这个 offset 数组就可能会有几十 MB 的规模,关键这个数组是会返回给 job master,当 operator 的并发数目很大时,很容易触发 job master 的内存超用问题。我们遇到过用户把 operator state 当做黑名单存储,结果这个黑名单规模很大,导致一旦开始执行 checkpoin...