PerfRecord allResultPerfRecord = new PerfRecord(taskGroupId, taskId, PerfRecord.PHASE.RESULT_NEXT_ALL); allResultPerfRecord.start(); long rsNextUsedTime = 0; long lastTime = System.nanoTime(); while (rs.next()) { rsNextUsedTime += (System.nanoTime() - lastTime); this.transportOneRe...
接着DataX Job 会调用 Scheduler 模块,根据配置的并发数量,将拆分成的 Task 重新组合,组装成 TaskGroup(任务组) 每一个 Task 都由 TaskGroup 负责启动,Task 启动后,会固定启动 Reader-->Channel-->Writer 线程来完成任务同步工作。 DataX 作业运行启动后,Job 会对 TaskGroup 进行监控操作,等待所有 TaskGroup ...
2021-08-2611:06:09.218[job-0]INFO StandAloneJobContainerCommunicator - Total20records,380bytes|Speed 38B/s,2records/s|Error0records,0bytes|All Task WaitWriterTime0.000s|All Task WaitReaderTime0.000s|Percentage100.00%2021-08-2611:06:09.223[job-0]INFO JobContainer - 任务启动时刻:2021-08-2611:05...
2021-08-2611:06:09.218[job-0]INFO StandAloneJobContainerCommunicator - Total20records,380bytes|Speed 38B/s,2records/s|Error0records,0bytes|All Task WaitWriterTime0.000s|All Task WaitReaderTime0.000s|Percentage100.00%2021-08-2611:06:09.223[job-0]INFO JobContainer - 任务启动时刻:2021-08-2611:05...
2021-12-13 19:26:28.829 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.060s | All Task WaitReaderTime 0.068s | Percentage 100.00% ...
2021-12-13 19:26:28.829 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.060s | All Task WaitReaderTime 0.068s | Percentage 100.00% ...
每一个Task都由TaskGroup负责启动,Task启动后,会固定启动Reader—>Channel—>Writer的线程来完成任务同步工作。 DataX作业运行起来之后, Job监控并等待多个TaskGroup模块任务完成,等待所有TaskGroup任务完成后Job成功退出。否则,异常退出,进程退出值非0 DataX调度流程: ...
[AnalysisStatistics.analysisStatisticsLog-53] 2021-12-11 22:28:51.852 [job-0] INFO StandAloneJobContainerCommunicator - Total 96768 records, 8931326 bytes | Speed 872.20KB/s, 9676 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 9.596s | All Task WaitReaderTime 0.239s | ...
2021-12-13 19:26:28.829 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.060s | All Task WaitReaderTime 0.068s | Percentage 100.00%2021-12-13 19:26:28.829 ...
2017-02-07 08:41:59.286 [job-0] INFO StandAloneJobContainerCommunicator - Total 324640 records, 65992457 bytes | Speed 1.03MB/s, 5312 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 5.474s | All Task WaitReaderTime 55.068s | Percentage 0.00% ...