9、然后再了解sqoop的过程中,get到sqoop export也是把任务转化成Hadoop的mapreduct job去执行的。其实在azkaban的错误日志中就有显示 INFO mapreduce.Job: Job job_1659629210017_4904 failed with state FAILED due to: Task failed task_1659629210017_4904_m_000000 只是当初没有注意到罢了,部分详细错误日志详见文末
2022-10-08 16:34:58,417 ERROR [Thread-11] org.apache.sqoop.mapreduce.AsyncSqlOutputFormat: Got exception in update thread: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'receiveDiscordIds' at row 1
15-09-2022 09:22:31 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:31 INFO mapreduce.Job: map 0% reduce 0% 15-09-2022 09:22:43 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:43 INFO mapreduce.Job: map 100% reduce 0% 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 INFO mapreduce.Job: Job job_1659629210017_4904 failed with state FAILED due to: Task failed task_1659629210017_4904_m_000000 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Job failed as tasks failed. failedMaps:1 failedReduces:0 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 INFO mapreduce.Job: Counters: 8 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Job Counters 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Failed map tasks=1 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Launched map tasks=1 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Rack-local map tasks=1 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Total time spent by all maps in occupied slots (ms)=469680 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Total time spent by all reduces in occupied slots (ms)=0 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Total time spent by all map tasks (ms)=9785 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Total vcore-milliseconds taken by all map tasks=9785 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Total megabyte-milliseconds taken by all map tasks=480952320 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 46.0112 seconds (0 bytes/sec) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 INFO mapreduce.ExportJobBase: Exported 0 records. 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 ERROR mapreduce.ExportJobBase: Export job failed! 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - 22/09/15 09:22:44 ERROR tool.ExportTool: Error during export: 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - Export job failed! 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.Sqoop.run(Sqoop.java:147) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) 15-09-2022 09:22:44 CST export_dmr_deceitful_delivery INFO - at org.apache.sqoop.Sqoop.main(Sqoop.java:252)