分享

请教kettle问题

justinyao 发表于 2015-3-24 15:05:42 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 3 59387
本帖最后由 justinyao 于 2015-3-24 15:07 编辑

环境:hadoop:cdh5.2
kettle:5.2


使用kettle的sqoop import报错

2015/03/24 14:38:32 - Spoon - 正在开始任务...
2015/03/24 14:38:36 - sqoop_import - 开始执行任务
2015/03/24 14:38:36 - sqoop_import - 开始项[Sqoop Import]
2015/03/24 14:38:36 - Sqoop Import - 2015/03/24 14:38:36 - fs.default.name is deprecated. Instead, use fs.defaultFS
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Running Sqoop version: 1.4.5-cdh5.2.0
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Setting your password on the command-line is insecure. Consider using -P instead.
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Data Connector for Oracle and Hadoop is disabled.
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Using default fetchSize of 1000
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Beginning code generation
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Time zone has been set to GMT
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - Executing SQL statement: SELECT t.* FROM t_cs_bu_call t WHERE 1=0
2015/03/24 14:38:37 - Sqoop Import - 2015/03/24 14:38:37 - $HADOOP_MAPRED_HOME is not set
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - Writing jar file: \tmp\sqoop-ly-yaojx\compile\b2c7234812d7592c349294f6ccf763d3\t_cs_bu_call.jar
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - Time zone has been set to GMT
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - Time zone has been set to GMT
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - Beginning import of t_cs_bu_call
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - mapred.jar is deprecated. Instead, use mapreduce.job.jar
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - Time zone has been set to GMT
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2015/03/24 14:38:38 - Sqoop Import - 2015/03/24 14:38:38 - SQOOP_HOME is unset. May not be able to find all job dependencies.
2015/03/24 14:38:39 - Sqoop Import - 2015/03/24 14:38:39 - Connecting to ResourceManager at ly-hadoop-02/172.26.156.12:8032
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - Using read commited transaction isolation
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - number of splits:1
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - fs.default.name is deprecated. Instead, use fs.defaultFS
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - Submitting tokens for job: job_1422855756341_35367
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - Submitted application application_1422855756341_35367
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - The url to track the job: http://ly-hadoop-02:8088/proxy/application_1422855756341_35367/
2015/03/24 14:38:41 - Sqoop Import - 2015/03/24 14:38:41 - Running job: job_1422855756341_35367
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 - Job job_1422855756341_35367 running in uber mode : false
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 -  map 0% reduce 0%
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 - Job job_1422855756341_35367 failed with state FAILED due to: Application application_1422855756341_35367 failed 2 times due to AM Container for appattempt_1422855756341_35367_000002 exited with  exitCode: 1 due to: Exception from container-launch.
Container id: container_1422855756341_35367_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 - Counters: 0
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 - Transferred 0 bytes in 27.2546 seconds (0 bytes/sec)
2015/03/24 14:39:06 - Sqoop Import - 2015/03/24 14:39:06 - Retrieved 0 records.
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 - Error during import: Import job failed!
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 - Got exception running Sqoop: java.lang.RuntimeException: Import job failed!
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 - java.lang.RuntimeException: Import job failed!
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:623)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.common.CommonSqoopShim.runTool(CommonSqoopShim.java:44)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.access$001(ClassPathModifyingSqoopShim.java:41)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim$1.call(ClassPathModifyingSqoopShim.java:80)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim$1.call(ClassPathModifyingSqoopShim.java:77)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.runWithModifiedClassPathProperty(ClassPathModifyingSqoopShim.java:64)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.runTool(ClassPathModifyingSqoopShim.java:77)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.hadoop.shim.cdh51.delegating.DelegatingSqoopShim.runTool(DelegatingSqoopShim.java:41)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.di.job.entries.sqoop.AbstractSqoopJobEntry.executeSqoop(AbstractSqoopJobEntry.java:291)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.pentaho.di.job.entries.sqoop.AbstractSqoopJobEntry$1.run(AbstractSqoopJobEntry.java:256)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at java.lang.Thread.run(Thread.java:745)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 - Caused by: Import job failed!
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:665)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:385)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : 2015/03/24 14:39:06 -         ... 15 more
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Error running Sqoop
2015/03/24 14:39:06 - Sqoop Import - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: Import job failed!
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.runWithModifiedClassPathProperty(ClassPathModifyingSqoopShim.java:67)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.runTool(ClassPathModifyingSqoopShim.java:77)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.delegating.DelegatingSqoopShim.runTool(DelegatingSqoopShim.java:41)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.di.job.entries.sqoop.AbstractSqoopJobEntry.executeSqoop(AbstractSqoopJobEntry.java:291)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.di.job.entries.sqoop.AbstractSqoopJobEntry$1.run(AbstractSqoopJobEntry.java:256)
2015/03/24 14:39:06 - Sqoop Import -         at java.lang.Thread.run(Thread.java:745)
2015/03/24 14:39:06 - Sqoop Import - Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Import job failed!
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
2015/03/24 14:39:06 - Sqoop Import -         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:49)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.common.CommonSqoopShim.runTool(CommonSqoopShim.java:44)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.access$001(ClassPathModifyingSqoopShim.java:41)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim$1.call(ClassPathModifyingSqoopShim.java:80)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim$1.call(ClassPathModifyingSqoopShim.java:77)
2015/03/24 14:39:06 - Sqoop Import -         at org.pentaho.hadoop.shim.cdh51.ClassPathModifyingSqoopShim.runWithModifiedClassPathProperty(ClassPathModifyingSqoopShim.java:64)
2015/03/24 14:39:06 - Sqoop Import -         ... 5 more
2015/03/24 14:39:06 - Sqoop Import - Caused by: java.lang.RuntimeException: Import job failed!
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:623)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
2015/03/24 14:39:06 - Sqoop Import -         ... 12 more
2015/03/24 14:39:06 - Sqoop Import - Caused by: Import job failed!
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:665)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:385)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
2015/03/24 14:39:06 - Sqoop Import -         at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
2015/03/24 14:39:06 - Sqoop Import -         ... 15 more
2015/03/24 14:39:06 - sqoop_import - 完成作业项[Sqoop Import] (结果=[false])
2015/03/24 14:39:06 - sqoop_import - 任务执行完毕
2015/03/24 14:39:06 - Spoon - 任务已经结束.



请问是什么原因呢

已有(3)人评论

跳转到指定楼层
jixianqiuxue 发表于 2015-3-25 09:41:43
这些设置下
$SQOOP_CONF_DIR  
Data Connector for Oracle and Hadoop is disabled.
$HADOOP_MAPRED_HOME
SQOOP_HOME

回复

使用道具 举报

seer1025 发表于 2016-12-4 00:22:27
请问大神们,怎样在kettle中使用sqoop语句啊?
回复

使用道具 举报

碎碎冰 发表于 2018-3-30 16:45:26
大神没kettle不支持CDH5.8吗?死活没找到。求解啊
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条