分享

hue中使用oozie执行sqoop job将oracle中的数据增量导入hbase

玩第八 发表于 2016-11-23 18:10:32 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 2 13632
在cdh hue中使用oozie执行sqoop job时遇到下面的问题。
sqoop命令

job --meta-connect jdbc:hsqldb:hsql://bigdata02:16000/sqoop  --exec UQE_DIM_DS_DJ_NSRXX

在服务器shell中执行该命令时运行正常

在oozie中执行时报错

t in the environment. Cannot check for additional configuration.3710 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.oracle.OraOopManagerFactory  - Data Connector for Oracle and Hadoop is disabled.3722 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.SqlManager  - Using default fetchSize of 10003722 [uber-SubtaskRunner] INFO  org.apache.sqoop.tool.CodeGenTool  - Beginning code generation4020 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.OracleManager  - Time zone has been set to GMT4073 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: select TO_CHAR(SYSDATE, 'yyyy-mm-dd') STARTIME, '9999-01-01' ENDTIME, TO_CHAR(SYSDATE, 'yyyy') YEARS, '654000' XZQH, '1' VERSION, TO_CHAR(DJXH) DJXH, GDSLX_DM, SSDABH, NSRSBH, NSRMC, KZZTDJLX_DM, DJZCLX_DM, FDDBRXM, FDDBRSFZJLX_DM, SCJYDZ, FDDBRSFZJHM, SCJYDZXZQHSZ_DM, NSRZT_DM, HY_DM, ZCDZ, ZCDZXZQHSZ_DM, JDXZ_DM, DWLSGX_DM, GDGHLX_DM, DJJG_DM, XGR_DM, ZZJG_DM, KQCCSZTDJBZ, LRR_DM, LRRQ, TO_CHAR(xgrq, 'yyyy-mm-dd  hh24:mi:ss') XGRQ, TO_CHAR(sjtb_sj, 'yyyy-mm-dd  hh24:mi:ss') SJTB_SJ, SJGSDQ, ZGSWJ_DM, ZGSWSKFJ_DM, SSGLY_DM, FJMQYBZ, SWDJBLBZ, NSRBM, YXBZ, SHXYDM, PGJG_DM, TO_CHAR(GSZXRQ, 'yyyy-mm-dd') GSZXRQ, TO_CHAR(DJRQ, 'yyyy-mm-dd') DJRQ from DJ_NSRXX  where  (1 = 0) 4080 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: select TO_CHAR(SYSDATE, 'yyyy-mm-dd') STARTIME, '9999-01-01' ENDTIME, TO_CHAR(SYSDATE, 'yyyy') YEARS, '654000' XZQH, '1' VERSION, TO_CHAR(DJXH) DJXH, GDSLX_DM, SSDABH, NSRSBH, NSRMC, KZZTDJLX_DM, DJZCLX_DM, FDDBRXM, FDDBRSFZJLX_DM, SCJYDZ, FDDBRSFZJHM, SCJYDZXZQHSZ_DM, NSRZT_DM, HY_DM, ZCDZ, ZCDZXZQHSZ_DM, JDXZ_DM, DWLSGX_DM, GDGHLX_DM, DJJG_DM, XGR_DM, ZZJG_DM, KQCCSZTDJBZ, LRR_DM, LRRQ, TO_CHAR(xgrq, 'yyyy-mm-dd  hh24:mi:ss') XGRQ, TO_CHAR(sjtb_sj, 'yyyy-mm-dd  hh24:mi:ss') SJTB_SJ, SJGSDQ, ZGSWJ_DM, ZGSWSKFJ_DM, SSGLY_DM, FJMQYBZ, SWDJBLBZ, NSRBM, YXBZ, SHXYDM, PGJG_DM, TO_CHAR(GSZXRQ, 'yyyy-mm-dd') GSZXRQ, TO_CHAR(DJRQ, 'yyyy-mm-dd') DJRQ from DJ_NSRXX  where  (1 = 0) 4093 [uber-SubtaskRunner] INFO  org.apache.sqoop.orm.CompilationManager  - $HADOOP_MAPRED_HOME is not set5594 [uber-SubtaskRunner] INFO  org.apache.sqoop.orm.CompilationManager  - Writing jar file: /tmp/sqoop-yarn/compile/4c4a96a4394eb993738adcf661868440/QueryResult.jar5605 [uber-SubtaskRunner] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: select TO_CHAR(SYSDATE, 'yyyy-mm-dd') STARTIME, '9999-01-01' ENDTIME, TO_CHAR(SYSDATE, 'yyyy') YEARS, '654000' XZQH, '1' VERSION, TO_CHAR(DJXH) DJXH, GDSLX_DM, SSDABH, NSRSBH, NSRMC, KZZTDJLX_DM, DJZCLX_DM, FDDBRXM, FDDBRSFZJLX_DM, SCJYDZ, FDDBRSFZJHM, SCJYDZXZQHSZ_DM, NSRZT_DM, HY_DM, ZCDZ, ZCDZXZQHSZ_DM, JDXZ_DM, DWLSGX_DM, GDGHLX_DM, DJJG_DM, XGR_DM, ZZJG_DM, KQCCSZTDJBZ, LRR_DM, LRRQ, TO_CHAR(xgrq, 'yyyy-mm-dd  hh24:mi:ss') XGRQ, TO_CHAR(sjtb_sj, 'yyyy-mm-dd  hh24:mi:ss') SJTB_SJ, SJGSDQ, ZGSWJ_DM, ZGSWSKFJ_DM, SSGLY_DM, FJMQYBZ, SWDJBLBZ, NSRBM, YXBZ, SHXYDM, PGJG_DM, TO_CHAR(GSZXRQ, 'yyyy-mm-dd') GSZXRQ, TO_CHAR(DJRQ, 'yyyy-mm-dd') DJRQ from DJ_NSRXX  where  (1 = 0) 5610 [uber-SubtaskRunner] INFO  org.apache.sqoop.tool.ImportTool  - Incremental import based on column LRRQ5610 [uber-SubtaskRunner] INFO  org.apache.sqoop.tool.ImportTool  - Lower bound value: TO_TIMESTAMP('2016-11-23 17:36:22.0', 'YYYY-MM-DD HH24:MI:SS.FF')5610 [uber-SubtaskRunner] INFO  org.apache.sqoop.tool.ImportTool  - Upper bound value: TO_TIMESTAMP('2016-11-23 17:46:00.0', 'YYYY-MM-DD HH24:MI:SS.FF')5614 [uber-SubtaskRunner] ERROR org.apache.sqoop.tool.ImportTool  - Error during import: HBase jars are not present in classpath, cannot import to HBase!Intercepting System.exit(1)
从日志上看是说hbase的jar包没有在classpath下不能导入到hbase,但是配置文件中是有hbase的配置的#set the path to where bin/hbase is availableexport HBASE_HOME=${HBASE_HOME:-/usr/lib/hbase}

不知道有人碰到过这个问题没有,如果有碰到的请指点一下,感激不尽!!!!

已有(2)人评论

跳转到指定楼层
starrycheng 发表于 2016-11-23 18:49:48
classpath是指Linux的环境变量,而且应该是lib文件夹也就是包含jar包的文件夹,最好都配置。然后记得生效
回复

使用道具 举报

babyxingqing 发表于 2017-2-10 17:02:31
我也遇到过这个问题,请问楼主解决没有,谢谢
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条