立即注册 登录
About云-梭伦科技 返回首页

qcbb001的个人空间 https://www.aboutyun.com/?1399 [收藏] [复制] [分享] [RSS]

日志

不能将pig关系存到Hbase中问题原因

已有 1230 次阅读2016-12-22 15:58

我试着将pig关系存到HBase中。
store result INTO 'hbase://hourlyAggregation' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('countDetails:ansCount countDetails:divCount countDetails:unansCount countDetails:engCount');

它在本地运行良好。当我试着在映射规约模式中运行pig时,该任务失败了,但是日志没有显示任何错误。


ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2244: Job failed, hadoop does not return any error message
Details at logfile: /home/HadoopUser/pig_1384412383791.log
  
Pig Stack Trace
---------------
ERROR 2244: Job failed, hadoop does not return any error message
  
org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed, hadoop does not return any error message
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:119)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:172)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
    at org.apache.pig.Main.run(Main.java:500)
    at org.apache.pig.Main.main(Main.java:107)

结构如下所示:

export JAVA_HOME=/home/hadoop/jdk1.6.0_39
export HADOOP_HOME=$MY_HOME/hadoop-0.20.2-cdh3u4
export CLASSPATH=$JAVA_HOME/lib/tools.jar:.
export HIVE_HOME=$MY_HOME/hive-0.7.1-cdh3u4
export PIG_HOME=$MY_HOME/pig-0.8.1-cdh3u4
export HBASE_HOME=$MY_HOME/hbase-0.90.6-cdh3u4
export PIG_CLASSPATH=”`${HBASE_HOME}/bin/hbase classpath`:$PIG_CLASSPATH”

我试着在pig_home/lib中注册zookeeper和hbase的jar。

JT日志:
14-Nov-2013 14:48:29 (17sec)
  
  
java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[countDetails:ansCount countDetails:divCount countDetails:unansCount countDetails:engCount]'
    at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:218)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:85)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:68)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:278)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:511)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
</init>
解决方案
在PIG/lib中注册hbase,zookeeper jar,在hbase lib中注册guava jar。











路过

雷人

握手

鲜花

鸡蛋

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 立即注册

关闭

推荐上一条 /2 下一条