立即注册 登录
About云-梭伦科技 返回首页

desehawk的个人空间 https://www.aboutyun.com/?29 [收藏] [复制] [分享] [RSS]

日志

flume1.5.0.1+hbase0.98.7+hadoop1.2.1遇到的奇怪问题。

已有 2612 次阅读2014-11-19 05:22

如题: 
hdfs:hadoop1.2.1搭建的 
hbase:0.98.7版本 
flume:1.5.0.1版本 
flume拿到日志sink到hbase中去,遇到的问题是这样的: 
往表中存入100条数据以后,就开始报错,是flume爆出来的: 

2014-11-01 11:18:35,168 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:353)] Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
2014-11-01 11:18:35,170 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:356)] Failed to commit transaction.Transaction rolled back.
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662)
我翻阅了flume1.5.0.1和hbase0.98.7版本的源代码以后发现确实是由hbase的0.98.7版本的Increment没有setWriteToWAL方法,这种情况如何破? 


找到了问题的替代解决方法: 
之前的flume配置是: 
agent0.sinks.log-sink0.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer 
替换成: 
agent0.sinks.log-sink0.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer 
不知道是不是SimpleHbaseEventSerializer的bug(不确定),flume1.5.0.1的Simple***lizer在所引用的hbase版本中不存在的方法。不过这个类在实际使用中应该都会自己重写。 
此处路略过了。

路过

雷人

握手

鲜花

鸡蛋

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 立即注册

关闭

推荐上一条 /2 下一条