分享

hbase协处理器统计行数

利用hbase的协处理器统计行数,抛出异常,信息如下:java.io.IOException: java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1200209: Call to dataserver1/192.168.21.29:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=45, waitTime=599814, rpcTimeout=599808 row '20120701000022ws10fhjsae4de5903ed14af79c262179c13b4681' on table 'point' at region=point,20120701000022ws10fhjsae4de5903ed14af79c262179c13b4681,1551167443768.aeb0245aea508ecbcf7ad7b3f0b3c0d0., hostname=dataserver1,16020,1551337949226, seqNum=1819
        at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:159)
        at org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:91)
        at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callMethod(SyncCoprocessorRpcChannel.java:52)
        at org.apache.hadoop.hbase.protobuf.generated.AggregateProtos$AggregateService$Stub.getRowNum(AggregateProtos.java:2113)
        at org.apache.hadoop.hbase.client.coprocessor.AggregationClient$3.call(AggregationClient.java:357)
        at org.apache.hadoop.hbase.client.coprocessor.AggregationClient$3.call(AggregationClient.java:351)
        at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Call to dataserver1/192.168.21.29:16020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=45, waitTime=599814, rpcTimeout=599808
        at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:185)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
        at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:96)
        at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:199)
        at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:663)
        at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:738)
        at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:466)
        ... 1 more
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=45, waitTime=599814, rpcTimeout=599808
        at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:200)
        ... 4 more

        at org.apache.hadoop.hbase.client.coprocessor.AggregationClient$3.call(AggregationClient.java:360)
        at org.apache.hadoop.hbase.client.coprocessor.AggregationClient$3.call(AggregationClient.java:351)
        at org.apache.hadoop.hbase.client.HTable$12.call(HTable.java:1012)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

统计2千万条出现上面错误信息,求解。。。。

已有(2)人评论

跳转到指定楼层
jasligea 发表于 2019-3-1 15:31:38
已解决,主要是由于超时引起的   CallTimeoutException
可以参考 https://blog.csdn.net/xuexi_39/article/details/88059925
回复

使用道具 举报

bioger_hit 发表于 2019-3-3 11:02:34
jasligea 发表于 2019-3-1 15:31
已解决,主要是由于超时引起的   CallTimeoutException
可以参考 https://blog.csdn.net/xuexi_39/article ...

嗯,很多时候都是超时造成的。
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条