分享

求助 sqoop导入的时候报错,请各位大神帮忙看下

credit 发表于 2016-3-17 10:27:12 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 4 12914
各位大神,谁见过这个错:16/03/17 10:16:37 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hadoop-yarn/staging/credit/.staging/job_1458179202514_0001/libjars/libthrift-0.9.3.jar could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1549)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3200)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:641)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)

        at org.apache.hadoop.ipc.Client.call(Client.java:1468)
        at org.apache.hadoop.ipc.Client.call(Client.java:1399)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
        at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:399)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
        at com.sun.proxy.$Proxy12.addBlock(Unknown Source)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1532)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1349)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588)


已有(4)人评论

跳转到指定楼层
s060403072 发表于 2016-3-17 10:44:54
应该是datanode坏掉了,确保hadoop 进程都正常,特别是僵尸进程。最好重启下集群
回复

使用道具 举报

credit 发表于 2016-3-17 11:09:32
s060403072 发表于 2016-3-17 10:44
应该是datanode坏掉了,确保hadoop 进程都正常,特别是僵尸进程。最好重启下集群

搭建的伪分布式的,清理了一下namenode 格式化了一下 重启了,但是sqoop不能用了,显示:/usr/lib/hcatalog does not exist hcatalog jobs will fail
回复

使用道具 举报

s060403072 发表于 2016-3-17 11:24:58
credit 发表于 2016-3-17 11:09
搭建的伪分布式的,清理了一下namenode 格式化了一下 重启了,但是sqoop不能用了,显示:/usr/lib/hcatal ...

使用sqoop 1.4.4从mysql向hdfs导入表出现链接数据库失败错误的解决办法
http://www.aboutyun.com/forum.php?mod=viewthread&tid=9302



回复

使用道具 举报

credit 发表于 2016-3-17 12:09:13
s060403072 发表于 2016-3-17 11:24
使用sqoop 1.4.4从mysql向hdfs导入表出现链接数据库失败错误的解决办法
http://www.aboutyun.com/forum. ...

显示的这个错误
Warning: /usr/lib/hcatalog does not exist! HCatalog jobswill fail.
Please set $HCAT_HOME to the root of your HCataloginstallation.
16/03/17 12:05:50 INFO sqoop.Sqoop: Running Sqoopversion: 1.4.5-cdh5.4.3
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Error parsingarguments for list-tables:
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: —connect
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: jdbc:mysql://localhost:3306/hive
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: ——username
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: hive
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: ——password
16/03/17 12:05:50 ERROR tool.BaseSqoopTool: Unrecognizedargument: hive

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条