分享

Eclipse上传文件到HDFS为空

Kevin517 发表于 2016-10-15 20:23:31 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 16 35618
本帖最后由 Kevin517 于 2016-10-15 20:30 编辑

问题说明:在 Eclipse 中 Upload files to DFS ,上传成功后,在 eclipse 中打开文件,为空。运行 单词统计案例,就没有结果。
环境说明:

采用云平台中的两台云主机搭建的 master 和 slaver 作为 Namenode 和 Datanode 集群。
云平台使用 GRE 网络。内网为 10.0.0.0/24;外网为192.168.200.0/24
两个节点通过 内网 (10.0.0.0/24 )通信。windows通过云主机的 floatIP (192.168.200.0/24)连接。

Hadoop 版本为 Hadoop 2.7.0
IP如下:
master 192.168.200.105 10.0.0.7
slaver 192.168.200.106 10.0.0.8
windows 7 (Kevin) 192.168.200.5



集群使用 root 用户搭建,以 root 用户启动 hadoop
集群启动正常,两个节点进程启动成功,日志无错误输出。

windows上访问 192.168.200.105:50070,网页正常打开。检查 namenode 和 datanode 信息正常。
windows上访问 192.168.200.105::8088,网页打开正常。检查 Yarn 正常。

在 master 对 HDFS 操作正常。
并且执行  
[size=13.3333px][root@master mapreduce]# hadoop jar hadoop-mapreduce-examples-2.7.0.jar wordcount /input /output可以正常运行,正常输出。

MapReduce

MapReduce

================================================================

问题描述:

windows  上 Eclipse 搭建Hadoop开发环境成功。
但是在创建成功 input 文件夹后,上传文件,也能上传成功。就是成功后,文件没有信息。
然后运行案例。 控制台正常输出运行过程信息。没有报错。也输出了统计结果。
因为上传文件没有内容,所以统计结果也没有信息。

WordCount

WordCount

上面截图是运行成功截图。


======================================================

下面贴出 重新上传到 input/ 目录的时候  namenode 的日志 :hadoop-root-namenode-master.log

两个时间
以下,是运行案例的日志。

2016-10-15 07:52:49,692 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:05,544 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:05,545 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getBlockLocations from 192.168.100.8:8819 Call#7 Retry#0: java.io.FileNotFoundException: File does not exist: /input/wordtest.txt
2016-10-15 07:59:17,400 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:18,082 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:18,084 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:19,813 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:30,958 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:31,146 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:31,168 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:31,439 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:31,440 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 16 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 10 SyncTimes(ms): 412
2016-10-15 07:59:31,790 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:31,840 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,028 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,074 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,084 INFO org.apache.hadoop.hdfs.StateChange: DIR* completeFile: /output/_temporary/0/_temporary/attempt_local29653720_0001_r_000000_0/part-r-00000 is closed by DFSClient_NONMAPREDUCE_-1631979872_1
2016-10-15 07:59:32,088 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,092 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,093 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,096 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,110 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,112 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,113 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,115 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,118 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,130 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,138 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,145 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:32,151 INFO org.apache.hadoop.hdfs.StateChange: DIR* completeFile: /output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-1631979872_1
2016-10-15 07:59:42,822 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,665 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,667 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,667 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,668 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,668 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,669 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,669 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,670 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,671 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 07:59:43,672 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:00:26,501 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:00:28,220 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin

以下,是重新上传文件的日志。

2016-10-15 08:04:32,059 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:04:32,060 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 26 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 18 SyncTimes(ms): 482
2016-10-15 08:04:32,099 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:04:39,188 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:04:39,214 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:04:39,215 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocate blk_1073741843_1019{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-20957220-9bf2-463a-88aa-88ed2446fd42:NORMAL:10.0.0.7:50010|RBW], ReplicaUC[[DISK]DS-a461d256-df88-473f-b0f8-08c7221a794b:NORMAL:10.0.0.8:50010|RBW]]} for /input/word.txt
2016-10-15 08:05:00,252 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:05:00,280 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:05:00,281 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 1 to reach 2 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2016-10-15 08:05:00,281 WARN org.apache.hadoop.hdfs.protocol.BlockStoragePolicy: Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=2, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2016-10-15 08:05:00,281 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 1 to reach 2 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2016-10-15 08:05:00,282 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocate blk_1073741844_1020{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a461d256-df88-473f-b0f8-08c7221a794b:NORMAL:10.0.0.8:50010|RBW]]} for /input/word.txt
2016-10-15 08:05:21,298 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:05:21,320 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
2016-10-15 08:05:21,321 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 2 to reach 2 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2016-10-15 08:05:21,321 WARN org.apache.hadoop.hdfs.protocol.BlockStoragePolicy: Failed to place enough replicas: expected size is 2 but only 0 storage types can be selected (replication=2, selected=[], unavailable=[DISK], removed=[DISK, DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2016-10-15 08:05:21,321 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 2 to reach 2 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2016-10-15 08:05:21,321 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.100.8:9029 Call#33 Retry#0
java.io.IOException: File /input/word.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and 2 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1550)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3067)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:722)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

[root@master logs]#


下面是重新上传后查看 HDFS 目录

[root@master ~]# hdfs dfs -ls /
Found 1 items
drwxr-xr-x   - Kevin supergroup          0 2016-10-15 08:23 /input

[root@master ~]# hdfs dfs -ls /input
Found 1 items
-rw-r--r--   3 Kevin supergroup          0 2016-10-15 08:23 /input/word.txt
[root@master ~]#



===================================================



希望,能帮忙分析以下。万分感谢。



已有(16)人评论

跳转到指定楼层
arsenduan 发表于 2016-10-15 21:08:11
[root@master ~]# hdfs dfs -ls /
Found 1 items
drwxr-xr-x   - Kevin supergroup          0 2016-10-15 08:23 /input

[root@master ~]# hdfs dfs -ls /input
Found 1 items
-rw-r--r--   3 Kevin supergroup          0 2016-10-15 08:23 /input/word.txt
[root@master ~]#

应该是word.txt权限不够,给他授权下。应该是权限不够,导致读取失败

回复

使用道具 举报

easthome001 发表于 2016-10-16 08:03:45
File /input/word.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and 2 node(s) are excluded in this operation.

这里说明,你的datanode可能有挂掉的,到各个节点上看看,进程是否还在,是否有僵尸进程。最好重启下
回复

使用道具 举报

easthome001 发表于 2016-10-16 08:10:07
方法一:
在core-site.xml里面添加如下配置:

<property>  
    <name>hadoop.http.staticuser.user</name>  
    <value>Kevin</value>  
</property>  

另外一个方法,在当前的用户组里面添加一个dr.who的用户
gpasswd -a dr.who hadoop
将dr.who用户添加到hadoop用户组里面


如此就能通过linux的权限验证,改完之后,重启hadoop集群之后,就不会出现这样的警告了

回复

使用道具 举报

qcbb001 发表于 2016-10-16 08:12:00
hdfs-site.xml
里面


是否设置为false了
回复

使用道具 举报

Kevin517 发表于 2016-10-16 08:38:49
easthome001 发表于 2016-10-16 08:03
File /input/word.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are  ...

datanode应该不会出问题,要不然我在 namenode 节点用命令行 mkdir ,put 文件,执行# hadoop jar hadoop-mapreduce-examples-2.7.0.jar wordcount /input /output应该不会成功。但是做成功了
回复

使用道具 举报

Kevin517 发表于 2016-10-16 08:41:46

设置过的

<configuration>
<property>
  <name>dfs.namenode.name.dir</name>
  <value>file:///home/hadoop/dfs/name</value>
</property>
<property>
  <name>dfs.datanode.data.dir</name>
  <value>file:///home/hadoop/dfs/data</value>
</property>
<property>
  <name>dfs.replication</name>
  <value>3</value>
</property>
<property>
  <name>dfs.namenode.secondary.http-address</name>
  <value>master:50090</value>
</property>
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>
<property>
  <name>dfs.permissions</name>
  <value>false</value>
</property>
</configuration>

回复

使用道具 举报

Kevin517 发表于 2016-10-16 09:14:37
easthome001 发表于 2016-10-16 08:10
方法一:
在core-site.xml里面添加如下配置:

你好,方法一,我试了,不行。

然后我在 root 组里面同时添加了 dr.who 用户和 Kevin 用户。然后重新上传晚间。

namenode的日志如下

2016-10-15 21:13:09,688 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 23 Total time for transactions(ms): 105 Number of transactions batched in Syncs: 0 Number of syncs: 15 SyncTimes(ms): 193
2016-10-15 21:13:23,056 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocate blk_1073741829_1005{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-7e695b37-0909-4e77-9692-8b937f8b02fa:NORMAL:10.0.0.8:50010|RBW], ReplicaUC[[DISK]DS-5c3e0bf7-72fc-4afa-91ce-9e2b73cf1d51:NORMAL:10.0.0.7:50010|RBW]]} for /input/word.txt
2016-10-15 21:13:44,079 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 1 to reach 2 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2016-10-15 21:13:44,079 WARN org.apache.hadoop.hdfs.protocol.BlockStoragePolicy: Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=2, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2016-10-15 21:13:44,079 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 1 to reach 2 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2016-10-15 21:13:44,079 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocate blk_1073741830_1006{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5c3e0bf7-72fc-4afa-91ce-9e2b73cf1d51:NORMAL:10.0.0.7:50010|RBW]]} for /input/word.txt
2016-10-15 21:14:05,110 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 2 to reach 2 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2016-10-15 21:14:05,110 WARN org.apache.hadoop.hdfs.protocol.BlockStoragePolicy: Failed to place enough replicas: expected size is 2 but only 0 storage types can be selected (replication=2, selected=[], unavailable=[DISK], removed=[DISK, DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2016-10-15 21:14:05,111 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 2 to reach 2 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2016-10-15 21:14:05,111 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9000, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.100.8:15557 Call#1748 Retry#0
java.io.IOException: File /input/word.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are 2 datanode(s) running and 2 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1550)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3067)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:722)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)


虽然没有出现 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user Kevin
但,还是有上面的那个错误。

回复

使用道具 举报

Kevin517 发表于 2016-10-16 09:15:41
arsenduan 发表于 2016-10-15 21:08
[root@master ~]# hdfs dfs -ls /
Found 1 items
drwxr-xr-x   - Kevin supergroup          0 2016-10-1 ...

我把 input/ 目录授予  777 权限上传还是一样问题。
回复

使用道具 举报

qcbb001 发表于 2016-10-16 10:00:27
本帖最后由 qcbb001 于 2016-10-16 10:14 编辑
Kevin517 发表于 2016-10-16 08:38
datanode应该不会出问题,要不然我在 namenode 节点用命令行 mkdir ,put 文件,执行# hadoop jar hadoop- ...

应该是easthome001所说,
第二次出现的还是这个问题。
很可能有的datanode挂掉了
回复

使用道具 举报

12下一页
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条