咨询大家一个很奇怪的问题i,我现在正在安装一套hadoop平台,在安装完Yarn HA环境之后,启动yarn resourcemanager 服务没有问题,启动nodemanager服务的时候,系统显示服务可以启动,然而nodemanger启动日志报这个错误:
2017-11-05 20:54:39,660 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /hdfs/data1/yarn_file/local error mkdir of /hdfs/data1/yarn_file failed, removing from the list of valid directories.
2017-11-05 20:54:39,661 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /hdfs/data2/yarn_file/local error mkdir of /hdfs/data2/yarn_file failed, removing from the list of valid directories.
2017-11-05 20:54:39,661 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /hdfs/data1/yarn_file/log error mkdir of /hdfs/data1/yarn_file failed, removing from the list of valid directories.
2017-11-05 20:54:39,662 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Unable to create directory /hdfs/data2/yarn_file/log error mkdir of /hdfs/data2/yarn_file failed, removing from the list of valid directories.
2017-11-05 20:54:39,662 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 2/2 local-dirs are bad: /hdfs/data1/yarn_file/local,/hdfs/data2/yarn_file/local; 2/2 log-dirs are bad: /hdfs/data1/yarn_file/log,/hdfs/data2/yarn_file/log
同时在网页里发现这些nodemanager节点状态都是unhealthy的状态,点开之后还是报 .../yarn_file/local 和.../yarn_file/log 目录错误的问题,报错和上面一样
我每一个节点上在启动前均创建了与之相对应的log和yarn目录,权限我尝试按yarn:yarn 775 给了,也尝试按777赋予了相应权限,但是nodemanager在启动过程中还是报这个错误。我是新创建集群,所以不存在这几个文件夹剩余空间少的问题,请问这个问题怎么解?
|