启动Hbase时，Hmaster 启动后又挂了

各位大侠好：
        启动Hbase 时，用jps看进程Hmaster是启动的，后面一会就又挂了，日志说，什么要选举出一台Hmaster,等等选举失败，网上说，应该是网络访问问题或，文件创建权限问题，文件权限我看了，都是Hadoop用户所属，应该不会是这个，网络权限修改了，碰到一个问题，就是修改 vi /etc/hosts时，把以下注释掉，后面又自动生成了，不懂为什么，重启也没有用。
127.0.0.1       localhost.localdomain   localhost
::1     Salve2.Hadoop   Salve2  localhost6.localdomain6 localhost6



master 报错信息：


2015-05-15 12:38:48,944 INFO  [main] util.ServerCommandLine: env:HOSTNAME=Master.Hadoop
"/usr/hbase/logs/hbase-hadoop-master-Master.Hadoop.log" 430L, 90412C
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2015-05-16 02:22:47,708 WARN  [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=192.168.186.101:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
2015-05-16 02:22:47,709 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts
2015-05-16 02:22:47,709 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1988)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:203)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2002)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:512)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:491)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1252)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1230)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:174)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:167)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:531)
        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1983)
        ... 5 more
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2015-05-16 02:22:44,595 INFO  [main-SendThread(192.168.186.101:2181)] zookeeper.ClientCnxn: Opening socket connection to server 192.168.186.101/192.168.186.101:2181. Will not attempt to authenticate using SASL (unknown error)
2015-05-16 02:22:47,607 WARN  [main-SendThread(192.168.186.101:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2015-05-16 02:22:47,708 WARN  [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=192.168.186.101:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
2015-05-16 02:22:47,709 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts
2015-05-16 02:22:47,709 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1988)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:203)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2002)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:512)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:491)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1252)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1230)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:174)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:167)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:531)
        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1983)
        ... 5 more
2015-05-16 02:22:43,493 WARN  [main-SendThread(192.168.186.101:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2015-05-16 02:22:44,595 INFO  [main-SendThread(192.168.186.101:2181)] zookeeper.ClientCnxn: Opening socket connection to server 192.168.186.101/192.168.186.101:2181. Will not attempt to authenticate using SASL (unknown error)
2015-05-16 02:22:47,607 WARN  [main-SendThread(192.168.186.101:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2015-05-16 02:22:47,708 WARN  [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=192.168.186.101:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
2015-05-16 02:22:47,709 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts
2015-05-16 02:22:47,709 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1988)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:203)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2002)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:512)
        at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:491)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1252)
        at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1230)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:174)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:167)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:531)
        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1983)
        ... 5 more

对了，有没有可能是linux对 IP  或着是hadoop与hbase 会缓存原来设置的ip，带我重启后自己把还原会，如果是这样子，要怎么处理呢？

aqi915 发表于 2015-5-16 15:53
对了，有没有可能是linux对 IP  或着是hadoop与hbase 会缓存原来设置的ip，带我重启后自己把还原会，如果是 ...

还原？这个还真没见过，尽量配置简单。这种形式，hosts配置正确是第一关，还有配置文件中，也有相关hosts的hostname引用等。这些一定注意

127.0.0.1 localhost

看看是否设置ntp时间同步

NIITYZU 发表于 2015-5-17 10:52
看看是否设置ntp时间同步

额，是的，我做了之前做了个任务计划，时间同步的，这个对什么有影响呢？


补充内容 (2015-5-17 20:39):
额，，那日志不会报错么？都没看到，让我很奇怪的说，

xuanxufeng 发表于 2015-5-16 16:54
还原？这个还真没见过，尽量配置简单。这种形式，hosts配置正确是第一关，还有配置文件中，也有相关hosts ...

有个奇怪的事：

改vi /etc/hosts为下面时，启动hadoop,namenode可以起来，DataNode,起不了，NodeManager是可以起来的，
127.0.0.1       localhost
192.168.186.100 M1
192.168.186.101 S1
192.168.186.102 S2


但重启下（用reboot），/etc/hosts文件内容发生变化了，多了红色的，而且namenode 起不来，DataNode起来了，NodeManager也可以起来了

[hadoop@S1 ~]$ vi /etc/hosts
127.0.0.1       S1      localhost.localdomain   localhost
::1     S1      localhost6.localdomain6 localhost6
192.168.186.100 M1
192.168.186.101 S1
192.168.186.102 S2




还有个疑问，DataNode与NodeManager的作用是啥呢，现在还有点迷糊，他们的作用与区别呢，谢谢啦！

aqi915 发表于 2015-5-17 22:05
有个奇怪的事：

改vi /etc/hosts为下面时，启动hadoop,namenode可以起来，DataNode,起不了，NodeManag ...

禁用ip 6试试

1.修改/etc/sysconfig/network，追加：

NETWORKING_IPV6=no

2.修改/etc/hosts,把ipv6的那句本地主机名解析的也注释掉：

#::1   localhost localhost6 localhost6.localdomain6

bob007 发表于 2015-5-17 23:17
禁用ip 6试试

1.修改/etc/sysconfig/network，追加：NETWORKING_IPV6=no2.修改/etc/hosts,把ipv6的那 ...

IPV6  之前已经禁用了，现在去看，master与slaver 的clusterID，不同耶，好奇怪，clusterID=CID-3e3ecd67-f606-4955-816c-2306686f097d

aqi915 发表于 2015-5-17 23:24
IPV6  之前已经禁用了，现在去看，master与slaver 的clusterID，不同耶，好奇怪，clusterID=CID-3e3ecd67 ...

多次格式化的结果，修改一样即可

aqi915 发表于 2015-5-17 11:40
额，是的，我做了之前做了个任务计划，时间同步的，这个对什么有影响呢？

时间如果不同步的话，那么zookeeper就没法启动，而HBase是使用zookeeper做协调服务的，所以会出现HMaster启动后又挂掉的情况，之前我就遇到这个情况的。你看看几台机器上时间是否一样。