分享

求助 关于Hbase完全分布式

mask 2019-6-1 22:06:14 发表于 疑问解答 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 1 5724
本帖最后由 mask 于 2019-6-1 22:07 编辑

用的hadoop-3.1.2,hbase-2.1.4,zookeeper-3.4.14。参照其他人的配置:https://blog.csdn.net/wx2007xing/article/details/88710235
habase配置文件:
设置了不使用自带的zookeeper
[mw_shl_code=shell,true]export HBASE_MANAGES_ZK=false
[/mw_shl_code]

hbase-site.xml
[mw_shl_code=xml,true]<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://master:9000/hbase</value>
    <description>hadoop集群名称</description>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master,server1,server2</value>
    <description>zookeeper集群</description>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
    <description>是否是完全分布式</description>
  </property>
  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
    <description>完全分布式式必须为false</description>
  </property>
</configuration>[/mw_shl_code]

我的启动顺序:zookeeper(zkServer.sh start) -> hadoop(start-all.sh) -> hbase(start-hbase.sh)
前两个都启动成功了,运行状态都正常,zookeeper有leader和follower,hdfs可以看到web界面。
但是当我启动hbase时就是直接报端口占用了:

2019-06-01 20:45:43,996 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:java.library.path=/usr/local/hadoop-3.1.2/lib/native
2019-06-01 20:45:43,996 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:java.io.tmpdir=/tmp
2019-06-01 20:45:43,996 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:java.compiler=<NA>
2019-06-01 20:45:43,996 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:os.name=Linux
2019-06-01 20:45:43,997 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:os.arch=amd64
2019-06-01 20:45:43,997 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:os.version=3.10.0-957.12.1.el7.x86_64
2019-06-01 20:45:43,998 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:user.name=root
2019-06-01 20:45:43,998 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:user.home=/root
2019-06-01 20:45:43,998 INFO  [main] server.ZooKeeperServer (Environment.java:logEnv(100)) - Server environment:user.dir=/usr/local
2019-06-01 20:45:44,020 INFO  [main] server.ZooKeeperServer (ZooKeeperServer.java:<init>(173)) - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /tmp/hbase-root/zookeeper/zookeeper_0/version-2 snapdir /tmp/hbase-root/zookeeper/zookeeper_0/version-2
2019-06-01 20:45:44,020 INFO  [main] server.ZooKeeperServer (ZooKeeperServer.java:setMinSessionTimeout(838)) - minSessionTimeout set to -1
2019-06-01 20:45:44,021 INFO  [main] server.ZooKeeperServer (ZooKeeperServer.java:setMaxSessionTimeout(847)) - maxSessionTimeout set to -1
2019-06-01 20:45:44,057 INFO  [main] server.NIOServerCnxnFactory (NIOServerCnxnFactory.java:configure(89)) - binding to port 0.0.0.0/0.0.0.0:2181
2019-06-01 20:45:44,058 INFO  [main] server.NIOServerCnxnFactory (NIOServerCnxnFactory.java:configure(89)) - binding to port 0.0.0.0/0.0.0.0:2182
2019-06-01 20:45:44,232 ERROR [main] server.ZooKeeperServer (ZooKeeperServer.java:setState(472)) - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
2019-06-01 20:45:44,252 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2182] server.NIOServerCnxnFactory (NIOServerCnxnFactory.java:run(192)) - Accepted socket connection from /127.0.0.1:41538
2019-06-01 20:45:44,264 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2182] server.ServerCnxn (ServerCnxn.java:isEnabled(318)) - The list of known four letter word commands is : [{1936881266=srvr, 1937006964=stat, 2003003491=wchc, 1685417328=dump, 1668445044=crst, 1936880500=srst, 1701738089=envi, 1668247142=conf, 2003003507=wchs, 2003003504=wchp, 1668247155=cons, 1835955314=mntr, 1769173615=isro, 1920298859=ruok, 1735683435=gtmk, 1937010027=stmk}]
2019-06-01 20:45:44,265 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2182] server.ServerCnxn (ServerCnxn.java:isEnabled(319)) - The list of enabled four letter word commands is : [[wchs, stat, wchp, stmk, conf, ruok, mntr, srvr, wchc, envi, srst, isro, dump, gtmk, crst, cons]]
2019-06-01 20:45:44,265 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2182] server.NIOServerCnxn (NIOServerCnxn.java:checkFourLetterWord(883)) - Processing stat command from /127.0.0.1:41538
2019-06-01 20:45:44,268 INFO  [Thread-2] server.NIOServerCnxn (NIOServerCnxn.java:commandRun(674)) - Stat command output
2019-06-01 20:45:44,269 INFO  [Thread-2] server.NIOServerCnxn (NIOServerCnxn.java:closeSock(1044)) - Closed socket connection for client /127.0.0.1:41538 (no session established for client)
2019-06-01 20:45:44,270 INFO  [main] zookeeper.MiniZooKeeperCluster (MiniZooKeeperCluster.java:startup(283)) - Started MiniZooKeeperCluster and ran successful 'stat' on client port=2182
Could not start ZK at requested port of 2181.  ZK was started at port: 2182.  Aborting as clients (e.g. shell) will not be able to find this ZK quorum.
2019-06-01 20:45:44,271 ERROR [main] master.HMasterCommandLine (HMasterCommandLine.java:startMaster(247)) - Master exiting
java.io.IOException: Could not start ZK at requested port of 2181.  ZK was started at port: 2182.  Aborting as clients (e.g. shell) will not be able to find this ZK quorum.
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:217)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3111)


如果我关闭zookeeper,直接启动hbase,就会报:

2019-06-01 20:55:11,436 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1032)) - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2019-06-01 20:55:11,437 WARN  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn (ClientCnxn.java:run(1162)) - Session 0x16b131b79f60001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: 拒绝连接
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)

这个应该是zookeeper没有全部启动报的错。

所以,求大神指点,为什么按照配置还是会去启动自带的zookeeper呢?是hbase配置有问题还是其他的配置问题?或者是配置文件本身的问题?(为了图方便,我用的winscp编辑的配置文件,可能出现编码问题。)

已有(1)人评论

跳转到指定楼层
s060403072 发表于 2019-6-2 10:49:04
自带的zookeeper可能没有关闭掉。导致启动了两个zookeeper发生冲突。
具体需要楼主自己检测下。
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条