本帖最后由 Oner 于 2017-6-1 20:54 编辑
问题导读： 1. ZookeeperListener是怎么实现的？ 2. RPC Server是怎么实现的？ 3. MasterFileSystem是怎么实现的？ 4. RegionServer和Catalog分别是怎么实现的？ 5. ExecutorService是怎么实现的？ 6. LoadBalancer是怎么实现的？ 7. TableDescriptor是怎么实现的？ 8. SnapshotManager是怎么实现的？ 9. CoprocessorProtocol Map是怎么实现的？ 本文基于HBase-0.94.1分析HMaster的主要类成员. HMaster是HBase主/从集群架构中的中央节点。通常一个HBase集群存在多个HMaster节点,其中一个为Active Master,其余为Backup Master. HMaster的主要类成员如下:
1.ZooKeeper侦听这些类都继承自ZookeeperListener. [mw_shl_code=applescript,true]/******************************ZooKeeperListener and ZooKeeperWatcher*********************************************/
// Our zk client.
private ZooKeeperWatcher zooKeeper;
// Manager and zk listener for master election//继承ZooKeeperListener
private ActiveMasterManager activeMasterManager;
// Region server tracker // 继承ZooKeeperListener
private RegionServerTracker regionServerTracker;
// Draining region server tracker//继承ZooKeeperListener
private DrainingServerTracker drainingServerTracker;
/** manager of assignment nodes in zookeeper*/
AssignmentManager assignmentManager;[/mw_shl_code] 1.1 ZooKeeperWatcher
ZooKeeperWatcher是HBase里唯一一个实现Watcher接口的类，其他需要感知ZooKeeper的内部类都要通过ZooKeeperWatcher.registerListener(ZooKeeperListener listener)方法向ZooKeeperWatcher实例注册。 zooKeeper是一个ZooKeeperWatcher对象，它在HMaster构造函数中初始化; 它是一个单一的ZooKeeper Watcher ，每个HMaster，RegionServer,Client都会实例化一个ZooKeeperWatcher。
1.2 ZooKeeperListener
ZooKeeperListener是HBase用来侦听Zookeeper事件的基类。
(1)一个进程的ZookeeperWatcher将执行ZookeeperListener父类的某个方法。为了从watcher收到事件，每个listener必须通过ZookeeperWatcher.registerListener注册自己. (2)ZooKeeperListener的子类需要重写自己感兴趣的方法 ( 注意,wather在调用listeners里的方法时将会被阻塞，所以listener里的方法不要long-running )
ActiveMasterManager
该类用来处理master端所有与master选举相关的事情.
(1).侦听并响应关于master znode的zk通知(ZK Notifications),包括nodeCreated和nodeDeleted. (2).包括一个阻塞方法:持有backup masters，等待active master挂掉. (3).这个class在HMaster里被初始化,HMaster调用blockUntilBecomingActiveMaster()以阻塞等待成为active master.
RegionServerTracker 该类用来通过ZK跟踪所有OnLine状态的RegionServer
DrainingServerTracker该类用来通过ZK跟踪所有处于上线/下线流水状态的RegionServer.
AssignmentManager assignmentManager 负责管理region的分配工作.
2. RPC Server[mw_shl_code=applescript,true]// RPC server for the HMaster
private final RpcServer rpcServer;
/**
* This servers address.
*/
private final InetSocketAddress isa;[/mw_shl_code] HMaster通过HBase RPC机制，将自己封装成一个RPC Server,对外提供RPC调用服务. [mw_shl_code=applescript,true]// RPC server for the HMaster
private final RpcServer rpcServer;
/**
* This servers address.
*/
private final InetSocketAddress isa;[/mw_shl_code]
3.HBase文件系统: MasterFileSystem
MasterFileSystem 抽象了HMaster与低层文件系统交互所需的一系列接口 它在HMaster.finishInitialization()方法中被初始化 [mw_shl_code=applescript,true]// file system manager for the master FS operations//抽象了HMaster与底层文件系统交互所需的一系列操作
private MasterFileSystem fileSystemManager[/mw_shl_code] [mw_shl_code=applescript,true]private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
//将是否为active master标识置true
isActiveMaster = true;
/*我们已经是active master了，开始初始化相关组件.
...
this.masterActiveTime = System.currentTimeMillis();
// TODO: Do this using Dependency Injection, using PicoContainer, Guice or Spring.
// 1.创建HBase文件系统 MasterFileSystem.(抽象了HMaster与低层文件系统交互所需的一系列接口)
this.fileSystemManager = new MasterFileSystem(this, this, metrics, masterRecovery);
...
}[/mw_shl_code] 4. RegionServer和Catalog管理
ServerManager
ServerManager管理所有的regionServer信息。实际上，RegionServerTracker,DrainingServerTracker,AssignmentManager 初始化时需要serverManager作为构造函数的参数. [mw_shl_code=applescript,true]/******************************ServerManager****************************************************/
/** server manager to deal with region server info*/
private ServerManager serverManager;[/mw_shl_code]
RegionServerTracker,DrainingServerTracker,AssignmentManager在initializeZKBasedSystemTracker()时进行初始化. [mw_shl_code=applescript,true]private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.catalogTracker = new CatalogTracker(this.zooKeeper, this.conf, this);
this.catalogTracker.start();
this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
this.assignmentManager = new AssignmentManager(this, serverManager,
this.catalogTracker, this.balancer, this.executorService);
zooKeeper.registerListenerFirst(assignmentManager);
this.regionServerTracker = new RegionServerTracker(zooKeeper, this,
this.serverManager);
this.regionServerTracker.start();
this.drainingServerTracker = new DrainingServerTracker(zooKeeper, this,
this.serverManager);
this.drainingServerTracker.start();[/mw_shl_code]
CatalogTracker
catalogTracker 跟踪"目录表"(-ROOT-和.META.表)的可用性
ClusterStatusTrackerclusterStatusTracker跟踪集群在zookeeper上的配置(Tracker on cluster settings up in zookeeper). ClusterStatusTracker和ClusterStatus不同，后者只是一个存储cluster当前视图的镜像的数据结构，而ClusterStatusTracker是用于跟踪集群在zookeeper上配置的属性信息.
[mw_shl_code=applescript,true]// manager of catalog regions
private CatalogTracker catalogTracker;
// Cluster status zk tracker and local setter
private ClusterStatusTracker clusterStatusTracker;[/mw_shl_code]
5.HBase 执行器服务:ExecutorService
ExecutorService是一个通用的执行器服务类,这个组件抽象了一个threadPool，一个队列，和一个Runnable线程(Handler角色) [mw_shl_code=applescript,true]/******************************ExecutorService****************************************************/
// Instance of the hbase executor service.
ExecutorService executorService;[/mw_shl_code]
该变量在finishInitialization方法中初始化 [mw_shl_code=applescript,true]private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
......
if (!masterRecovery) {
//初始化ExecutorService和serverManager
this.executorService = new ExecutorService(getServerName().toString());//维护一个threadPool和队列
this.serverManager = new ServerManager(this, this);//管理所有的regionserver
}[/mw_shl_code] 6.负载均衡:LoadBalancer
LoadBalancer的职责是维护regionServers之间的负载均衡; balancerChore线程做一些Balancer相关的清理工作.
用户可以通过实现LoadBalancer接口来定制自己的负载均衡策略. 默认情况下，HBase采用的是org.apache.hadoop.hbase.master.DefaultLoadBalancer类做负载均衡. [mw_shl_code=applescript,true]/******************************LoadBalancer, BalancerChore****************************************/
private LoadBalancer balancer;
private Thread balancerChore;[/mw_shl_code]
loadBalancer是在initializeZKBasedSystemTrackers()方法中调用LoadBalancerFactory.getLoadBalancer(conf)初始化的. [mw_shl_code=applescript,true]private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
...
}[/mw_shl_code]
从LoadBalancerFactory代码可以看出, LoadBalancer由参数HBASE_MASTER_LOADBALANCER_CLASS ( hbase.master.loadbalancer.class)指定，默认值是DefaultLoadBalancer.class [mw_shl_code=applescript,true]public class LoadBalancerFactory {
/**
* Create a loadblanacer from the given conf.
* @param conf
* @return A {@link LoadBalancer}
*/
public static LoadBalancer getLoadBalancer(Configuration conf) {
// Create the balancer
Class<? extends LoadBalancer> balancerKlass = conf.getClass(
HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
DefaultLoadBalancer.class, LoadBalancer.class);
return ReflectionUtils.newInstance(balancerKlass, conf);
}
}[/mw_shl_code]
7. Table描述符管理：TableDescriptor
TableDescriptor 接口描述了用来管理Table描述符的一系列操作,FSTableDescriptor是该接口的实现类。 [mw_shl_code=applescript,true]/******************************TableDescriptors***************************************************/
private TableDescriptors tableDescriptors;[/mw_shl_code]
在HMaster.finishInitialization方法中初始化 [mw_shl_code=applescript,true]private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
....
// 2.初始化tableDescriptors，从文件系统读取HTable的描述信息.
this.tableDescriptors =
new FSTableDescriptors(this.fileSystemManager.getFileSystem(),
<pre name="code" class="java"> this.fileSystemManager.getRootDir());[/mw_shl_code]
FSTableDescriptor的构造函数如下: FSTableDescriptors根据fs和rootdir可以读取、更改所有的table描述符 [mw_shl_code=applescript,true]/**
* @param fs
* @param rootdir
* @param fsreadOnly True if we are read-only when it comes to filesystem
* operations; i.e. on remove, we do not do delete in fs.
*/
public FSTableDescriptors(final FileSystem fs, final Path rootdir,
final boolean fsreadOnly) {
super();
this.fs = fs;
this.rootdir = rootdir;
this.fsreadonly = fsreadOnly;
}[/mw_shl_code]
以FSTableDescriptor。FSTableDescriptor通过rootdir读取所有的table描述符 [mw_shl_code=applescript,true]/* (non-Javadoc)
* @see org.apache.hadoop.hbase.TableDescriptors#getTableDescriptors(org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path)
*/
@Override
public Map<String, HTableDescriptor> getAll()
throws IOException {
<pre name="code" class="java"> Map<String, HTableDescriptor> htds = new TreeMap<String, HTableDescriptor>();[/mw_shl_code]
[mw_shl_code=applescript,true]//获取所有的tableName
List<Path> tableDirs = FSUtils.getTableDirs(fs, rootdir);
for (Path d: tableDirs) {
HTableDescriptor htd = null;
try {
htd = get(d.getName());
}
catch (FileNotFoundException fnfe) {
// inability of retrieving one HTD shouldn't stop getting the remaining
LOG.warn("Trouble retrieving htd", fnfe);
}
if (htd == null) continue;
htds.put(d.getName(), htd);
}
return htds;
}[/mw_shl_code]
8. HTable镜像管理:SnapshotManager
SnapshotManager管理snapshots的生成和装载(taking and restoring)过程. [mw_shl_code=applescript,true]// monitor for snapshot of hbase tables
private SnapshotManager snapshotManager; //HBase tables的镜像管理者[/mw_shl_code]
9.协同处理器管理: CoprocessorProtocol Map
[mw_shl_code=applescript,true]/******************************protocolHandlers 管理coprocessor protocol的注册*********************/
// Registered master protocol handlers
private ClassToInstanceMap<CoprocessorProtocol>
protocolHandlers = MutableClassToInstanceMap.create();[/mw_shl_code]
10.健康检查和垃圾清理: 各种Chore线程
[mw_shl_code=applescript,true]/** The health check chore. */
private HealthCheckChore healthCheckChore;//健康检查
/******************************垃圾回收:CatalogJanitor,LogCleaner,HFileCleaner*********************/
private CatalogJanitor catalogJanitorChore; //定时扫描-META-，对无用的region进行垃圾回收
private LogCleaner logCleaner;
private HFileCleaner hfileCleaner;[/mw_shl_code]
11. 其他//todo
来源：http://www.tuicool.com/articles/IjQjAnN
| 
/2 
京ICP备2020039040号
简书网举报电话：021-34700000
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
千斤顶
显身卡
