hive + kerberos spring 配置 DruidDataSource 数据库连接池

spring mvc 项目在 xml 配置文件中配置DruidDataSource 数据库连接池
[mw_shl_code=xml,true] <bean id="hiveDataSource" class="com.alibaba.druid.pool.DruidDataSource">
        <property name="driverClassName" value="org.apache.hive.jdbc.HiveDriver"/>
        <property name="url" value="${hive.url}"/>
        <property name="username" value="${hive.username}"/>
        <property name="password" value="${hive.password}"/>
        <property name="testWhileIdle" value="false"/>
    </bean>
[/mw_shl_code]


若没有采取 kerberos 加密则这种方式正常运转。但是现在添加了 kerberos 权限，如何在不换思路的情况下，即仍采取  xml 配置文件中配置DruidDataSource 数据库连接池的方法解决kerberos 的权限配置问题。

如今网上普遍的解法是单独创建 jdbc 连接，不采取连接池的方式，更不用说在 xml 配置解决。
[mw_shl_code=java,true] public static Connection get_conn() throws SQLException, ClassNotFoundException {
        /** 使用Hadoop安全登录 **/
        Configuration conf = new org.apache.hadoop.conf.Configuration();
        conf.set("hadoop.security.authentication", "Kerberos");
        try {
            UserGroupInformation.setConfiguration(conf);
            UserGroupInformation.loginUserFromKeytab("test/localhost@EXAMPLE.COM", "test.keytab");
        } catch (IOException e1) {
            e1.printStackTrace();
        }
        Class.forName(driverName);
        Connection conn = DriverManager.getConnection(url);
        return conn;
    }[/mw_shl_code]


如何在xml 配置 连接池的方式来解决 kerberos 权限问题呢？

应该是可以通过配置：
修改Druid项目中distribution目录下的pom文件,在project.build.plugins.plugin.executions.execution..configuration.arguments 标签加入以下两行代码:
[mw_shl_code=bash,true]<argument>-c</argument>
<argument>io.druid.extensions:druid-kerberos</argument>[/mw_shl_code]

########################
更多：
Druid集成Kerberos
1.为Druid创建用户
根据Druid官网给出的关于Druid-Kerberos插件配置:

[mw_shl_code=bash,true]druid.hadoop.security.kerberos.principal=druid/_HOST@SUGO.COM
druid.hadoop.security.spnego.principal=HTTP/_HOST@SUGO.COM[/mw_shl_code]
Druid内部交流需要用到druid/_HOST@SUGO.COM和HTTP/_HOST@SUGO.COM用户,因此我们需要为集群中的每台机器创建以上两个用户,在 dev224.sugo.net上执行以下命令:
[mw_shl_code=bash,true]kadmin.local -q "addprinc -randkey druid/dev223.sugo.net@SUGO.COM"
kadmin.local -q "addprinc -randkey druid/dev224.sugo.net@SUGO.COM"
kadmin.local -q "addprinc -randkey druid/dev225.sugo.net@SUGO.COM"
kadmin.local -q "addprinc -randkey HTTP/dev223.sugo.net@SUGO.COM"
kadmin.local -q "addprinc -randkey HTTP/dev224.sugo.net@SUGO.COM"
kadmin.local -q "addprinc -randkey HTTP/dev225.sugo.net@SUGO.COM"[/mw_shl_code]

-randkey标志没有为新principal设置密码，而是指示kadmin生成一个随机密钥。之所以在这里使用这个标志，是因为以上principal不需要用户交互,仅是Druid内部使用,同时也可以隔绝其它principal的访问,保证了Druid内部的安全性.
&#8195;&#8195;创建完成后，查看：
[mw_shl_code=bash,true]$ kadmin.local -q "listprincs"
[/mw_shl_code]



2. 创建keytab文件
&#8195;&#8195;keytab是包含principals和加密principal key的文件. keytab文件对于每个host是唯一的,因为key中包含 hostname. keytab文件用于不需要人工交互和保存纯文本密码,实现到kerberos上验证一个主机上的principal.
因为服务器上可以访问keytab文件即可以以principal的身份通过kerberos的认证,所以,keytab文件应该被妥善保存,应该只有少数的用户可以访问.
&#8195;&#8195;在dev224.sugo.net节点，即 KDC server 节点上执行下面命令为druid/_HOST@SUGO.COM和HTTP/_HOST@SUGO.COM用户创建keytab文件：

[mw_shl_code=bash,true]$ cd /var/kerberos/krb5kdc/
$ kadmin.local -q "xst  -k druid-unmerged.keytab  druid/dev223.sugo.net@SUGO.COM"
$ kadmin.local -q "xst  -k druid-unmerged.keytab  druid/dev224.sugo.net@SUGO.COM"
$ kadmin.local -q "xst  -k druid-unmerged.keytab  druid/dev225.sugo.net@SUGO.COM"

$ kadmin.local -q "xst  -k HTTP.keytab  HTTP/dev223.sugo.net@SUGO.COM"
$ kadmin.local -q "xst  -k HTTP.keytab  HTTP/dev224.sugo.net@SUGO.COM"
$ kadmin.local -q "xst  -k HTTP.keytab  HTTP/dev225.sugo.net@SUGO.COM"[/mw_shl_code]
这样,就会在/var/kerberos/krb5kdc/目录下生成druid-unmerged.keytab和HTTP.keytab两个文件，接下来使用ktutil合并这两个文件为 druid.keytab
[mw_shl_code=bash,true]$ cd /var/kerberos/krb5kdc/

$ ktutil
ktutil: rkt druid-unmerged.keytab
ktutil: rkt HTTP.keytab
ktutil: wkt druid.keytab
ktutil: exit[/mw_shl_code]

使用 klist 显示 druid.keytab 文件列表：
[mw_shl_code=bash,true]$ klist -ket  druid.keytab
Keytab name: FILE:druid.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   2 11/16/17 09:41:21 druid/dev223.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:21 druid/dev223.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:21 druid/dev223.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:21 druid/dev223.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:22 druid/dev223.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:22 druid/dev223.sugo.net@SUGO.COM (des-cbc-md5)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:22 druid/dev224.sugo.net@SUGO.COM (des-cbc-md5)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:22 druid/dev225.sugo.net@SUGO.COM (des-cbc-md5)
   2 11/16/17 09:41:22 HTTP/dev223.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 HTTP/dev223.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:22 HTTP/dev223.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:22 HTTP/dev223.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:22 HTTP/dev223.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:23 HTTP/dev223.sugo.net@SUGO.COM (des-cbc-md5)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:23 HTTP/dev224.sugo.net@SUGO.COM (des-cbc-md5)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (aes256-cts-hmac-sha1-96)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (aes128-cts-hmac-sha1-96)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (des3-cbc-sha1)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (arcfour-hmac)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (des-hmac-sha1)
   2 11/16/17 09:41:23 HTTP/dev225.sugo.net@SUGO.COM (des-cbc-md5) [/mw_shl_code]


验证是否正确合并了key,使用合并后的keytab,分别使用druid和HTTP principals来获取证书.
[mw_shl_code=bash,true]$ kinit -k -t druid.keytab druid/dev224.sugo.net@SUGO.COM
$ kinit -k -t druid.keytab HTTP/dev224.sugo.net@SUGO.COM[/mw_shl_code]


如果出现错误：kinit: Key table entry not found while getting initial credentials,
则上面的合并有问题，重新执行前面的操作

3. 部署kerberos keytab文件
拷贝dev224.sugo.net机器上的druid.keytab 文件到其他节点的 /opt/apps/kerberos/keytabs 目录

[mw_shl_code=bash,true]$ cd /var/kerberos/krb5kdc/

$ scp druid.keytab dev223.sugo.net:/opt/apps/kerberos/keytabs
$ scp druid.keytab dev225.sugo.net:/opt/apps/kerberos/keytabs[/mw_shl_code]
设置权限,分别在 dev223.sugo.net、dev224.sugo.net、dev225.sugo.net 上执行：
[mw_shl_code=bash,true]$ chown druid:druid /opt/apps/kerberos/keytabs/hdfs.keytab
$ chmod 400 /opt/apps/kerberos/keytabs/hdfs.keytab[/mw_shl_code]

由于拥有keytab相当于有了永久凭证,不需要提供密码(如果修改kdc中的principal的密码,则该keytab就会失效),所以其他用户如果对该文件有读权限，就可以冒充keytab中指定的用户身份访问druid,所以 keytab文件需要确保只对 owner 有读权限(0400)

4. 修改 Druid 配置文件
修改Druid项目中distribution目录下的pom文件,在project.build.plugins.plugin.executions.execution..configuration.arguments 标签加入以下两行代码:

[mw_shl_code=bash,true]<argument>-c</argument>
<argument>io.druid.extensions:druid-kerberos</argument>[/mw_shl_code]
&#8195;&#8195;在Druid的公共配置里,增加kerberos相关的配置:
[mw_shl_code=bash,true]druid.extensions.loadList
=["postgresql-metadata-storage", "druid-hdfs-storage", "druid-lucene-extensions", "druid-kerberos"]     //增加druid-kerberos插件
druid.hadoop.security.kerberos.keytab=/opt/apps/kerberos/keytabs/druid.keytab
druid.hadoop.security.kerberos.principal=druid/_HOST@SUGO.COM
druid.hadoop.security.spnego.keytab=/opt/apps/kerberos/keytabs/druid.keytab
druid.hadoop.security.spnego.principal=HTTP/_HOST@SUGO.COM[/mw_shl_code]

将Druid项目重新编译打包,在集群中更新启动.在集群机器中查看各个服务的后台日志,若没有出现错误则说明集成成功.

hive + kerberos spring 配置 DruidDataSource 数据库!!

谢谢大家，问题已解决。

在连接池初始化之前，初始化Kerberos 验证的 bean。下方见 HiveKerberosDruidDataSource

[mw_shl_code=xml,true]<bean id="hiveKerberos" class="com.myhexin.kyc.utils.HiveKerberosDruidDataSource">
        <constructor-arg name="kerberos" value="true"/>
        <constructor-arg name="path" value="${hive.path}"/>
        <constructor-arg name="user" value="${hive.user}"/>
    </bean>

    <bean id="hiveDataSource" class="com.alibaba.druid.pool.DruidDataSource" depends-on="hiveKerberos">
        <property name="driverClassName" value="org.apache.hive.jdbc.HiveDriver"/>
        <property name="url" value="${hive.url}"/>
        <property name="username" value="${hive.username}"/>
        <property name="password" value="${hive.password}"/>
        <property name="testWhileIdle" value="false"/>
    </bean>[/mw_shl_code]

HiveKerberosDruidDataSource：
[mw_shl_code=java,true]public HiveKerberosDruidDataSource(String user,String path,Boolean kerberos){

        this.user = user;
        this.path = path;
        this.kerberos = kerberos;

        if (kerberos){
            Configuration conf = new Configuration();
            conf.set("hadoop.security.authentication", "Kerberos");
            try {
                UserGroupInformation.setConfiguration(conf);
                UserGroupInformation.loginUserFromKeytab(user, path);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

    }[/mw_shl_code]

chyeers 发表于 2018-8-23 15:11
谢谢大家，问题已解决。

在连接池初始化之前，初始化Kerberos 验证的 bean。下方见 HiveKerberosDruidDa ...

66666