图文精华

About云-梭伦科技»专题 › 交流区 › 技术交流 › Hadoop|YARN › hadoop使用kerbero后webhdfs的配置问题
分享

hadoop使用kerbero后webhdfs的配置问题

pengsuyun 发表于 2014-10-24 15:59:51 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 11 52106
About云VIP帮助找工作和提升
本帖最后由 pengsuyun 于 2014-10-24 15:59 编辑

如题，在配置如上应用的时候，遇到一个问题“javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store”，导致namenode启动不了，其中有个比较奇怪的问题，我为namenode配置的principal nn/admin@psy.com登录却是没有问题。
core-site.xml中的关键配置： 
  1. <property>

  2.                 <name>hadoop.security.authentication</name>

  3.                 <value>kerberos</value>

  4.         </property>

  5.         <property>

  6.                 <name>hadoop.security.authorization</name>

  7.                 <value>true</value>

  8.         </property>
复制代码

hdfs-site.xml中的配置
  1. <property>

  2.                    <name>dfs.webhdfs.enabled</name>

  3.                    <value>true</value>

  4.            </property>

  5.         <property>

  6.                    <name>dfs.web.authentication.kerberos.principal</name>

  7.                    <value>http/admin@psy.com</value>

  8.            </property>

  9.         <property>

  10.                    <name>dfs.web.authentication.kerberos.keytab</name>

  11.                    <value>/hadoop-data/etc/hadoop/http.service.keytab</value>

  12.            </property>
复制代码

问题报错信息：
  1. 2014-10-24 00:43:00,384 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /hadoop-data/etc/hadoop/http.service.keytab, for principal http/admin@psy.com

  2. 2014-10-24 00:43:00,406 WARN org.mortbay.log: failed SpnegoFilter: javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store

  3. 2014-10-24 00:43:00,407 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@4f98b5c{/,file:/opt/hadoop-2.4.1/share/hadoop/hdfs/webapps/hdfs}

  4. javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store

  5.         at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:203)

  6.         at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)

  7.         at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)

  8.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  9.         at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)

  10.         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)

  11.         at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)

  12.         at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)

  13.         at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)

  14.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  15.         at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)

  16.         at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)

  17.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  18.         at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)

  19.         at org.mortbay.jetty.Server.doStart(Server.java:224)

  20.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  21.         at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)

  22.         at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)

  23.         at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)

  24.         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)

  25.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)

  26.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)

  27.         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)

  28.         at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)

  29. Caused by: javax.security.auth.login.LoginException: No key to store

  30.         at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)

  31.         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  32.         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  33.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  34.         at java.lang.reflect.Method.invoke(Method.java:606)

  35.         at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)

  36.         at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)

  37.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)

  38.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)

  39.         at java.security.AccessController.doPrivileged(Native Method)

  40.         at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)

  41.         at javax.security.auth.login.LoginContext.login(LoginContext.java:596)

  42.         at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)

  43.         ... 23 more

  44. 2014-10-24 00:43:00,418 WARN org.mortbay.log: Nested in javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store:

  45. javax.security.auth.login.LoginException: No key to store

  46.         at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)

  47.         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  48.         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  49.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  50.         at java.lang.reflect.Method.invoke(Method.java:606)

  51.         at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)

  52.         at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)

  53.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)

  54.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)

  55.         at java.security.AccessController.doPrivileged(Native Method)

  56.         at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)

  57.         at javax.security.auth.login.LoginContext.login(LoginContext.java:596)

  58.         at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)

  59.         at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)

  60.         at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)

  61.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  62.         at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)

  63.         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)

  64.         at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)

  65.         at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)

  66.         at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)

  67.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  68.         at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)

  69.         at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)

  70.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  71.         at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)

  72.         at org.mortbay.jetty.Server.doStart(Server.java:224)

  73.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  74.         at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)

  75.         at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)

  76.         at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)

  77.         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)

  78.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)

  79.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)

  80.         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)

  81.         at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)

  82. 2014-10-24 00:43:00,447 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50070

  83. 2014-10-24 00:43:00,447 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:50070

  84. 2014-10-24 00:43:00,454 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...

  85. 2014-10-24 00:43:00,455 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.

  86. 2014-10-24 00:43:00,455 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.

  87. 2014-10-24 00:43:00,455 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join

  88. java.io.IOException: Unable to initialize WebAppContext

  89.         at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:818)

  90.         at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:132)

  91.         at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:666)

  92.         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:557)

  93.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)

  94.         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)

  95.         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)

  96.         at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)

  97. Caused by: javax.servlet.ServletException: javax.security.auth.login.LoginException: No key to store

  98.         at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:203)

  99.         at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:150)

  100.         at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)

  101.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  102.         at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)

  103.         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)

  104.         at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)

  105.         at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)

  106.         at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)

  107.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  108.         at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)

  109.         at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)

  110.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  111.         at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)

  112.         at org.mortbay.jetty.Server.doStart(Server.java:224)

  113.         at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)

  114.         at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:796)

  115.         ... 7 more

  116. Caused by: javax.security.auth.login.LoginException: No key to store

  117.         at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1072)

  118.         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  119.         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  120.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  121.         at java.lang.reflect.Method.invoke(Method.java:606)

  122.         at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)

  123.         at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)

  124.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)

  125.         at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)

  126.         at java.security.AccessController.doPrivileged(Native Method)

  127.         at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)

  128.         at javax.security.auth.login.LoginContext.login(LoginContext.java:596)

  129.         at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:187)

  130.         ... 23 more

  131. 2014-10-24 00:43:00,464 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1

  132. 2014-10-24 00:43:00,466 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
复制代码




回复

使用道具 举报

自定义广告语

已有(11)人评论

跳转到指定楼层
正序浏览
bioger_hit 发表于 2014-10-24 18:08:49



建议按照下面配置，改一下

  1. <!-- General HDFS security config -->

  2. <property>

  3.   <name>dfs.block.access.token.enable</name>

  4.   <value>true</value>

  5. </property>

  6.  

  7. <!-- NameNode security config -->

  8. <property>

  9.   <name>dfs.https.address</name>

  10.   <value><fully qualified domain name of NN>:50470</value>

  11. </property>

  12. <property>

  13.   <name>dfs.https.port</name>

  14.   <value>50470</value>

  15. </property>

  16. <property>

  17.   <name>dfs.namenode.keytab.file</name>

  18.   <value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->

  19. </property>

  20. <property>

  21.   <name>dfs.namenode.kerberos.principal</name>

  22.   <value>hdfs/_HOST@YOUR-REALM.COM</value>

  23. </property>

  24. <property>

  25.   <name>dfs.namenode.kerberos.https.principal</name>

  26.   <value>host/_HOST@YOUR-REALM.COM</value>

  27. </property>

  28.  

  29. <!-- Secondary NameNode security config -->

  30. <property>

  31.   <name>dfs.secondary.https.address</name>

  32.   <value><fully qualified domain name of 2NN>:50495</value>

  33. </property>

  34. <property>

  35.   <name>dfs.secondary.https.port</name>

  36.   <value>50495</value>

  37. </property>

  38. <property>

  39.   <name>dfs.secondary.namenode.keytab.file</name>

  40.   <value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->

  41. </property>

  42. <property>

  43.   <name>dfs.secondary.namenode.kerberos.principal</name>

  44.   <value>hdfs/_HOST@YOUR-REALM.COM</value>

  45. </property>

  46. <property>

  47.   <name>dfs.secondary.namenode.kerberos.https.principal</name>

  48.   <value>host/_HOST@YOUR-REALM.COM</value>

  49. </property>

  50.  

  51. <!-- DataNode security config -->

  52. <property>

  53.   <name>dfs.datanode.data.dir.perm</name>

  54.   <value>700</value>

  55. </property>

  56. <property>

  57.   <name>dfs.datanode.address</name>

  58.   <value>0.0.0.0:1004</value>

  59. </property>

  60. <property>

  61.   <name>dfs.datanode.http.address</name>

  62.   <value>0.0.0.0:1006</value>

  63. </property>

  64. <property>

  65.   <name>dfs.datanode.keytab.file</name>

  66.   <value>/usr/local/hadoop/conf/hdfs.keytab</value> <!-- path to the HDFS keytab -->

  67. </property>

  68. <property>

  69.   <name>dfs.datanode.kerberos.principal</name>

  70.   <value>hdfs/_HOST@YOUR-REALM.COM</value>

  71. </property>

  72. <property>

  73.   <name>dfs.datanode.kerberos.https.principal</name>

  74.   <value>host/_HOST@YOUR-REALM.COM</value>

  75. </property>
复制代码



回复

使用道具 举报

desehawk 发表于 2014-10-24 18:33:29
感觉你的配置文档有些问题，可以参考这个
hadoop 添加kerberos认证


回复

使用道具 举报

pengsuyun 发表于 2014-10-24 23:21:08
desehawk 发表于 2014-10-24 18:33
感觉你的配置文档有些问题，可以参考这个
hadoop 添加kerberos认证

你的这篇帖子我之前也看过了，可能我的能力有限理解得不太好，但是我想理解其中的这个原理是什么？方便留下你的qq吗？我好直接找你请教问题。

点评

hiqj
最后，你们这个问题解决了么？！能否把解决方法分享一下！我也遇到这个问题了！  发表于 2014-11-7 20:07
回复

使用道具 举报

desehawk 发表于 2014-10-25 09:25:44
刚开始学习，如果不能理解，就先参照别人的来学习，等时间长了，熟悉了，自然理解了
回复

使用道具 举报

pengsuyun 发表于 2014-11-10 08:46:31
确实是这样，好吧！
回复

使用道具 举报

xbings 发表于 2015-4-27 14:27:40
请问这个问题最终怎么解决的呢？
回复

使用道具 举报

z812620498 发表于 2015-9-17 11:46:55
估计是没有解决吧
回复

使用道具 举报

z812620498 发表于 2015-9-17 11:47:34
我也遇到类似的问题 无奈
回复

使用道具 举报

linqingrui 发表于 2016-3-2 13:42:54
应该是时间同步的问题
回复

使用道具 举报

12下一页
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

发表新帖
pengsuyun

高级会员

关注

  • 39

    主题

  • 246

    帖子

  • 8

    粉丝

TA的主题
关闭

推荐上一条 /2 下一条

关于我们· 联系我们· 加入我们· 小黑屋· 合作伙伴   

Copyright © 2001-2024 About云-梭伦科技 Powered by Discuz! X3.4 Licensed Discuz Team.

简书 / Smrz 京ICP备2020039040号 Wxb 简书网举报电话：021-34700000