图文精华

About云-梭伦科技»专题 › 技术学习(版主发帖区) › 大数据学习 › Hadoop|YARN › 总结型› 编译hadoop2.5.2经验总结、问题分析及下载32位、64位安 ...
分享

编译hadoop2.5.2经验总结、问题分析及下载32位、64位安装包下载

 
nettman 2014-11-24 01:08:53 发表于 总结型 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 118 150923
About云VIP帮助找工作和提升
本帖最后由 nettman 于 2014-11-24 01:33 编辑



编译的时候遇到了不少问题，在后面会贴出错误，参考
从零教你在Linux环境下（ubuntu 12.04）如何编译hadoop2.4环境重新安装了一遍，但是却没有预想的那么顺利，遇到了不少问题，最终归结于编译环境不纯，什么是不纯那，举个例子

当时想通过在线安装maven，可是网络实在是太慢了，所以终止在线安装，直接从网上了下载了个离线包，然后配置了一下，结果编译不成功。


最后产生了如下的问题：
问题1：

  1. [INFO] ------------------------------------------------------------------------

  2. [INFO] Reactor Summary:

  3. [INFO] 

  4. [INFO] Apache Hadoop Main ................................. SUCCESS [04:41 min]

  5. [INFO] Apache Hadoop Project POM .......................... FAILURE [32:27 min]

  6. [INFO] Apache Hadoop Annotations .......................... SKIPPED

  7. [INFO] Apache Hadoop Assemblies ........................... SKIPPED

  8. [INFO] Apache Hadoop Project Dist POM ..................... SKIPPED

  9. [INFO] Apache Hadoop Maven Plugins ........................ SKIPPED

  10. [INFO] Apache Hadoop MiniKDC .............................. SKIPPED

  11. [INFO] Apache Hadoop Auth ................................. SKIPPED

  12. [INFO] Apache Hadoop Auth Examples ........................ SKIPPED

  13. [INFO] Apache Hadoop Common ............................... SKIPPED

  14. [INFO] Apache Hadoop NFS .................................. SKIPPED

  15. [INFO] Apache Hadoop Common Project ....................... SKIPPED

  16. [INFO] Apache Hadoop HDFS ................................. SKIPPED

  17. [INFO] Apache Hadoop HttpFS ............................... SKIPPED

  18. [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED

  19. [INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED

  20. [INFO] Apache Hadoop HDFS Project ......................... SKIPPED

  21. [INFO] hadoop-yarn ........................................ SKIPPED

  22. [INFO] hadoop-yarn-api .................................... SKIPPED

  23. [INFO] hadoop-yarn-common ................................. SKIPPED

  24. [INFO] hadoop-yarn-server ................................. SKIPPED

  25. [INFO] hadoop-yarn-server-common .......................... SKIPPED

  26. [INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED

  27. [INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED

  28. [INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED

  29. [INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED

  30. [INFO] hadoop-yarn-server-tests ........................... SKIPPED

  31. [INFO] hadoop-yarn-client ................................. SKIPPED

  32. [INFO] hadoop-yarn-applications ........................... SKIPPED

  33. [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED

  34. [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED

  35. [INFO] hadoop-yarn-site ................................... SKIPPED

  36. [INFO] hadoop-yarn-project ................................ SKIPPED

  37. [INFO] hadoop-mapreduce-client ............................ SKIPPED

  38. [INFO] hadoop-mapreduce-client-core ....................... SKIPPED

  39. [INFO] hadoop-mapreduce-client-common ..................... SKIPPED

  40. [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED

  41. [INFO] hadoop-mapreduce-client-app ........................ SKIPPED

  42. [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED

  43. [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED

  44. [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED

  45. [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED

  46. [INFO] hadoop-mapreduce ................................... SKIPPED

  47. [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED

  48. [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED

  49. [INFO] Apache Hadoop Archives ............................. SKIPPED

  50. [INFO] Apache Hadoop Rumen ................................ SKIPPED

  51. [INFO] Apache Hadoop Gridmix .............................. SKIPPED

  52. [INFO] Apache Hadoop Data Join ............................ SKIPPED

  53. [INFO] Apache Hadoop Extras ............................... SKIPPED

  54. [INFO] Apache Hadoop Pipes ................................ SKIPPED

  55. [INFO] Apache Hadoop OpenStack support .................... SKIPPED

  56. [INFO] Apache Hadoop Client ............................... SKIPPED

  57. [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED

  58. [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED

  59. [INFO] Apache Hadoop Tools Dist ........................... SKIPPED

  60. [INFO] Apache Hadoop Tools ................................ SKIPPED

  61. [INFO] Apache Hadoop Distribution ......................... SKIPPED

  62. [INFO] ------------------------------------------------------------------------

  63. [INFO] BUILD FAILURE

  64. [INFO] ------------------------------------------------------------------------

  65. [INFO] Total time: 40:20 min

  66. [INFO] Finished at: 2014-11-23T00:36:03-08:00

  67. [INFO] Final Memory: 32M/84M

  68. [INFO] ------------------------------------------------------------------------

  69. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-project: Execution module-javadocs of goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar failed: Plugin org.apache.maven.plugins:maven-javadoc-plugin:2.8.1 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-javadoc-plugin:jar:2.8.1 -> org.apache.maven.shared:maven-common-artifact-filters:jar:1.3: Failed to read artifact descriptor for org.apache.maven.shared:maven-common-artifact-filters:jar:1.3: Could not transfer artifact org.apache.maven:maven-parent:pom:13 from/to central (http://repo.maven.apache.org/maven2): GET request of: org/apache/maven/maven-parent/13/maven-parent-13.pom from central failed: Read timed out -> [Help 1]
复制代码



问题2

  1. [INFO] Apache Hadoop Main ................................. SUCCESS [  2.484 s]

  2. [INFO] Apache Hadoop Project POM .......................... SUCCESS [  2.025 s]

  3. [INFO] Apache Hadoop Annotations .......................... SUCCESS [  4.016 s]

  4. [INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.353 s]

  5. [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  2.283 s]

  6. [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  5.380 s]

  7. [INFO] Apache Hadoop MiniKDC .............................. FAILURE [  0.874 s]

  8. [INFO] Apache Hadoop Auth ................................. SKIPPED

  9. [INFO] Apache Hadoop Auth Examples ........................ SKIPPED

  10. [INFO] Apache Hadoop Common ............................... SKIPPED

  11. [INFO] Apache Hadoop NFS .................................. SKIPPED

  12. [INFO] Apache Hadoop Common Project ....................... SKIPPED

  13. [INFO] Apache Hadoop HDFS ................................. SKIPPED

  14. [INFO] Apache Hadoop HttpFS ............................... SKIPPED

  15. [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED

  16. [INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED

  17. [INFO] Apache Hadoop HDFS Project ......................... SKIPPED

  18. [INFO] hadoop-yarn ........................................ SKIPPED

  19. [INFO] hadoop-yarn-api .................................... SKIPPED

  20. [INFO] hadoop-yarn-common ................................. SKIPPED

  21. [INFO] hadoop-yarn-server ................................. SKIPPED

  22. [INFO] hadoop-yarn-server-common .......................... SKIPPED

  23. [INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED

  24. [INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED

  25. [INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED

  26. [INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED

  27. [INFO] hadoop-yarn-server-tests ........................... SKIPPED

  28. [INFO] hadoop-yarn-client ................................. SKIPPED

  29. [INFO] hadoop-yarn-applications ........................... SKIPPED

  30. [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED

  31. [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED

  32. [INFO] hadoop-yarn-site ................................... SKIPPED

  33. [INFO] hadoop-yarn-project ................................ SKIPPED

  34. [INFO] hadoop-mapreduce-client ............................ SKIPPED

  35. [INFO] hadoop-mapreduce-client-core ....................... SKIPPED

  36. [INFO] hadoop-mapreduce-client-common ..................... SKIPPED

  37. [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED

  38. [INFO] hadoop-mapreduce-client-app ........................ SKIPPED

  39. [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED

  40. [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED

  41. [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED

  42. [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED

  43. [INFO] hadoop-mapreduce ................................... SKIPPED

  44. [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED

  45. [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED

  46. [INFO] Apache Hadoop Archives ............................. SKIPPED

  47. [INFO] Apache Hadoop Rumen ................................ SKIPPED

  48. [INFO] Apache Hadoop Gridmix .............................. SKIPPED

  49. [INFO] Apache Hadoop Data Join ............................ SKIPPED

  50. [INFO] Apache Hadoop Extras ............................... SKIPPED

  51. [INFO] Apache Hadoop Pipes ................................ SKIPPED

  52. [INFO] Apache Hadoop OpenStack support .................... SKIPPED

  53. [INFO] Apache Hadoop Client ............................... SKIPPED

  54. [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED

  55. [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED

  56. [INFO] Apache Hadoop Tools Dist ........................... SKIPPED

  57. [INFO] Apache Hadoop Tools ................................ SKIPPED

  58. [INFO] Apache Hadoop Distribution ......................... SKIPPED

  59. [INFO] ------------------------------------------------------------------------

  60. [INFO] BUILD FAILURE

  61. [INFO] ------------------------------------------------------------------------

  62. [INFO] Total time: 22.376 s

  63. [INFO] Finished at: 2014-11-23T04:20:31-08:00

  64. [INFO] Final Memory: 38M/91M

  65. [INFO] ------------------------------------------------------------------------

  66. [ERROR] Failed to execute goal on project hadoop-minikdc: Could not resolve dependencies for project org.apache.hadoop:hadoop-minikdc:jar:2.5.2: Failure to find org.apache.directory.server:apacheds-core:jar:2.0.0-M15 in https://repository.apache.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots.https has elapsed or updates are forced -> [Help 1]

  67. [ERROR] 

  68. [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

  69. [ERROR] Re-run Maven using the -X switch to enable full debug logging.

  70. [ERROR] 

  71. [ERROR] For more information about the errors and possible solutions, please read the following articles:

  72. [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

  73. [ERROR] 

  74. [ERROR] After correcting the problems, you can resume the build with the command

  75. [ERROR]   mvn <goals> -rf :hadoop-minikdc
复制代码

问题3


  1. 

  2. [INFO] ------------------------------------------------------------------------

  3. [INFO] Reactor Summary:

  4. [INFO] 

  5. [INFO] Apache Hadoop Main ................................. SUCCESS [  2.713 s]

  6. [INFO] Apache Hadoop Project POM .......................... SUCCESS [  2.034 s]

  7. [INFO] Apache Hadoop Annotations .......................... SUCCESS [  3.880 s]

  8. [INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.411 s]

  9. [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  2.901 s]

  10. [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  5.805 s]

  11. [INFO] Apache Hadoop Auth ................................. FAILURE [01:09 min]

  12. [INFO] Apache Hadoop Auth Examples ........................ SKIPPED

  13. [INFO] Apache Hadoop Common ............................... SKIPPED

  14. [INFO] Apache Hadoop NFS .................................. SKIPPED

  15. [INFO] Apache Hadoop MiniKDC .............................. SKIPPED

  16. [INFO] Apache Hadoop Common Project ....................... SKIPPED

  17. [INFO] Apache Hadoop HDFS ................................. SKIPPED

  18. [INFO] Apache Hadoop HttpFS ............................... SKIPPED

  19. [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED

  20. [INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED

  21. [INFO] Apache Hadoop HDFS Project ......................... SKIPPED

  22. [INFO] hadoop-yarn ........................................ SKIPPED

  23. [INFO] hadoop-yarn-api .................................... SKIPPED

  24. [INFO] hadoop-yarn-common ................................. SKIPPED

  25. [INFO] hadoop-yarn-server ................................. SKIPPED

  26. [INFO] hadoop-yarn-server-common .......................... SKIPPED

  27. [INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED

  28. [INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED

  29. [INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED

  30. [INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED

  31. [INFO] hadoop-yarn-server-tests ........................... SKIPPED

  32. [INFO] hadoop-yarn-client ................................. SKIPPED

  33. [INFO] hadoop-yarn-applications ........................... SKIPPED

  34. [INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED

  35. [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED

  36. [INFO] hadoop-yarn-site ................................... SKIPPED

  37. [INFO] hadoop-yarn-project ................................ SKIPPED

  38. [INFO] hadoop-mapreduce-client ............................ SKIPPED

  39. [INFO] hadoop-mapreduce-client-core ....................... SKIPPED

  40. [INFO] hadoop-mapreduce-client-common ..................... SKIPPED

  41. [INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED

  42. [INFO] hadoop-mapreduce-client-app ........................ SKIPPED

  43. [INFO] hadoop-mapreduce-client-hs ......................... SKIPPED

  44. [INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED

  45. [INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED

  46. [INFO] Apache Hadoop MapReduce Examples ................... SKIPPED

  47. [INFO] hadoop-mapreduce ................................... SKIPPED

  48. [INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED

  49. [INFO] Apache Hadoop Distributed Copy ..................... SKIPPED

  50. [INFO] Apache Hadoop Archives ............................. SKIPPED

  51. [INFO] Apache Hadoop Rumen ................................ SKIPPED

  52. [INFO] Apache Hadoop Gridmix .............................. SKIPPED

  53. [INFO] Apache Hadoop Data Join ............................ SKIPPED

  54. [INFO] Apache Hadoop Extras ............................... SKIPPED

  55. [INFO] Apache Hadoop Pipes ................................ SKIPPED

  56. [INFO] Apache Hadoop OpenStack support .................... SKIPPED

  57. [INFO] Apache Hadoop Client ............................... SKIPPED

  58. [INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED

  59. [INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED

  60. [INFO] Apache Hadoop Tools Dist ........................... SKIPPED

  61. [INFO] Apache Hadoop Tools ................................ SKIPPED

  62. [INFO] Apache Hadoop Distribution ......................... SKIPPED

  63. [INFO] ------------------------------------------------------------------------

  64. [INFO] BUILD FAILURE

  65. [INFO] ------------------------------------------------------------------------

  66. [INFO] Total time: 01:32 min

  67. [INFO] Finished at: 2014-11-23T04:50:17-08:00

  68. [INFO] Final Memory: 42M/102M

  69. [INFO] ------------------------------------------------------------------------

  70. [ERROR] Failed to execute goal on project hadoop-auth: Could not resolve dependencies for project org.apache.hadoop:hadoop-auth:jar:2.5.2: Failure to find org.apache.directory.server:apacheds-core:jar:2.0.0-M15 in https://repository.apache.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots.https has elapsed or updates are forced -> [Help 1]

  71. [ERROR] 

  72. [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

  73. [ERROR] Re-run Maven using the -X switch to enable full debug logging.

  74. [ERROR] 

  75. [ERROR] For more information about the errors and possible solutions, please read the following articles:

  76. [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

  77. [ERROR] 

  78. [ERROR] After correcting the problems, you can resume the build with the command

  79. [ERROR]   mvn <goals> -rf :hadoop-auth

复制代码

上面上的含义都是找不到包，可能的原因是找不到包，
但是我的网络又是通的，所以所以肯定是maven出了问题。
最后重新部署了干净的环境。

终于编译成功

编译成功1.png


编译成功.png


成功编译包.png


hadoop-aboutyun-Linux64-2.5.2-.tar.gz


链接：http://pan.baidu.com/s/1bnH94xp 密码：
游客，如果您要查看本帖隐藏内容请回复



hadoop-2.5.2-官网.tar.gz（名字自行修改）
链接：http://pan.baidu.com/s/1ntHpj1R 密码：m0nw




文章推荐：

编译hadoop2.5.2经验总结、问题分析及下载32位、64位安装包下载




加微信w3aboutyun，可拉入技术爱好者群
回复

使用道具 举报

自定义广告语

已有(118)人评论

跳转到指定楼层
正序浏览
gjy20075982 发表于 2014-11-24 09:13:51
提示: 作者被禁止或删除 内容自动屏蔽
回复

使用道具 举报

韩克拉玛寒 发表于 2014-11-24 09:21:57
很好的文章，分享学习了
回复

使用道具 举报

CM潜修 发表于 2014-11-24 10:39:26
学习 了。这几天我也在看这方面知识
回复

使用道具 举报

凌飞羽 发表于 2014-11-24 22:21:58
安装这个确实折腾，现在都要用maven 编译源码吗，能否直接用官网编译好的
回复

使用道具 举报

desehawk 发表于 2014-11-24 23:30:02
凌飞羽 发表于 2014-11-24 22:21
安装这个确实折腾，现在都要用maven 编译源码吗，能否直接用官网编译好的
官网是32位的
回复

使用道具 举报

凌飞羽 发表于 2014-11-24 23:51:52
回复

使用道具 举报

meta 发表于 2014-11-26 11:27:22
学习中     多 多多
回复

使用道具 举报

290390573 发表于 2014-11-28 15:47:39
thanks for share


回复

使用道具 举报

12345678910... 12下一页
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

发表新帖
nettman

超级版主

关注

  • 2849

    主题

  • 3142

    帖子

  • 477

    粉丝

TA的主题
关闭

推荐上一条 /2 下一条

关于我们· 联系我们· 加入我们· 小黑屋· 合作伙伴   

Copyright © 2001-2024 About云-梭伦科技 Powered by Discuz! X3.4 Licensed Discuz Team.

简书 / Smrz 京ICP备2020039040号 Wxb 简书网举报电话：021-34700000