分享

hadoop eclipse-plugin编译失败

想用eclipse远程连接集群开发,但是hadoop插件我在hadoop2.6下找了,发行版没有提供插件jar包。

于是我在git hub下载了zip文件
下载hadoop2x-eclipse-plugin
https://github.com/winghc/hadoop2x-eclipse-plugin

分别将hadoop-2.6.0.tar.gz、hadoop-2.6.0-src.tar.gz、hadoop2x-eclipse-pluginhadoop-common-2.2.0-bin下载解压到F:\hadoop\目录下

1.添加环境变量HADOOP_HOME=F:\hadoop\hadoop-2.6.0\
追加环境变量path内容:%HADOOP_HOME%/bin

2.修改编译包及依赖包版本信息
修改F:\hadoop\hadoop2x-eclipse-plugin-master\ivy\libraries.properties
hadoop.version=2.6.0
jackson.version=1.9.13

3.ant编译
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>
ant jar -Dversion=2.6.0 -Declipse.home=D:\eclipse\eclipse-hadoop\eclipse -Dhadoop.home=F:\hadoop\hadoop-2.6.0

我在cmd窗口下运行上面命令行,却在ivy-resolve-common卡住了,12分钟过去后显示
BUILD FAILED
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76
: C:\Program\plugins does not exist.



附上完整的编译过程:
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dve
rsion=2.6.0 -Declipse.home=C:\Program Files\eclipse\eclipse -Dhadoop.home=F:\had
oop\hadoop-2.6.0
Buildfile: F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\b
uild.xml
check-contrib:
init:
     [echo] contrib: eclipse-plugin
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\classes
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\test
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\system
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\system\classes
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\examples
    [mkdir] Created dir: F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\
eclipse-plugin\test\logs
init-contrib:
ivy-probe-antlib:
ivy-init-antlib:
ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = F:\hadoop\hadoop2x-eclipse-plugin-
master\ivy\ivysettings.xml
ivy-resolve-common:
ivy-retrieve-common:
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.fil
e' instead
[ivy:cachepath] :: loading settings :: file = F:\hadoop\hadoop2x-eclipse-plugin-
master\ivy\ivysettings.xml
compile:
     [echo] contrib: eclipse-plugin
    [javac] F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\
build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysc
lasspath=last; set to false for repeatable builds
    [javac] Compiling 45 source files to F:\hadoop\hadoop2x-eclipse-plugin-maste
r\build\contrib\eclipse-plugin\classes
BUILD FAILED
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76
: C:\Program\plugins does not exist.
Total time: 12 minutes 48 seconds
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>

已有(10)人评论

跳转到指定楼层
huntagain 发表于 2015-5-6 10:46:29
编译方法是网上找的,我的ant版本1.9的,应该不是版本问题

ant环境配置
apache-ant-1.9.4-bin.zip解压到D:\apache\目录下,配置环境变量ANT_HOME,并将bin目录配置到path
下载地址:http://mirror.bit.edu.cn/apache//ant/binaries/apache-ant-1.9.4-bin.zip
回复

使用道具 举报

huntagain 发表于 2015-5-6 11:07:39
不好意思,eclipse.home后面的路径写错了,已改为正确的,但是仍旧是一样的错误。
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dversion=2.6.0 -Declipse.home=C:\Program Files\eclipse -Dhadoop.home=F:\hadoop\hadoop-2.6.0
Buildfile: F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml

check-contrib:

init:
     [echo] contrib: eclipse-plugin

init-contrib:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = F:\hadoop\hadoop2x-eclipse-plugin-master\ivy\ivysettings.xml

ivy-resolve-common:

ivy-retrieve-common:
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = F:\hadoop\hadoop2x-eclipse-plugin-master\ivy\ivysettings.xml

compile:
     [echo] contrib: eclipse-plugin
    [javac] F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatab
le builds
    [javac] Compiling 45 source files to F:\hadoop\hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin\classes

BUILD FAILED
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76: C:\Program\plugins does not exist.

Total time: 3 seconds

F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant -version
Apache Ant(TM) version 1.9.4 compiled on April 29 2014
回复

使用道具 举报

Alkaloid0515 发表于 2015-5-6 13:56:13
本帖最后由 Alkaloid0515 于 2015-5-6 13:57 编辑
huntagain 发表于 2015-5-6 11:07
不好意思,eclipse.home后面的路径写错了,已改为正确的,但是仍旧是一样的错误。
F:\hadoop\hadoop2x-ecli ...

这个版本只适合hadoop2.2的
其它版本修改的内容很多,如下面错误,修改ivy.conf.file为 'ivy.settings.file',还有includeantruntime没有设置,在F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:中是否配置 C:\Program\plugins。如果没有配置,这个应该是必须的,所以需要找到相关内容

  1. DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
  2.   [javac] F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatab
  3. F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76: C:\Program\plugins does not exist.
复制代码
假如上面的问题都解决掉了,其实楼主还会遇到很多的坑。
这个坑很深,如果对ant或则maven不熟悉,只能一个个坑往里踩,如果没有准备好,会被折磨的死去回来,建议提早回头。
其它的解决办法:
建议按照这个帖子尝试下
Hadoop2.5 Eclipse插件制作、连接集群视频、及hadoop-eclipse-plugin-2.5.0插件下载
这个帖子应该可以编译成功。


如果万一失败了,楼主可以采用另外的开发方式:也就是非插件方式开发

hadoop开发方式总结及操作指导





回复

使用道具 举报

huntagain 发表于 2015-5-6 14:31:30
Alkaloid0515 发表于 2015-5-6 13:56
这个版本只适合hadoop2.2的
其它版本修改的内容很多,如下面错误,修改ivy.conf.file为 'ivy.settings.f ...

我刚刚在CSDN下载了hadoop-eclipse-plugin-2.6.0.jar,然后我把jar放在eclipse.home/plugins下面。重启eclipse,接着Window->Preference->Hadoop Map/Reduce设置Hadoop安装路径,Window->Show View->other->Map/Reduce Locations。再新建Hadoop locations

输入host:master,默认端口50020。HDFS端口,我查了core-site.xml fs.defaultFS的值位hdfs://master:9000,所以HDFS端口我填的9000。完成。可是还是没有连上,什么都没发生。我考虑可能仍旧是jar包不适合?准备下载论坛的jar包试试看。
回复

使用道具 举报

huntagain 发表于 2015-5-6 14:51:21
Alkaloid0515 发表于 2015-5-6 13:56
这个版本只适合hadoop2.2的
其它版本修改的内容很多,如下面错误,修改ivy.conf.file为 'ivy.settings.f ...

An internal error occurred during: "Map/Reduce location status updater".
java.lang.NullPointerException

这个报错信息,看百度知道上有的人用eclipse原版出现问题

Hadoop2.5 Eclipse插件制作、连接集群视频、及hadoop-eclipse-plugin-2.5.0插件下载

我就下载这个帖子上的eclipse试试看,但我电脑装的jdk是64位,如果eclipse是64位就刚好,若是32位,那么jdk也要32位了。

点评

这个应该都是64位的  发表于 2015-5-6 14:57
回复

使用道具 举报

huntagain 发表于 2015-5-6 15:08:49
Alkaloid0515 发表于 2015-5-6 13:56
这个版本只适合hadoop2.2的
其它版本修改的内容很多,如下面错误,修改ivy.conf.file为 'ivy.settings.f ...

好的,我终于成功了。hadoop-eclipse-plugin-2.6.0.jar现在用的是在论坛下载的

Hadoop-2.6.0在Eclipse中如何安装插件问题

eclipse 需要javaEE企业版的才行,原版不行。也是在论坛下载的

Hadoop2.5 Eclipse插件制作、连接集群视频、及hadoop-eclipse-plugin-2.5.0插件下载
回复

使用道具 举报

arBen 发表于 2015-5-7 08:26:30
我觉得编译插件就是个坑.我编译第一次jar,放入eclipse没任何反应,过了一段时间编译第二次放入eclipse,小象出来啦,执行new location又没反应,调不出hdf配置界面.
回复

使用道具 举报

wx_Rn4sKINt 发表于 2017-10-5 10:28:59
BUILD FAILED
F:\hadoop\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml:76
: C:\Program\plugins does not exist.


----是因为eclipse安装目录不能有特殊字符
回复

使用道具 举报

czs208112 发表于 2017-11-16 16:03:56
编译是可以的,只是确实要改不少配置,
这是我用hadoop2.7.4编译的版本,自测eclipse版本兼容性非常不错,4.4-4.7应该都没问题,http://download.csdn.net/download/czs208112/10021906
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条