分享

eclipse运行wordcount出错

tzy1023 发表于 2014-11-17 17:16:13 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 3 8252
本帖最后由 tzy1023 于 2014-11-17 22:15 编辑

14/11/17 22:13:00 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
14/11/17 22:13:00 INFO input.FileInputFormat: Total input paths to process : 0
14/11/17 22:13:00 INFO mapred.JobClient: Running job: job_201409181653_0086
14/11/17 22:13:01 INFO mapred.JobClient:  map 0% reduce 0%
14/11/17 22:13:08 INFO mapred.JobClient: Task Id : attempt_201409181653_0086_r_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:211)
    at org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1394)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1945)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:385)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 9 more

14/11/17 22:13:13 INFO mapred.JobClient: Task Id : attempt_201409181653_0086_r_000000_1, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:211)
    at org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1394)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1945)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:385)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 9 more

14/11/17 22:13:19 INFO mapred.JobClient: Task Id : attempt_201409181653_0086_r_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:211)
    at org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1394)
    at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1945)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:385)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: WordCount$IntSumReducer
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
    ... 9 more

14/11/17 22:13:28 INFO mapred.JobClient: Job complete: job_201409181653_0086
14/11/17 22:13:28 INFO mapred.JobClient: Counters: 6
14/11/17 22:13:28 INFO mapred.JobClient:   Job Counters
14/11/17 22:13:28 INFO mapred.JobClient:     Launched reduce tasks=4
14/11/17 22:13:28 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=4348
14/11/17 22:13:28 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/11/17 22:13:28 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/11/17 22:13:28 INFO mapred.JobClient:     Failed reduce tasks=1
14/11/17 22:13:28 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=20204
我换了台电脑还是一样的错误,同学当时和我一起配置的hadoop,他运行的就没有错误我在终端里用命令运行的话,有结果,但是还会产生一个logs文件夹,不知道为啥。

已有(3)人评论

跳转到指定楼层
pig2 发表于 2014-11-17 18:45:38

上面有些描述不太清楚,存在比较多的盲点:这个是在eclipse还是在集群。
这个程序是否在同学机器上运行过。

整体描述来看,就是环境的问题,看看classpath,及运行的位置、环境变量这些都是有可能产生这个问题。所以仔细检查下。

回复

使用道具 举报

tzy1023 发表于 2014-11-17 22:08:04
pig2 发表于 2014-11-17 18:45
上面有些描述不太清楚,存在比较多的盲点:这个是在eclipse还是在集群。
这个程序是否在同学机器上运行过 ...

无论是在集群还是伪分布集群运行都是这样,这个就是自带的wordcount例子啊,程序本身是没问题的,网上搜了很多了,始终没有解决
回复

使用道具 举报

bioger_hit 发表于 2014-11-17 22:16:27
tzy1023 发表于 2014-11-17 22:08
无论是在集群还是伪分布集群运行都是这样,这个就是自带的wordcount例子啊,程序本身是没问题的,网上搜 ...

集群安装有问题的可能性比较大
参考下面两个帖子,或许对你有帮助
hadoop2.X使用手册2:如何运行自带wordcount
hadoop2.2运行mapreduce(wordcount)问题总结

如果还不能解决,说明集群有问题,可以参考下面帖子,搭建集群
hadoop2.2完全分布式最新高可靠安装文档
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条