mapreduce报错 java.net.ConnectException: Connection refused

nextuser 发表于 2016-11-1 08:30
问题的关键已经找到了。如果可以尝试解决下。
如果不行，可以贴出相关配置来，特别是hdfs的访问端口

Call From master/10.162.30.129 to localhost:50868 failed on connection exception
每次执行ｍａｐｒｅｄｕｃｅ任务的时候　出现的错误端口还不一样？

稍等  我把所有xml 配置信息 都贴出来

Wyy_Ck 发表于 2016-11-1 10:44
Call From master/10.162.30.129 to localhost:50868 failed on connection exception
每次执行ｍａｐｒ ...

数据副本，
<property>
               <name>dfs.replication</name>
               <value>3</value>
        </property>

没有配置。参考的什么文档，还是自己配置的。
还有你配置了两个节点？？
最好配置三个。
配置信息，可以参考这个
hadoop（2.x）以hadoop2.2为例完全分布式最新高可靠安装文档
http://www.aboutyun.com/forum.php?mod=viewthread&tid=7684

Wyy_Ck 发表于 2016-11-1 10:44
Call From master/10.162.30.129 to localhost:50868 failed on connection exception
每次执行ｍａｐｒ ...

还有跟你格式化什么关系。
格式化是否成功。
贴出来看下。
你出现的错误，贴出截图来

nextuser 发表于 2016-11-1 15:57
数据副本，

               dfs.replication

我增加了一个子节点， 现在2个子节点， 就这样没用修改什么  就OK了！  如下，　这样就ＯＫ了吗　　

[mw_shl_code=bash,true]hadoop@master ~]$ hadoop jar /opt/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar  wordcount /data/input /data/output
16/11/02 16:17:21 INFO client.RMProxy: Connecting to ResourceManager at master/10.162.30.129:8032
16/11/02 16:17:21 INFO input.FileInputFormat: Total input paths to process : 1
16/11/02 16:17:21 INFO mapreduce.JobSubmitter: number of splits:1
16/11/02 16:17:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1478068499295_0002
16/11/02 16:17:22 INFO impl.YarnClientImpl: Submitted application application_1478068499295_0002
16/11/02 16:17:22 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1478068499295_0002/
16/11/02 16:17:22 INFO mapreduce.Job: Running job: job_1478068499295_0002
16/11/02 16:17:27 INFO mapreduce.Job: Job job_1478068499295_0002 running in uber mode : false
16/11/02 16:17:27 INFO mapreduce.Job:  map 0% reduce 0%
16/11/02 16:17:33 INFO mapreduce.Job:  map 100% reduce 0%
16/11/02 16:17:39 INFO mapreduce.Job:  map 100% reduce 100%
16/11/02 16:17:39 INFO mapreduce.Job: Job job_1478068499295_0002 completed successfully
16/11/02 16:17:39 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=57
                FILE: Number of bytes written=229629
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=135
                HDFS: Number of bytes written=31
                HDFS: Number of read operations=6
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=1
                Launched reduce tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=4077
                Total time spent by all reduces in occupied slots (ms)=2946
                Total time spent by all map tasks (ms)=4077
                Total time spent by all reduce tasks (ms)=2946
                Total vcore-seconds taken by all map tasks=4077
                Total vcore-seconds taken by all reduce tasks=2946
                Total megabyte-seconds taken by all map tasks=4174848
                Total megabyte-seconds taken by all reduce tasks=3016704
        Map-Reduce Framework
                Map input records=1
                Map output records=7
                Map output bytes=59
                Map output materialized bytes=57
                Input split bytes=104
                Combine input records=7
                Combine output records=5
                Reduce input groups=5
                Reduce shuffle bytes=57
                Reduce input records=5
                Reduce output records=5
                Spilled Records=10
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=131
                CPU time spent (ms)=1580
                Physical memory (bytes) snapshot=435048448
                Virtual memory (bytes) snapshot=4218990592
                Total committed heap usage (bytes)=311951360
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=31
        File Output Format Counters
                Bytes Written=31[/mw_shl_code]

Wyy_Ck 发表于 2016-11-2 16:25
我增加了一个子节点， 现在2个子节点， 就这样没用修改什么  就OK了！  如下，　这样就ＯＫ了吗　　

[ ...

对的，就是这样的

nextuser 发表于 2016-11-2 16:53
对的，就是这样的

卡死了  [mw_shl_code=bash,true][hadoop@master ~]$ hadoop fs -put /opt/test/wordcount/wordcount /data/input
[hadoop@master ~]$ hadoop jar /opt/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar  wordcount /data/input /data/output
16/11/02 18:04:05 INFO client.RMProxy: Connecting to ResourceManager at master/10.162.30.129:8032
16/11/02 18:04:06 INFO input.FileInputFormat: Total input paths to process : 1
16/11/02 18:04:06 INFO mapreduce.JobSubmitter: number of splits:1
16/11/02 18:04:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1478080130552_0008
16/11/02 18:04:07 INFO impl.YarnClientImpl: Submitted application application_1478080130552_0008
16/11/02 18:04:07 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1478080130552_0008/
16/11/02 18:04:07 INFO mapreduce.Job: Running job: job_1478080130552_0008
16/11/02 18:04:24 INFO mapreduce.Job: Job job_1478080130552_0008 running in uber mode : false
16/11/02 18:04:24 INFO mapreduce.Job:  map 0% reduce 0%[/mw_shl_code]