mapreduce报错 java.net.ConnectException: Connection refused

弄了半天，系统是centos 7，本想执行一个测试下，如下（文件已经上传到/data/input）：
hadoop jar /opt/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar  wordcount /data/input /opt/test/result
报错如下，实在焦虑，帮忙看看：


16/10/30 10:47:11 INFO mapreduce.Job: Job job_1477704898495_0008 failed with state FAILED due to: Application application_1477704898495_0008 failed 2 times due to Error launching appattempt_1477704898495_0008_000002. Got exception: java.net.ConnectException: Call From master/10.162.30.129 to localhost:36109 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
    at org.apache.hadoop.ipc.Client.call(Client.java:1480)
    at org.apache.hadoop.ipc.Client.call(Client.java:1407)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
    at com.sun.proxy.$Proxy83.startContainers(Unknown Source)
    at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
    at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119)
    at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:609)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:707)
    at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
    at org.apache.hadoop.ipc.Client.call(Client.java:1446)`enter code here`



感谢各位！

原因太多了：
1.确保集群已经启动了，查看是否有僵尸进程
2.防火墙是否关闭
3.hdfs-site.xml文件里面的权限访问是否设置为false


4.hosts里面是如何配置的

以上四项如果不确定，最好都贴出图来

langke93 发表于 2016-10-31 15:23
原因太多了：
1.确保集群已经启动了，查看是否有僵尸进程
2.防火墙是否关闭

感谢您的回复；稍后我把相关配置全部贴出来，  就是感觉问题很多，无从下手。。。。。。。。。

1, master:
[mw_shl_code=bash,true] [hadoop@master hadoop]$ jps
91651 NameNode
91891 SecondaryNameNode
111023 Jps
92078 ResourceManager[/mw_shl_code]

slave:
[mw_shl_code=applescript,true][root@slave hdfs]# jps
28647 DataNode
28792 NodeManager
44376 Jps[/mw_shl_code]

2. 防火墙使用systemcl stop firewall.service 已经关闭了centos 系统
    [mw_shl_code=shell,true][hadoop@master wordcount]$ systemctl status firewalld.service
firewalld.service
   Loaded: masked (/dev/null)
   Active: inactive (dead) since Wed 2016-10-12 17:27:10 CST; 2 weeks 4 days ago
Main PID: 816 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/firewalld.service[/mw_shl_code]
3. 主从节点 hdfs-site.xml：
    [mw_shl_code=xml,true]<configuration>
     <property>
       <name>dfs.namenode.name.dir</name>
       <value>file:///data/hadoop/storage/hdfs/name</value>
    </property>
    <property>
       <name>dfs.datanode.data.dir</name>
       <value>file:///data/hadoop/storage/hdfs/data</value>
    </property>
    <property>
       <name>dfs.datanode.http-address</name>
       <value>10.162.30.162:50075</value>
    </property>
    <property>
       <name>dfs.permissions</name>
       <value>false</value>
    </property>
    <property>
      <name>dfs.secondary.http.address</name>
      <value>master:50090</value>
    </property>
    <property>
      <name>dfs.http.address</name>
      <value>master:50070</value>
    </property>
    <property>
       <name>dfs.datanode.ipc.address</name>
       <value>10.162.30.162:50020</value>
    </property>
    <property>
       <name>dfs.datanode.address</name>
       <value>10.162.30.162:50010</value>
    </property>
</configuration>[/mw_shl_code]

4. hosts文件：
   [mw_shl_code=shell,true]
#127.0.0.1 localhost
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.162.30.129 master
10.162.30.162 slave[/mw_shl_code]

Wyy_Ck 发表于 2016-10-31 15:28
感谢您的回复；稍后我把相关配置全部贴出来，  就是感觉问题很多，无从下手。。。。。。。。。

已经回复在下面， 这样的配置 还是报错  

Wyy_Ck 发表于 2016-10-31 17:39
已经回复在下面， 这样的配置 还是报错

Call From master/10.162.30.129 to localhost:36109
上面端口是在哪配置的

nextuser 发表于 2016-10-31 21:09
Call From master/10.162.30.129 to localhost:36109
上面端口是在哪配置的

这个端口我没有配置，  而且每一次hadoop namenode -format后  这个端口还不一样    我也没有找到这个端口哪里的

Wyy_Ck 发表于 2016-10-31 21:23
这个端口我没有配置，  而且每一次hadoop namenode -format后  这个端口还不一样    我也没有找到这个端 ...

问题的关键已经找到了。如果可以尝试解决下。
如果不行，可以贴出相关配置来，特别是hdfs的访问端口

nextuser 发表于 2016-11-1 08:30
问题的关键已经找到了。如果可以尝试解决下。
如果不行，可以贴出相关配置来，特别是hdfs的访问端口

所有配置信息：
core-site.xml：
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://master:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/data/hadoop/storage/tmp</value>
    </property>
</configuration>

hdfs-site.xml 上面已经贴出/////////////////////////

mapred-site.xml：
<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>master:10020</value>
    </property>
    <property>
        <name>mapred.job.tracker</name>
        <value>master:9001</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>master:19888</value>
    </property>
</configuration>


yarn-site.xml:

       <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>master</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>master:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>master:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>master:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>master:8033</value>
    </property>
</configuration>



当前报错信息端口又变了，是不是这个需要在哪里配置，否则会随机起端口？

16/11/01 09:29:53 INFO mapreduce.Job: Job job_1477905891965_0005 failed with state FAILED due to: Application application_1477905891965_0005 failed 2 times due to Error launching appattempt_1477905891965_0005_000002. Got exception: java.net.ConnectException: Call From master/10.162.30.129 to localhost:47222 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

nextuser 发表于 2016-11-1 08:30
问题的关键已经找到了。如果可以尝试解决下。
如果不行，可以贴出相关配置来，特别是hdfs的访问端口

所有配置信息：
core-site.xml：
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://master:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/data/hadoop/storage/tmp</value>
    </property>
</configuration>

hdfs-site.xml 上面已经贴出/////////////////////////

mapred-site.xml：
<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>master:10020</value>
    </property>
    <property>
        <name>mapred.job.tracker</name>
        <value>master:9001</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>master:19888</value>
    </property>
</configuration>


yarn-site.xml:

       <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>master</value>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>master:8030</value>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>master:8031</value>
    </property>
    <property>
        <name>yarn.resourcemanager.address</name>
        <value>master:8032</value>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>master:8033</value>
    </property>
</configuration>



当前报错信息端口又变了，是不是这个需要在哪里配置，否则会随机起端口？

16/11/01 09:29:53 INFO mapreduce.Job: Job job_1477905891965_0005 failed with state FAILED due to: Application application_1477905891965_0005 failed 2 times due to Error launching appattempt_1477905891965_0005_000002. Got exception: java.net.ConnectException: Call From master/10.162.30.129 to localhost:47222 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)