分享

wordcount结果出来了,可还是有问题求大神帮解决下

第一个问题
15/11/17 17:48:21 INFO mapreduce.Job: Job job_1447748706370_0002 running in uber mode : false
第二个问题
Caused by: java.net.ConnectException: Call From zhou-virtual-machine/127.0.1.1 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

15/11/17 17:47:08 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/11/17 17:47:10 INFO input.FileInputFormat: Total input paths to process : 1
15/11/17 17:47:10 INFO mapreduce.JobSubmitter: number of splits:1
15/11/17 17:47:11 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1447748706370_0002
15/11/17 17:47:17 INFO impl.YarnClientImpl: Submitted application application_1447748706370_0002
15/11/17 17:47:17 INFO mapreduce.Job: The url to track the job: http://zhou-virtual-machine:8088/proxy/application_1447748706370_0002/
15/11/17 17:47:17 INFO mapreduce.Job: Running job: job_1447748706370_0002
15/11/17 17:48:21 INFO mapreduce.Job: Job job_1447748706370_0002 running in uber mode : false
15/11/17 17:48:23 INFO mapreduce.Job:  map 0% reduce 0%
15/11/17 17:50:01 INFO mapreduce.Job:  map 100% reduce 0%
15/11/17 17:52:02 INFO mapreduce.Job:  map 100% reduce 100%
15/11/17 17:52:19 INFO mapreduce.Job: Job job_1447748706370_0002 completed successfully
15/11/17 17:52:31 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=1836
                FILE: Number of bytes written=234835
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=1469
                HDFS: Number of bytes written=1306
                HDFS: Number of read operations=6
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=1
                Launched reduce tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=115414
                Total time spent by all reduces in occupied slots (ms)=101610
                Total time spent by all map tasks (ms)=115414
                Total time spent by all reduce tasks (ms)=101610
                Total vcore-seconds taken by all map tasks=115414
                Total vcore-seconds taken by all reduce tasks=101610
                Total megabyte-seconds taken by all map tasks=118183936
                Total megabyte-seconds taken by all reduce tasks=104048640
        Map-Reduce Framework
                Map input records=31
                Map output records=179
                Map output bytes=2055
                Map output materialized bytes=1836
                Input split bytes=103
                Combine input records=179
                Combine output records=131
                Reduce input groups=131
                Reduce shuffle bytes=1836
                Reduce input records=131
                Reduce output records=131
                Spilled Records=262
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=6895
                CPU time spent (ms)=5070
                Physical memory (bytes) snapshot=207556608
                Virtual memory (bytes) snapshot=3775623168
                Total committed heap usage (bytes)=138489856
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=1366
        File Output Format Counters
                Bytes Written=1306
15/11/17 17:52:38 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
15/11/17 17:52:39 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:40 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:41 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:42 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:43 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:44 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:45 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:46 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:47 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:48 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:48 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
15/11/17 17:52:49 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:50 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:51 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:52 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:53 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:54 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:55 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:56 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:57 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:58 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:52:59 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
15/11/17 17:53:00 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:01 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:02 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:03 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:04 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:05 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:06 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:07 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:08 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/11/17 17:53:09 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:10020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
java.io.IOException: java.net.ConnectException: Call From zhou-virtual-machine/127.0.1.1 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:343)
        at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:428)
        at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:568)
        at org.apache.hadoop.mapreduce.Job$1.run(Job.java:323)
        at org.apache.hadoop.mapreduce.Job$1.run(Job.java:320)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:320)
        at org.apache.hadoop.mapreduce.Job.isSuccessful(Job.java:616)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1323)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Call From zhou-virtual-machine/127.0.1.1 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
        at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
        at org.apache.hadoop.ipc.Client.call(Client.java:1480)
        at org.apache.hadoop.ipc.Client.call(Client.java:1407)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
        at com.sun.proxy.$Proxy14.getJobReport(Unknown Source)
        at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
        at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:324)
        ... 24 more
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:609)
        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:707)
        at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
        at org.apache.hadoop.ipc.Client.call(Client.java:1446)
        ... 32 more
zhou@zhou-virtual-machine:/usr/local/hadoop$ bin/hdfs dfs -ls /
Found 3 items
drwxr-xr-x   - zhou supergroup          0 2015-11-17 17:40 /input
drwxr-xr-x   - zhou supergroup          0 2015-11-17 17:52 /output
drwx------   - zhou supergroup          0 2015-11-17 17:44 /tmp
zhou@zhou-virtual-machine:/usr/local/hadoop$ bin/hdfs dfs -ls /output
Found 2 items
-rw-r--r--   1 zhou supergroup          0 2015-11-17 17:52 /output/_SUCCESS
-rw-r--r--   1 zhou supergroup       1306 2015-11-17 17:52 /output/part-r-00000
zhou@zhou-virtual-machine:/usr/local/hadoop$ bin/hdfs dfs -cat /output/part-r-00000
(BIS),        1
(ECCN)        1
(TSU)        1
(see        1
5D002.C.1,        1
740.13)        1
<http://www.wassenaar.org/>        1
Administration        1
Apache        1
BEFORE        1
BIS        1
Bureau        1
Commerce,        1
Commodity        1
Control        1
Core        1
Department        1
ENC        1
Exception        1
Export        2
For        1
Foundation        1
Government        1
Hadoop        1
Hadoop,        1
Industry        1
Jetty        1
License        1
Number        1
Regulations,        1
SSL        1
Section        1
Security        1
See        1
Software        2
Technology        1
The        4
This        1
U.S.        1
Unrestricted        1
about        1
algorithms.        1
and        6
and/or        1
another        1
any        1
as        1
asymmetric        1
at:        2
both        1
by        1
check        1
classified        1
code        1
code.        1
concerning        1
country        1
country's        1
country,        1
cryptographic        3
currently        1
details        1
distribution        2
eligible        1
encryption        3
exception        1
export        1
following        1
for        3
form        1
from        1
functions        1
has        1
have        1
http://hadoop.apache.org/core/        1
http://wiki.apache.org/hadoop/        1
if        1
import,        2
in        1
included        1
includes        2
information        2
information.        1
is        1
it        1
latest        1
laws,        1
libraries        1
makes        1
manner        1
may        1
more        2
mortbay.org.        1
object        1
of        5
on        2
or        2
our        2
performing        1
permitted.        1
please        2
policies        1
possession,        2
project        1
provides        1
re-export        2
regulations        1
reside        1
restrictions        1
security        1
see        1
software        2
software,        2
software.        2
software:        1
source        1
the        8
this        3
to        2
under        1
use,        2
uses        1
using        2
visit        1
website        1
which        2
wiki,        1
with        1
written        1
you        1
your        1
配置如下:



1.png
3.png
4.png

已有(4)人评论

跳转到指定楼层
Alkaloid0515 发表于 2015-11-17 19:45:46
第一个问题,楼主本身配置的就是非安全模式
hadoop入门手册1:hadoop【2.7.1】【多节点】集群配置【必知配置知识1】
http://www.aboutyun.com/thread-15609-1-1.html
建议阅读下相关hadoop入门知识

第二个问题,10020 端口楼主是在哪配置的,配置文件中并没有体现。

回复

使用道具 举报

lmlm1234 发表于 2015-11-18 15:14:48
可能是模式和安全问题,比较初级问题
回复

使用道具 举报

lanxueren121 发表于 2015-11-19 14:57:55
回复

使用道具 举报

koafty 发表于 2016-12-6 10:38:16
lanxueren121 发表于 2015-11-19 14:57
谢谢大神~问题已解决~~~

你怎么解决的,求分享
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条