分享

nutch 2.3 网站注入失败

aqi915 发表于 2015-8-27 18:19:15 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 7 14912

现在在linux 安装eclipse 并导入 ant eclipse 编译的nutch ,运行injectorjob想把已有的几个网站地址注入到hbase出错了,
而在runtime/deploy/bin/nutch inject /urls/seed.txt 是可以把网站注入到hbase的,这是什么原因呢?请大神指导下下呀,感激不尽呀!

报错日志:
datanode
2015-08-27 17:04:04,342 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-08-27 17:04:04,386 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2015-08-27 17:04:04,386 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 11 cluster_timestamp: 1440657120403 } attemptId: 1 } keyId: -290220289)
2015-08-27 17:04:04,538 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2015-08-27 17:04:05,231 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2015-08-27 17:04:05,288 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.gora.mapreduce.GoraOutputFormat not found
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.gora.mapreduce.GoraOutputFormat not found
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.call(MRAppMaster.java:478)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.call(MRAppMaster.java:458)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1560)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:458)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:377)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1518)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1515)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1448)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.gora.mapreduce.GoraOutputFormat not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.call(MRAppMaster.java:474)
        ... 11 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.gora.mapreduce.GoraOutputFormat not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 13 more


成功日志
dataname
2015-08-27 16:41:07,445 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-08-27 16:41:07,492 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2015-08-27 16:41:07,492 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 10 cluster_timestamp: 1440657120403 } attemptId: 1 } keyId: -290220289)
2015-08-27 16:41:07,758 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2015-08-27 16:41:08,411 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2015-08-27 16:41:08,481 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.gora.mapreduce.NullOutputCommitter
2015-08-27 16:41:08,575 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2015-08-27 16:41:08,576 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2015-08-27 16:41:08,579 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2015-08-27 16:41:08,580 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2015-08-27 16:41:08,580 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2015-08-27 16:41:08,604 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2015-08-27 16:41:08,604 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2015-08-27 16:41:08,606 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2015-08-27 16:41:08,677 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ktbigdata1:9000]
2015-08-27 16:41:08,703 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ktbigdata1:9000]
2015-08-27 16:41:08,733 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ktbigdata1:9000]
2015-08-27 16:41:08,740 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2015-08-27 16:41:08,807 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2015-08-27 16:41:09,041 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-08-27 16:41:09,164 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-08-27 16:41:09,164 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2015-08-27 16:41:09,174 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1440657120403_0010 to jobTokenSecretManager
2015-08-27 16:41:09,294 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1440657120403_0010 because: not enabled;
2015-08-27 16:41:09,314 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1440657120403_0010 = 216. Number of splits = 1
2015-08-27 16:41:09,314 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1440657120403_0010 = 0
2015-08-27 16:41:09,314 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1440657120403_0010Job Transitioned from NEW to INITED
2015-08-27 16:41:09,316 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1440657120403_0010.
2015-08-27 16:41:09,378 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-08-27 16:41:09,388 INFO [Socket Reader #1 for port 57489] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 57489
2015-08-27 16:41:09,408 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2015-08-27 16:41:09,408 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-08-27 16:41:09,409 INFO [IPC Server listener on 57489] org.apache.hadoop.ipc.Server: IPC Server listener on 57489: starting
2015-08-27 16:41:09,409 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at ktbigdata2/192.168.100.142:57489
2015-08-27 16:41:09,526 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-08-27 16:41:09,538 INFO [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2015-08-27 16:41:09,543 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2015-08-27 16:41:09,567 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2015-08-27 16:41:09,593 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2015-08-27 16:41:09,593 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2015-08-27 16:41:09,597 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2015-08-27 16:41:09,597 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2015-08-27 16:41:09,627 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 43037
2015-08-27 16:41:09,627 INFO [main] org.mortbay.log: jetty-6.1.26
2015-08-27 16:41:09,674 INFO [main] org.mortbay.log: Extract jar:file:/home/hadoop/hadoop-2.7.1/share/hadoop/yarn/hadoop-yarn-common-2.7.1.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_43037_mapreduce____.b6zjdo/webapp
2015-08-27 16:41:10,084 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:43037
2015-08-27 16:41:10,085 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 43037
2015-08-27 16:41:10,694 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2015-08-27 16:41:10,700 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE job_1440657120403_0010
2015-08-27 16:41:10,703 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-08-27 16:41:10,703 INFO [Socket Reader #1 for port 54992] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 54992
2015-08-27 16:41:10,711 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-08-27 16:41:10,711 INFO [IPC Server listener on 54992] org.apache.hadoop.ipc.Server: IPC Server listener on 54992: starting
2015-08-27 16:41:10,745 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2015-08-27 16:41:10,745 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2015-08-27 16:41:10,745 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2015-08-27 16:41:10,806 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /192.168.100.141:8030
2015-08-27 16:41:10,937 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: <memory:8192, vCores:32>
2015-08-27 16:41:10,937 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: default
2015-08-27 16:41:10,942 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2015-08-27 16:41:10,942 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10
2015-08-27 16:41:10,950 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2015-08-27 16:41:10,964 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1440657120403_0010Job Transitioned from INITED to SETUP
2015-08-27 16:41:10,967 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2015-08-27 16:41:10,970 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1440657120403_0010Job Transitioned from SETUP to RUNNING
2015-08-27 16:41:11,004 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved ktbigdata2 to /default-rack
2015-08-27 16:41:11,006 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved ktbigdata3 to /default-rack
2015-08-27 16:41:11,015 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1440657120403_0010_m_000000 Task Transitioned from NEW to SCHEDULED
2015-08-27 16:41:11,016 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1440657120403_0010_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-08-27 16:41:11,021 INFO [Thread-50] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:1024, vCores:1>
2015-08-27 16:41:11,131 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1440657120403_0010, File: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010/job_1440657120403_0010_1.jhist
2015-08-27 16:41:11,943 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2015-08-27 16:41:11,997 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1440657120403_0010: ask=4 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:14336, vCores:1> knownNMs=2
2015-08-27 16:41:13,043 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1
2015-08-27 16:41:13,052 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1440657120403_0010_01_000002 to attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:13,053 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2015-08-27 16:41:13,102 INFO [AsyncDispatcher event handler] org.apache.hadoop.yarn.util.RackResolver: Resolved ktbigdata3 to /default-rack
2015-08-27 16:41:13,117 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010/job.jar
2015-08-27 16:41:13,119 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010/job.xml
2015-08-27 16:41:13,120 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #0 tokens and #1 secret keys for NM use for launching container
2015-08-27 16:41:13,120 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 1
2015-08-27 16:41:13,121 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2015-08-27 16:41:13,212 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1440657120403_0010_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2015-08-27 16:41:13,217 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1440657120403_0010_01_000002 taskAttempt attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:13,219 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:13,219 INFO [ContainerLauncher #0] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : ktbigdata3:37633
2015-08-27 16:41:13,308 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1440657120403_0010_m_000000_0 : 13562
2015-08-27 16:41:13,309 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1440657120403_0010_m_000000_0] using containerId: [container_1440657120403_0010_01_000002 on NM: [ktbigdata3:37633]
2015-08-27 16:41:13,312 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1440657120403_0010_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2015-08-27 16:41:13,313 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: ATTEMPT_START task_1440657120403_0010_m_000000
2015-08-27 16:41:13,313 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1440657120403_0010_m_000000 Task Transitioned from SCHEDULED to RUNNING
2015-08-27 16:41:14,162 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1440657120403_0010: ask=4 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:13312, vCores:1> knownNMs=2
2015-08-27 16:41:17,937 INFO [Socket Reader #1 for port 54992] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1440657120403_0010 (auth:SIMPLE)
2015-08-27 16:41:17,956 INFO [IPC Server handler 1 on 54992] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1440657120403_0010_m_000002 asked for a task
2015-08-27 16:41:17,957 INFO [IPC Server handler 1 on 54992] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1440657120403_0010_m_000002 given task: attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:20,658 INFO [IPC Server handler 0 on 54992] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1440657120403_0010_m_000000_0 is : 0.0
2015-08-27 16:41:20,838 INFO [IPC Server handler 1 on 54992] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1440657120403_0010_m_000000_0 is : 1.0
2015-08-27 16:41:20,848 INFO [IPC Server handler 0 on 54992] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:20,849 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1440657120403_0010_m_000000_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP
2015-08-27 16:41:20,850 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_1440657120403_0010_01_000002 taskAttempt attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:20,850 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:20,850 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : ktbigdata3:37633
2015-08-27 16:41:20,880 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1440657120403_0010_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2015-08-27 16:41:20,896 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1440657120403_0010_m_000000_0
2015-08-27 16:41:20,897 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1440657120403_0010_m_000000 Task Transitioned from RUNNING to SUCCEEDED
2015-08-27 16:41:20,899 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2015-08-27 16:41:20,899 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1440657120403_0010Job Transitioned from RUNNING to COMMITTING
2015-08-27 16:41:20,905 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
2015-08-27 16:41:20,929 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent
2015-08-27 16:41:20,930 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1440657120403_0010Job Transitioned from COMMITTING to SUCCEEDED
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: We are finishing cleanly so this is the last retry
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services
2015-08-27 16:41:20,940 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0
2015-08-27 16:41:20,971 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010/job_1440657120403_0010_1.jhist to hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010-1440664858583-hadoop-inject+%2Ftemp%2Furls.txt-1440664880928-1-0-SUCCEEDED-default-1440664870951.jhist_tmp
2015-08-27 16:41:20,993 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010-1440664858583-hadoop-inject+%2Ftemp%2Furls.txt-1440664880928-1-0-SUCCEEDED-default-1440664870951.jhist_tmp
2015-08-27 16:41:20,995 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010/job_1440657120403_0010_1_conf.xml to hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010_conf.xml_tmp
2015-08-27 16:41:21,041 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010_conf.xml_tmp
2015-08-27 16:41:21,048 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010.summary_tmp to hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010.summary
2015-08-27 16:41:21,049 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010_conf.xml_tmp to hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010_conf.xml
2015-08-27 16:41:21,051 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010-1440664858583-hadoop-inject+%2Ftemp%2Furls.txt-1440664880928-1-0-SUCCEEDED-default-1440664870951.jhist_tmp to hdfs://ktbigdata1:9000/tmp/hadoop-yarn/staging/history/done_intermediate/hadoop/job_1440657120403_0010-1440664858583-hadoop-inject+%2Ftemp%2Furls.txt-1440664880928-1-0-SUCCEEDED-default-1440664870951.jhist
2015-08-27 16:41:21,051 INFO [Thread-64] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2015-08-27 16:41:21,052 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to
2015-08-27 16:41:21,053 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://ktbigdata1:19888/jobhistory/job/job_1440657120403_0010
2015-08-27 16:41:21,058 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2015-08-27 16:41:22,060 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:1 RackLocal:0
2015-08-27 16:41:22,061 INFO [Thread-64] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://ktbigdata1:9000 /tmp/hadoop-yarn/staging/hadoop/.staging/job_1440657120403_0010
2015-08-27 16:41:22,145 INFO [Thread-64] org.apache.hadoop.ipc.Server: Stopping server on 54992
2015-08-27 16:41:22,148 INFO [IPC Server listener on 54992] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 54992
2015-08-27 16:41:22,149 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2015-08-27 16:41:22,149 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted


.


已有(7)人评论

跳转到指定楼层
NEOGX 发表于 2015-8-27 18:46:39
找不到这个类
Class org.apache.gora.mapreduce.GoraOutputFormat

回复

使用道具 举报

aqi915 发表于 2015-8-27 19:22:00
NEOGX 发表于 2015-8-27 18:46
找不到这个类
Class org.apache.gora.mapreduce.GoraOutputFormat

这个好明显,就不知道为啥,怎么解决
回复

使用道具 举报

NEOGX 发表于 2015-8-27 19:38:40
aqi915 发表于 2015-8-27 19:22
这个好明显,就不知道为啥,怎么解决

是不是包没有引用进来
回复

使用道具 举报

aqi915 发表于 2015-8-27 19:41:12
NEOGX 发表于 2015-8-27 19:38
是不是包没有引用进来

这个有点可能 ,知道ant eclipse 与ant runtime  什么区别么?
回复

使用道具 举报

NEOGX 发表于 2015-8-27 19:47:12
aqi915 发表于 2015-8-27 19:41
这个有点可能 ,知道ant eclipse 与ant runtime  什么区别么?

这是从哪看到的命令
回复

使用道具 举报

aqi915 发表于 2015-8-27 19:51:19
NEOGX 发表于 2015-8-27 19:47
这是从哪看到的命令

nutch  目录下执行ant  就有了,不然你们用什么呢?纠结呀,,
回复

使用道具 举报

NEOGX 发表于 2015-8-27 20:11:07
aqi915 发表于 2015-8-27 19:51
nutch  目录下执行ant  就有了,不然你们用什么呢?纠结呀,,

都用maven,一般不用ant
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条