分享

Spark Streaming程序跑得好好的 突然就挂了,日志里面也没有错误信息

zstu 发表于 2017-4-5 17:29:36 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 1 7356
我运行了一个Spark streaming程序,这个应用跑得好好的,然后就听了,然后多次重新启动都启动不了。
[mw_shl_code=java,true]17/04/05 09:25:33 INFO spark.SecurityManager: Changing view acls to: root
17/04/05 09:25:33 INFO spark.SecurityManager: Changing modify acls to: root
17/04/05 09:25:33 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
17/04/05 09:25:33 INFO yarn.Client: Submitting application 6 to ResourceManager
17/04/05 09:25:33 INFO impl.YarnClientImpl: Submitted application application_1491381169632_0006
17/04/05 09:25:34 INFO yarn.Client: Application report for application_1491381169632_0006 (state: ACCEPTED)
17/04/05 09:25:34 INFO yarn.Client:
         client token: N/A
         diagnostics: N/A
         ApplicationMaster host: N/A
         ApplicationMaster RPC port: -1
         queue: root.root
         start time: 1491384333041
         final status: UNDEFINED
         tracking URL: http://CDH-Virtual-Master:8088/proxy/application_1491381169632_0006/
         user: root
17/04/05 09:25:35 INFO yarn.Client: Application report for application_1491381169632_0006 (state: ACCEPTED)
17/04/05 09:25:35 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(null)
17/04/05 09:25:35 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> CDH-Virtual-Master, PROXY_URI_BASES -> http://CDH-Virtual-Master:8088/proxy/application_1491381169632_0006), /proxy/application_1491381169632_0006
17/04/05 09:25:35 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
17/04/05 09:25:36 INFO yarn.Client: Application report for application_1491381169632_0006 (state: RUNNING)
17/04/05 09:25:36 INFO yarn.Client:
         client token: N/A
         diagnostics: N/A
         ApplicationMaster host: 10.25.26.182
         ApplicationMaster RPC port: 0
         queue: root.root
         start time: 1491384333041
         final status: UNDEFINED
         tracking URL: http://CDH-Virtual-Master:8088/proxy/application_1491381169632_0006/
         user: root
17/04/05 09:25:36 INFO cluster.YarnClientSchedulerBackend: Application application_1491381169632_0006 has started running.
17/04/05 09:25:36 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 47736.
17/04/05 09:25:36 INFO netty.NettyBlockTransferService: Server created on 47736
17/04/05 09:25:36 INFO storage.BlockManagerMaster: Trying to register BlockManager
17/04/05 09:25:36 INFO storage.BlockManagerMasterEndpoint: Registering block manager 10.25.26.176:47736 with 511.5 MB RAM, BlockManagerId(driver, 10.25.26.176, 47736)
17/04/05 09:25:36 INFO storage.BlockManagerMaster: Registered BlockManager
17/04/05 09:25:38 INFO cluster.YarnClientSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (CDH-Virtual-Nodes1:41745) with ID 3
17/04/05 09:25:39 INFO storage.BlockManagerMasterEndpoint: Registering block manager CDH-Virtual-Nodes1:33286 with 511.5 MB RAM, BlockManagerId(3, CDH-Virtual-Nodes1, 33286)
17/04/05 09:25:40 INFO cluster.YarnClientSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (CDH-Virtual-Nodes2:36079) with ID 5
17/04/05 09:25:40 INFO storage.BlockManagerMasterEndpoint: Registering block manager CDH-Virtual-Nodes2:59322 with 511.5 MB RAM, BlockManagerId(5, CDH-Virtual-Nodes2, 59322)
17/04/05 09:25:40 INFO cluster.YarnClientSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (CDH-Virtual-Nodes2:36080) with ID 2
17/04/05 09:25:41 INFO storage.BlockManagerMasterEndpoint: Registering block manager CDH-Virtual-Nodes2:60759 with 511.5 MB RAM, BlockManagerId(2, CDH-Virtual-Nodes2, 60759)
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (CDH-Virtual-Nodes3:54465) with ID 4
17/04/05 09:25:41 INFO storage.BlockManagerMasterEndpoint: Registering block manager CDH-Virtual-Nodes3:37671 with 511.5 MB RAM, BlockManagerId(4, CDH-Virtual-Nodes3, 37671)
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (CDH-Virtual-Nodes3:54466) with ID 1
17/04/05 09:25:41 INFO storage.BlockManagerMasterEndpoint: Registering block manager CDH-Virtual-Nodes3:50843 with 511.5 MB RAM, BlockManagerId(1, CDH-Virtual-Nodes3, 50843)
17/04/05 09:25:41 INFO utils.VerifiableProperties: Verifying properties
17/04/05 09:25:41 INFO utils.VerifiableProperties: Property group.id is overridden to zhongtai_real_time
17/04/05 09:25:41 WARN utils.VerifiableProperties: Property serializer.class is not valid
17/04/05 09:25:41 INFO utils.VerifiableProperties: Property zookeeper.connect is overridden to
17/04/05 09:25:41 INFO spark.SparkContext: Invoking stop() from shutdown hook
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/metrics/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/api,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/json,null}
17/04/05 09:25:41 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs,null}
17/04/05 09:25:41 INFO ui.SparkUI: Stopped Spark web UI at http://10.25.26.176:4040
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Interrupting monitor thread
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Shutting down all executors
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Asking each executor to shut down
17/04/05 09:25:41 INFO cluster.YarnClientSchedulerBackend: Stopped
17/04/05 09:25:41 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/04/05 09:25:41 INFO storage.MemoryStore: MemoryStore cleared
17/04/05 09:25:41 INFO storage.BlockManager: BlockManager stopped
17/04/05 09:25:41 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
17/04/05 09:25:41 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
17/04/05 09:25:41 INFO spark.SparkContext: Successfully stopped SparkContext
17/04/05 09:25:41 INFO util.ShutdownHookManager: Shutdown hook called
17/04/05 09:25:41 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-82115324-14b7-482f-9c73-f10124c8f8e3/httpd-95395197-7673-482d-8689-1a8f08c6897d
17/04/05 09:25:41 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
17/04/05 09:25:41 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
17/04/05 09:25:41 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-82115324-14b7-482f-9c73-f10124c8f8e3
[/mw_shl_code]

已有(1)人评论

跳转到指定楼层
easthome001 发表于 2017-4-5 18:58:24
是不是跑完了。
在跑一个试试
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条