map reduce卡住

felix0221 · 发表于 2016-6-16 12:53:31

Joker 发表于 2016-6-16 09:59
没有具体的Job运行的log信息吗？你的lib和你的集群安装的lib是否一致的？

有的，但坦白說我看不太懂，不知道該取哪個Log，我貼過來看看。RpcActivityForPort50020 registered. 2016-06-16 12:49:55,636 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(slave-1:50010, storageID=DS-1061459805-120.97.32.114-50010-1464657472333, infoPort=50075, ipcPort=50020) 2016-06-16 12:49:55,639 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.1.1.189:50010, storageID=DS-1061459805-120.97.32.114-50010-1464657472333, infoPort=50075, ipcPort=50020)In DataNode.run, data = FSDataset{dirpath='/usr/local/hadoop/data1/current,/usr/local/hadoop/data2/current'} 2016-06-16 12:49:55,641 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2016-06-16 12:49:55,641 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2016-06-16 12:49:55,642 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50020: starting 2016-06-16 12:49:55,643 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 50020: starting 2016-06-16 12:49:55,643 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: using BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec 2016-06-16 12:49:55,643 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 50020: starting 2016-06-16 12:49:55,904 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 6063 blocks took 136 msec to generate and 122 msecs for RPC and NN processing 2016-06-16 12:49:55,905 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic block scanner. 2016-06-16 12:49:56,226 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification succeeded for blk_4627584800478203878_74626 2016-06-16 12:50:27,820 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-7200225616549542570_93408 src: /10.1.1.118:55638 dest: /10.1.1.189:50010 2016-06-16 12:50:27,831 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.118:55638, dest: /10.1.1.189:50010, bytes: 4, op: HDFS_WRITE, cliID: DFSClient_1030093440, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_-7200225616549542570_93408, duration: 2756169 2016-06-16 12:50:27,832 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-7200225616549542570_93408 terminating 2016-06-16 12:50:31,663 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Deleting block blk_-3875837162657267399_93407 file /usr/local/hadoop/data2/current/subdir23/subdir41/blk_-3875837162657267399 2016-06-16 12:50:31,664 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Deleting block blk_-1774515014920393950_89763 file /usr/local/hadoop/data2/current/subdir56/subdir58/blk_-1774515014920393950 2016-06-16 12:50:31,664 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Deleting block blk_-580209064372898878_86111 file /usr/local/hadoop/data1/current/blk_-580209064372898878 2016-06-16 12:52:10,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Deleting block blk_5911797076744834150_93406 file /usr/local/hadoop/data2/current/blk_5911797076744834150 2016-06-16 12:52:21,392 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-6700680431092273632_93410 src: /10.1.1.150:45046 dest: /10.1.1.189:50010 2016-06-16 12:52:21,400 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.150:45046, dest: /10.1.1.189:50010, bytes: 1603, op: HDFS_WRITE, cliID: DFSClient_820978766, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_-6700680431092273632_93410, duration: 3607122 2016-06-16 12:52:21,401 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-6700680431092273632_93410 terminating 2016-06-16 12:52:21,423 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_6412582820171929584_93411 src: /10.1.1.118:55658 dest: /10.1.1.189:50010 2016-06-16 12:52:21,426 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.118:55658, dest: /10.1.1.189:50010, bytes: 341, op: HDFS_WRITE, cliID: DFSClient_820978766, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_6412582820171929584_93411, duration: 1409572 2016-06-16 12:52:21,428 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_6412582820171929584_93411 terminating 2016-06-16 12:52:21,508 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_4688205799457954425_93412 src: /10.1.1.118:55662 dest: /10.1.1.189:50010 2016-06-16 12:52:21,517 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.118:55662, dest: /10.1.1.189:50010, bytes: 20556, op: HDFS_WRITE, cliID: DFSClient_820978766, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_4688205799457954425_93412, duration: 6355777 2016-06-16 12:52:21,519 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_4688205799457954425_93412 terminating 2016-06-16 12:52:21,703 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_6630805162166441298_93414 src: /10.1.1.118:55670 dest: /10.1.1.189:50010 2016-06-16 12:52:21,708 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.118:55670, dest: /10.1.1.189:50010, bytes: 20621, op: HDFS_WRITE, cliID: DFSClient_-1236540019, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_6630805162166441298_93414, duration: 2490221 2016-06-16 12:52:21,709 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_6630805162166441298_93414 terminating 2016-06-16 12:52:21,726 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-7821599669992056668_93415 src: /10.1.1.118:55674 dest: /10.1.1.189:50010 2016-06-16 12:52:21,731 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /10.1.1.118:55674, dest: /10.1.1.189:50010, bytes: 106, op: HDFS_WRITE, cliID: DFSClient_1030093440, offset: 0, srvID: DS-1061459805-120.97.32.114-50010-1464657472333, blockid: blk_-7821599669992056668_93415, duration: 3630575 2016-06-16 12:52:21,731 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-7821599669992056668_93415 terminating 2016-06-16 12:52:26,298 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification succeeded for blk_-4255848265022210512_64323 2016-06-16 12:53:27,831 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-5116250755033604854_93409 src: /10.1.1.118:55682 dest: /10.1.1.189:50010 2016-06-16 12:53:28,011 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Received block blk_-5116250755033604854_93409 src: /10.1.1.118:55682 dest: /10.1.1.189:50010 of size 25934052