hadoop 2.4.1 集群安装

导读：

hadoop 2.4.1的安装与hadoop2.2安装基本差不多，如果hadoop2.2安装没有问题的话，hadoop2.4.1问题也不不大。在安装的过程中，一些准备工作反而做的更好，比如jdk的安装，ssh无密码登陆目录的创建等。详细参考hadoop2.2完全分布式最新高可靠安装文档

下面为hadoop2.4.1安装：

1：创建目录

[jifeng@feng01 hadoop]$ mkdir tmp[jifeng@feng01 hadoop]$ mkdir name[jifeng@feng01 hadoop]$ mkdir data[jifeng@feng01 hadoop]$ lsdata hadoop-1.2.1.tar.gz hadoop-2.4.1.tar.gzhadoop-1.2.1 hadoop-2.4.1 namehadoop-1.2.1-bin.tar.gz hadoop-2.4.1-src.tar.gz tmp[jifeng@feng01 hadoop]$

2：先其它2个节点复制hadoop-2.4.1

[jifeng@feng01 hadoop]$ scp -r ./hadoop-2.4.1 feng03:/home/jifeng/hadoop[jifeng@feng01 hadoop]$ scp -r ./hadoop-2.4.1 feng02:/home/jifeng/hadoop

3：格式化分布式文件系统

[jifeng@feng01 hadoop-2.4.1]$ ./bin/hdfs namenode -format14/07/31 15:40:14 INFO namenode.NameNode: STARTUP_MSG: /************************************************************STARTUP_MSG: Starting NameNodeSTARTUP_MSG: host = feng01/10.6.3.84STARTUP_MSG: args = [-format]STARTUP_MSG: version = 2.4.1STARTUP_MSG: classpath = /home/jifeng/hadoop/hadoop-2.4.1/etc/hadoop:。。。。。。。。。。。。。。。。/************************************************************SHUTDOWN_MSG: Shutting down NameNode at feng01/10.6.3.84************************************************************/[jifeng@feng01 hadoop-2.4.1]$

4：启动hadoop

启动dfs

./sbin/start-dfs.sh

[jifeng@feng01 hadoop-2.4.1]$ ./sbin/start-dfs.sh
14/07/31 16:20:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicableStarting namenodes on [Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.feng01]sed：-e 表达式 #1，字符 6：unknown option to `s'HotSpot(TM): ssh: HotSpot(TM): Temporary failure in name resolution64-Bit: ssh: 64-Bit: Temporary failure in name resolutionServer: ssh: Server: Temporary failure in name resolutionJava: ssh: Java: Temporary failure in name resolutionVM: ssh: VM: Temporary failure in name resolution-c: Unknown cipher type 'cd'warning:: ssh: warning:: Temporary failure in name resolutionhave: ssh: have: Temporary failure in name resolutionloaded: ssh: loaded: Temporary failure in name resolution...........You: ssh: You: Temporary failure in name resolutionfeng01: Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.feng01: It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.feng03: starting datanode, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/hadoop-jifeng-datanode-feng03.outfeng02: starting datanode, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/hadoop-jifeng-datanode-feng02.outfeng03: Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.feng03: It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.Starting secondary namenodes [Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.feng01]sed：-e 表达式 #1，字符 6：unknown option to `s'-c: Unknown cipher type 'cd'Java: ssh: Java: Temporary failure in name resolution64-Bit: ssh: 64-Bit: Temporary failure in name resolutionVM: ssh: VM: Temporary failure in name resolutionServer: ssh: Server: Temporary failure in name resolutionHotSpot(TM): ssh: HotSpot(TM): Temporary failure in name resolutionwarning:: ssh: warning:: Temporary failure in name resolutionYou: ssh: You: Temporary failure in name resolutionhave: ssh: have: Temporary failure in name resolutiondisabled: ssh: disabled: Temporary failure in name resolutionhave: ssh: have: Temporary failure in name resolutionmight: ssh: might: Temporary failure in name resolutionguard.: ssh: guard.: Temporary failure in name resolutionlibrary: ssh: library: Temporary failure in name resolutionloaded: ssh: loaded: Temporary failure in name resolutionwhich: ssh: which: Temporary failure in name resolutionThe: ssh: The: Temporary failure in name resolutionVM: ssh: VM: Temporary failure in name resolutionstack: ssh: stack: Temporary failure in name resolutionIt's: ssh: It's: Temporary failure in name resolution。。。。。。。。。。。noexecstack'.: ssh: noexecstack'.: Temporary failure in name resolutionfeng01: starting secondarynamenode, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/hadoop-jifeng-secondarynamenode-feng01.out'-z: ssh: '-z: Temporary failure in name resolutionto: ssh: to: Temporary failure in name resolutionfeng01: Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.feng01: It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.14/07/31 16:20:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

检测守护进程

[jifeng@feng01 hadoop-2.4.1]$ jps9114 SecondaryNameNode9232 Jps8832 NameNode

启动yarn

[jifeng@feng01 hadoop-2.4.1]$ ./sbin/start-yarn.shstarting yarn daemonsstarting resourcemanager, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/yarn-jifeng-resourcemanager-feng01.outJava HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/jifeng/hadoop/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.feng03: starting nodemanager, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/yarn-jifeng-nodemanager-feng03.outfeng02: starting nodemanager, logging to /home/jifeng/hadoop/hadoop-2.4.1/logs/yarn-jifeng-nodemanager-feng02.out[jifeng@feng01 hadoop-2.4.1]$ jps9599 Jps9336 ResourceManager9114 SecondaryNameNode8832 NameNode

启劢hdfs: ./sbin/start-dfs.sh

 此时在feng01上面运行的迕程有：namenode，secondarynamenode

 feng02和feng03上面运行的迕程有：datanode

 启劢yarn: ./sbin/start-yarn.sh

 此时在feng01上运行的迕程有：namenode，secondarynamenode，resourcemanager

 feng02和feng03上面运行的迕程有：datanode，nodemanager