storm on yarn最简单安装

问题导读：
    我们知道了storm on yarn的架构和意义，那么如何一步一步安装storm on yarn呢？






storm on yarn下载地址：https://github.com/yahoo/storm-yarn
下载好的storm-yarn-master.zip 得放到linux进行解压，在windows解压的话，lib目录下的软连接会丢失
上面在window下，可能获取不到，通过Linux获取：

wget https://github.com/yahoo/storm-yarn/archive/master.zip
复制代码

或则百度网盘：
链接：http://pan.baidu.com/s/1mgqP5wO 密码：clv1

1:storm on yarn 需要编译
unzip storm-yarn-master.zip

然后进入 storm-yarn-master 目录通过如下命令进行编译

mvn package -DskipTests

注意：我用的maven-3.1.1 和jdk1.7.0_45

编译好后解压storm-yarn-master/lib/storm-0.9.0-wip21.zip，得到 storm-0.9.0-wip21目录。

将得到 storm-0.9.0-wip21 目录移动到 和 storm-yarn-master同级。
最终目录是

/usr/local/storm/storm-yarn-master
/usr/local/storm/storm-0.9.0-wip21

然后配置storm的启停用户的环境变量

vim ~/.basarc
添加如下环境变量


export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar


export STORM_WORK=/opt/storm
export STORM_HOME=$STORM_WORK
export PATH=$PATH:$STORM_WORK/storm-yarn-master/bin:$STORM_WORK/storm-0.9.0-wip21/bin

export HADOOP_INSTALL=/opt/hadoop
export HADOOP_HOME=$HADOOP_INSTALL
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL


2:将编译好后的storm-yarn-master/lib/storm.zip 添加进hdfs中，可以通过如下命令
hadoop fs -mkdir -p /lib/storm/0.9.0-wip21hadoop fs -moveFromLocal storm.zip /lib/storm/0.9.0-wip21

3：需要在hdfs里面添加storm工作目录,并将storm工作目录的所有者设置为storm job提交用户
hadoop fs -mkdir -p /user/storm
hadoop fs -chown storm /user/storm

4：确保storm的启停用户配置环境变量如下：
export JAVA_HOME=/usr/java/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export STORM_WORK=/usr/local/storm
export STORM_HOME=$STORM_WORK
export PATH=$PATH:$STORM_WORK/storm-yarn-master/bin:$STORM_WORK/storm-0.9.0-wip21/bin
export HADOOP_INSTALL=/usr/local/hadoop
export HADOOP_HOME=$HADOOP_INSTALL
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL

5:启动storm
storm-yarn launch /usr/local/storm/storm-0.9.0-wip21/conf/master.yaml

6：关闭storm
storm-yarn shutdown -appId application_1398822427232_0143 -output ~/.storm/storm.yaml

7:运行wordcount
storm jar /opt/storm/storm-yarn-master/lib/myStorm-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.myStorm.App WordCountTopology -c nimbus.host=192.168.109.241

其中nimbus.host是你提交storm到yarn后，yarn会给你分配一个地址，你得去自己找哦。




总结，有时候发现supervisor启动不来，会发现是内存资源不够。在虚拟机环境的同志要注意这点哦。





转载自：http://blog.csdn.net/jiushuai/article/details/26693311

最近正准备用上，GOOD