分享

storm(ubuntu12.04)入门及分布式集群的搭建

pig2 发表于 2014-8-26 02:08:07 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 2 26811

问题导读
1.storm安装包,在哪可以下载?
2.storm依赖哪些环境?
3.通过什么命令可以启动Nimbus?






环境:
storm0.9.2
ubuntu12.04

     一、什么是STORM?
  在上一篇文章《mac10.9下eclipse的storm开发环境搭建》中,已经有详细的介绍。

  二、STORM的官方网站在哪里?
  http://storm.incubator.apache.org/

  三、在哪里下载?需要哪些组件的支持?
  Storm0.9.2在下面的地址可以下载:
  http://www.apache.org/dyn/closer.cgi/incubator/storm/apache-storm-0.9.2-incubating/apache-storm-0.9.2-incubating.tar.gz

  1)、根据官方文档描述,主要依赖于JDK和Python。
  Next you need to install Storm’s dependencies on Nimbus and the workermachines. These are:
  • Java 6
  • Python 2.6.6

  2)、Zookeeper集群的依赖


  四、如何安装?
  1)、解压下载到的apache-storm-0.9.2-incubating.zip


  2)、编辑storm.yaml配置文件
  1. ### ldir
  2. storm.local.dir: "/home/hadoop/storm-0.9.2-incubating/ldir"
  3. ### zookeeper
  4. storm.zookeeper.servers:
  5.     - "m1"
  6.     - "m2"
  7.     - "s1"
  8.     - "s2"
  9. ### nimbus host
  10. nimbus.host: "m1"
  11. ### ui.* configs are for the master
  12. ui.port: 8081 # 我这里修改了storm的ui端口
  13. ########### These MUST be filled in for a storm configuration
  14. # storm.zookeeper.servers:
  15. #     - "server1"
  16. #     - "server2"
  17. #
  18. # nimbus.host: "nimbus"
  19. #
  20. #
  21. # ##### These may optionally be filled in:
  22. #   
  23. ## List of custom serializations
  24. # topology.kryo.register:
  25. #     - org.mycompany.MyType
  26. #     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
  27. #
  28. ## List of custom kryo decorators
  29. # topology.kryo.decorators:
  30. #     - org.mycompany.MyDecorator
  31. #
  32. ## Locations of the drpc servers
  33. # drpc.servers:
  34. #     - "server1"
  35. #     - "server2"
  36. ## Metrics Consumers
  37. # topology.metrics.consumer.register:
  38. #   - class: "backtype.storm.metric.LoggingMetricsConsumer"
  39. #     parallelism.hint: 1
  40. #   - class: "org.mycompany.MyMetricsConsumer"
  41. #     parallelism.hint: 1
  42. #     argument:
  43. #       - endpoint: "metrics-collector.mycompany.org"
复制代码

  3)、将m1下的storm的目录复制到其它发服务器下去
  1. root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@m2:/home/hadoop
  2. root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@s1:/home/hadoop
  3. root@m1:/home/hadoop# scp -r /home/hadoop/storm-0.9.2-incubating root@s2:/home/hadoop
复制代码

  4)、在m1上启动Nimbus
  1. root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/storm-0.9.2-incubating/bin/storm nimbus &
  2. [1] 1171
  3. root@m1:/home/hadoop/storm-0.9.2-incubating# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.nimbus
复制代码

  5)、在s1、s2上启动Supervisor

  1. root@s1:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
  2. [1] 31158
  3. root@s1:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
  4. ------上面的代码是s1上的---------------------下面的代码是s2上的-------------------
  5. root@s2:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm supervisor &
  6. [1] 32676
  7. root@s2:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating/conf -Xmx256m -Dlogfile.name=supervisor.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.supervisor
复制代码

  6)、在m1上启动Storm UI
  1. root@m1:/home/hadoop# /home/hadoop/storm-0.9.2-incubating/bin/storm ui &
  2. [1] 1221
  3. root@m1:/home/hadoop# Running: /usr/lib/jvm/java-7-oracle/bin/java -server -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/storm-0.9.2-incubating:/home/hadoop/storm-0.9.2-incubating/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/home/hadoop/storm-0.9.2-incubating/logback/cluster.xml backtype.storm.ui.core
复制代码

在浏览器打开:http://m1:8081/index.html,可以看到如下界面:


  7)、Zookeeper下查看是否有对应storm的目录,在下面的第29行,可以看出zookeeper已经管理storm了。

  1. root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/zookeeper-3.4.5/bin/zkCli.sh
  2. Connecting to localhost:2181
  3. 2014-08-02 12:54:04,172 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
  4. 2014-08-02 12:54:04,178 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=m1
  5. 2014-08-02 12:54:04,179 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.7.0_65
  6. 2014-08-02 12:54:04,179 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
  7. 2014-08-02 12:54:04,180 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-7-oracle/jre
  8. 2014-08-02 12:54:04,180 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/hadoop/zookeeper-3.4.5/bin/../build/classes:/home/hadoop/zookeeper-3.4.5/bin/../build/lib/*.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/netty-3.2.2.Final.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/log4j-1.2.15.jar:/home/hadoop/zookeeper-3.4.5/bin/../lib/jline-0.9.94.jar:/home/hadoop/zookeeper-3.4.5/bin/../zookeeper-3.4.5.jar:/home/hadoop/zookeeper-3.4.5/bin/../src/java/lib/*.jar:/home/hadoop/zookeeper-3.4.5/bin/../conf:/usr/lib/jvm/java-7-oracle/lib
  9. 2014-08-02 12:54:04,181 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
  10. 2014-08-02 12:54:04,181 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
  11. 2014-08-02 12:54:04,182 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
  12. 2014-08-02 12:54:04,182 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
  13. 2014-08-02 12:54:04,183 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
  14. 2014-08-02 12:54:04,183 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.11.0-15-generic
  15. 2014-08-02 12:54:04,184 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
  16. 2014-08-02 12:54:04,184 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
  17. 2014-08-02 12:54:04,185 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/hadoop/storm-0.9.2-incubating
  18. 2014-08-02 12:54:04,187 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@19b1ebe5
  19. 2014-08-02 12:54:04,210 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@966] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
  20. Welcome to ZooKeeper!
  21. 2014-08-02 12:54:04,233 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@849] - Socket connection established to localhost/127.0.0.1:2181, initiating session
  22. JLine support is enabled
  23. 2014-08-02 12:54:04,260 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x14794a964e20006, negotiated timeout = 30000
  24. WATCHER::
  25. WatchedEvent state:SyncConnected type:None path:null
  26. [zk: localhost:2181(CONNECTED) 0] ls /
  27. [storm, hadoop-ha, hbase, zookeeper]
  28. [zk: localhost:2181(CONNECTED) 1] quit
  29. Quitting...
  30. 2014-08-02 12:54:11,130 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down
  31. 2014-08-02 12:54:11,131 [myid:] - INFO  [main:ZooKeeper@684] - Session: 0x14794a964e20006 closed
  32. root@m1:/home/hadoop/storm-0.9.2-incubating#
复制代码

  8)、storm的测试,将《mac10.9下eclipse的storm开发环境搭建》一文章中的Eclipse项目,打包导出为idoall_storm.jar,并上传到storm的根目录
  1. root@m1:/home/hadoop/storm-0.9.2-incubating# ll
  2. 总用量 140
  3. drwxr-xr-x 11 root   root    4096 Aug  2 12:51 ./
  4. drwxr-xr-x 39 hadoop hadoop  4096 Aug  1 14:38 ../
  5. drwxr-xr-x  2 root   root    4096 Aug  1 14:38 bin/
  6. -rw-r--r--  1    502 staff  34239 Jun 13 08:46 CHANGELOG.md
  7. drwxr-xr-x  2 root   root    4096 Aug  2 12:31 conf/
  8. -rw-r--r--  1    502 staff    538 Mar 13 11:17 DISCLAIMER
  9. drwxr-xr-x  3    502 staff   4096 May  6 03:13 examples/
  10. drwxr-xr-x  3 root   root    4096 Aug  1 14:38 external/
  11. -rw-r--r--  1 root   root    8984 Aug  2 11:25 idoall_storm.jar
  12. drwxr-xr-x  3 root   root    4096 Aug  2 12:51 ldir/
  13. drwxr-xr-x  2 root   root    4096 Aug  1 14:38 lib/
  14. -rw-r--r--  1    502 staff  22822 Jun 12 04:07 LICENSE
  15. drwxr-xr-x  2 root   root    4096 Aug  1 14:38 logback/
  16. drwxr-xr-x  2 root   root    4096 Aug  1 15:07 logs/
  17. -rw-r--r--  1    502 staff    981 Jun 11 01:10 NOTICE
  18. drwxr-xr-x  5 root   root    4096 Aug  1 14:38 public/
  19. -rw-r--r--  1    502 staff   7445 Jun 10 02:24 README.markdown
  20. -rw-r--r--  1    502 staff     17 Jun 17 00:22 RELEASE
  21. -rw-r--r--  1    502 staff   3581 May 30 00:20 SECURITY.md
  22. root@m1:/home/hadoop/storm-0.9.2-incubating# /home/hadoop/storm-0.9.2-incubating/bin/storm jar idoall_storm.jar storm.starter.WordCountTopology word_count
  23. Running: /usr/lib/jvm/java-7-oracle/bin/java -client -Dstorm.options= -Dstorm.home=/home/hadoop/storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/storm-0.9.2-incubating/idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:idoall_storm.jar:/home/hadoop/storm-0.9.2-incubating/conf:/home/hadoop/storm-0.9.2-incubating/bin -Dstorm.jar=idoall_storm.jar storm.starter.WordCountTopology word_count
  24. 500  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
  25. 516  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar idoall_storm.jar to assigned location: /home/hadoop/storm-0.9.2-incubating/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567.jar
  26. 533  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /home/hadoop/storm-0.9.2-incubating/ldir/nimbus/inbox/stormjar-8a07e8d6-ac9d-4347-ac3a-321906349567.jar
  27. 534  [main] INFO  backtype.storm.StormSubmitter - Submitting topology word_count in distributed mode with conf {"topology.workers":3,"topology.debug":true}
  28. 1057 [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: word_count
  29. root@m1:/home/hadoop/storm-0.9.2-incubating#
复制代码

这时再打开浏览器http://m1:8081/index.html,可以看到我们刚才提产的代码已经在集群中显示了。
130353qgkg9es5p075fzk5.png


  五、FAQ
  1)过程中发现storm ui显示supervisor个数与实际不符的解决  
  解决方法:在删除了local dir,重启后恢复了。











已有(2)人评论

跳转到指定楼层
gibson915 发表于 2014-9-5 11:28:09
回复

使用道具 举报

loreting 发表于 2014-11-7 17:45:00
提示: 作者被禁止或删除 内容自动屏蔽
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条