Storm 非HA 部署

鸡汤: 有时候你会觉得自己就是个SB，别灰心少年，至少感觉是对的。

1. 环境

序列号	IP 地址	主机名	角色	安装软件
1	192.168.186.10	master	storm master	zookeeper,jdk1.8,storm0.9.3
2	192.168.186.11	slave	storm slave1	zookeeper,jdk1.8,storm0.9.3
3	192.168.186.12	slave	storm slave2	zookeeper,jdk1.8,storm0.9.3

下载地址: http://mirror.bit.edu.cn/apache/storm/ http://storm.apache.org/downloads.html https://archive.apache.org/dist/storm/

2. 部署

  storm在1.x之前nimbus是不支持HA的了，接来下我会分别部署两个不同版本。他们部署几乎一样。

2.1 JDK 检查

安装之前线检查jdk

[root@master ~]# java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

其他两台机器省略，保证三台机器能均安装JDK

2.2 zookeeper 检查

同时检查zookeeper是否安装[storm 依赖zookeeper]

[root@master ~]# /usr/local/zookeeper/bin/zkServer.sh  status
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader

[root@slave1 ~]# /usr/local/zookeeper/bin/zkServer.sh  status
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower

[root@slave2 ~]# /usr/local/zookeeper/bin/zkServer.sh  status
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower

保证有zookeeper集群

2.3 部署

分布式部署，官网参考链接: http://storm.apache.org/releases/1.2.3/Setting-up-a-Storm-cluster.html

[root@master conf]# grep 192* /etc/hosts
192.168.186.10 master
192.168.186.11 slave1
192.168.186.12 slave2
[root@master ~]# cd /usr/local/src/
[root@master src]# ls apache-storm-0.9.3.master.tgz 
apache-storm-0.9.3.master.tgz
[root@master src]# tar xf apache-storm-0.9.3.master.tgz 
[root@master conf]# cd /usr/local/
[root@master local]# mv src/apache-storm-0.9.3 /usr/local/
[root@master local]# ln -sf /usr/local/apache-storm-0.9.3 /usr/local/storm
[root@master local]# mkdir -p /usr/local/storm/data  # data是存放storm数据地方

2.4 环境变量

[root@master storm]# tail -2 /etc/profile
export STORM_HOME=/usr/local/storm
export PATH=$PATH:$STORM_HOME/bin
[root@master storm]# source /etc/profile
[root@master conf]#  egrep -v '#|^$' storm.yaml 
storm.zookeeper.servers:
    - "master"
    - "slave1"
    - "slave2"
nimbus.host: "master"
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703
    - 6704
storm.local.dir: "/usr/local/storm/data

高版本废除了nimbus.host参数，而采用了nimbus.seeds参数替代nimbus.host，主要用于发现nimbus leader

配置文件

# 设置Zookeeper的主机名称
storm.zookeeper.servers:
     - "master"
     - "slave1"
     - "slave2"

# 设置主节点的主机名称
nimbus.seeds: "master"

# 设置Storm的数据存储路径
storm.local.dir: "/usr/local/storm/data"

# 设置Worker的端口号
supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

注意 - 67*参数 这几个前面是 空格 不是 TAB键

2.5 包分发

[root@master ~]# rsync -az /usr/local/{storm,apache-storm-0.9.3} slave1:/usr/local/
[root@master ~]# rsync -az /usr/local/{storm,apache-storm-0.9.3} slave2:/usr/local/
[root@slave1 ~]# tail -2 /etc/profile
export STORM_HOME=/usr/local/storm
export PATH=$PATH:$STORM_HOME/bin
[root@slave1 ~]# source /etc/profile
[root@slave2 ~]# tail -2 /etc/profile
export STORM_HOME=/usr/local/storm
export PATH=$PATH:$STORM_HOME/bin
[root@slave2 ~]# source /etc/profile

2.6 启动集群

• master 节点启动nimbus、supervisor、ui和logviewer

/usr/local/storm/bin/storm nimbus >/dev/null 2>&1 &
/usr/local/storm/bin/storm supervisor >/dev/null 2>&1 &
/usr/local/storm/bin/storm ui >/dev/null 2>&1 &
/usr/local/storm/bin/storm logviewer >/dev/null 2>&1 &

[root@master ~]# /usr/local/storm/bin/storm nimbus >/dev/null 2>&1 &
[1] 65667
[root@master ~]# /usr/local/storm/bin/storm supervisor >/dev/null 2>&1 &
[2] 65791
[root@master ~]# /usr/local/storm/bin/storm ui >/dev/null 2>&1 &
[3] 65906
[root@master ~]# /usr/local/storm/bin/storm logviewer >/dev/null 2>&1 &
[4] 66057

检查
[root@master ~]# jps|egrep 'nimbus|supervisor|logviewer|core'
65667 nimbus
65906 core
66057 logviewer
65791 supervisor

core 是 UI

• slave 节点启动Supervisor

/usr/local/storm/bin/storm supervisor >/dev/null 2>&1 &
/usr/local/storm/bin/storm logviewer >/dev/null 2>&1 &

[root@slave1 ~]# /usr/local/storm/bin/storm supervisor >/dev/null 2>&1 &
[2] 18527
[root@slave2 ~]# /usr/local/storm/bin/storm supervisor >/dev/null 2>&1 &
[2] 28714

[root@slave1 ~]# /usr/local/storm/bin/storm logviewer >/dev/null 2>&1 &
[3] 18691
[root@slave2 ~]# /usr/local/storm/bin/storm logviewer >/dev/null 2>&1 &
[3] 28859

检查
[root@slave1 ~]# jps|egrep 'supervisor|logviewer'
18691 logviewer
18527 supervisor
[root@slave2 ~]# jps|egrep 'supervisor|logviewer'
28714 supervisor
28859 logviewer

logviewer，可以在web页面点击相应的端口号即可查看日志 UI，是web展示

2.7 UI 登录

2.8 Storm命令行操作

1）nimbus：启动nimbus守护进程
    storm nimbus

2）supervisor：启动supervisor守护进程
    storm supervisor

3）ui：启动UI守护进程。
    storm ui

4）list：列出正在运行的拓扑及其状态
    storm list

5）logviewer：Logviewer提供一个web接口查看Storm日志文件。
    storm logviewer

6）jar：
storm jar 【jar路径】 【拓扑包名.拓扑类名】 【拓扑名称】

7）kill：杀死名为Topology-name的拓扑
    storm kill topology-name [-w wait-time-secs]
    -w：等待多久后杀死拓扑

8）active：激活指定的拓扑spout。
storm activate topology-name

9）deactivate：禁用指定的拓扑Spout。
    storm deactivate topology-name 

10）help：打印一条帮助消息或者可用命令的列表。
    storm help
    storm help <command>