Openstack存储swift多代理节点安装配置

问题导读：
1. Openstack存储swift多代理节点如何安装配置？
2. 如何新增一个节点？










Openstack的存储多代理的架构国内比较有名就是新浪SAE部署的Swift集群

本文参考Openstack官网openstack-install-guide-essex-ubuntu.pdf和os-compute-starterguide-essex.pdf文档，文档汉化版本貌似还没有，如需下载请移步Openstack官网

回归正题，以下是配置过程简单步骤

一、当前配置环境

系统：Ubuntu Server 12.04×64 存储设置：vdb1 20G

架构部署
主机名                 IP                    作用
Proxy            192.168.100.7         代理节点
object1          192.168.100.3        存储节点1(zone1)
object2          192.168.100.4        存储节点2(zone2)

增加代理节点
Proxybak         192.168.100.8      代理做冗余备份
复制代码

二、配置部署

1、安装代理节点Proxy

apt-get update
apt-get install swift swift-proxy memcached python-keystone python-keystoneclient
复制代码

2、创建swift.conf

cat >/etc/swift/swift.conf <<EOF
[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = `od -t x8 -N 8 -A n </dev/random`
EOF
复制代码

3、创建Swift 的工作目录

<b>mkdir -p /etc/swift
chown -R swift:swift /etc/swift/</b>
复制代码

4、配置memcached ，监听默认的端口

perl -pi -e "s/-l 127.0.0.1/-l 192.168.100.7/" /etc/memcached.conf
service memcached restart
复制代码

5、创建/etc/swift/proxy-server.conf

[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift

[pipeline:main]
pipeline = catch_errors healthcheck cache authtoken keystone proxy-server

[app:proxy-server]
use = egg:swift#proxy
account_autocreate = true

[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = admin, swiftoperator

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
# Delaying the auth decision is required to support token-less
# usage for anonymous referrers ('.r:*').
delay_auth_decision = 1
auth_protocol = http
auth_host = 192.168.3.97
auth_port = 35357
auth_token = admin
service_protocol = http
service_host = 192.168.3.97
service_port = 5000
admin_token = admin
admin_tenant_name = service
admin_user = swift
admin_password = swift

[filter:cache]
use = egg:swift#memcache
memcache_servers = 192.168.100.7:11211
set log_name = cache

[filter:catch_errors]
use = egg:swift#catch_errors

[filter:healthcheck]
use = egg:swift#healthcheck
复制代码

6、创建account,container,object

cd /etc/swift
swift-ring-builder object.builder create 7 2 1
swift-ring-builder container.builder create 7 2 1
swift-ring-builder account.builder create 7 2 1

创建Ring
需要记录每个Store Node /srv/node 中记录的存储设备
swift-ring-builder account.builder add z1-192.168.100.3:6002/vdb1 1
swift-ring-builder container.builder add z1-192.168.100.3:6001/vdb1 1
swift-ring-builder object.builder add z1-192.168.100.3:6000/vdb1 1
swift-ring-builder account.builder add z2-192.168.100.4:6002/vdb1 1
swift-ring-builder container.builder add z2-192.168.100.4:6001/vdb1 1
swift-ring-builder object.builder add z2-192.168.100.4:6000/vdb1 1

确认ring 的内容是否正确:
swift-ring-builder account.builder
swift-ring-builder container.builder
swift-ring-builder object.builder

平衡(Rebalance) ring
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance

开启Proxy 服务
chown -R swift:swift /etc/swift
#需要存储节点启动后再启动
swift-init proxy start
复制代码

三、部署配置存储节点

存储节点二台配置基本一致，此处只以其中一个为例

1、安装

apt-get update
apt-get install swift swift-account swift-container swift-object xfsprogs
复制代码

2、创建一个新的分区

fdisk /dev/vdb
#创建xfs 文件系统
mkfs.xfs -i size=1024 /dev/vdb1
#添加到启动时挂载分区
echo "/dev/vdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab

mkdir -p /srv/node/vdb1
mount /dev/vdb1 /srv/node/vdb1
chown -R swift:swift /srv/node
复制代码

3、创建Swift 的工作目录

mkdir -p /etc/swift
chown -R swift:swift /etc/swift/
复制代码

4、从proxy 节点中复制配置文件

scp 192.168.100.7:/etc/swift/swift.conf /etc/swift/
scp 192.168.100.7:/etc/swift/object.ring.gz /etc/swift/
scp 192.168.100.7:/etc/swift/container.ring.gz /etc/swift/
scp 192.168.100.7:/etc/swift/account.ring.gz /etc/swift/
复制代码

5、创建/etc/rsyncd.conf

uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 192.168.100.3 (第二台存储即写ip：192.168.100.4)
[account]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock
复制代码

6、修改/etc/default/rsync 中的RSYNC_ENABLE= 属性

perl -pi -e 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync
service rsync start
复制代码

7、创建/etc/swift/account-server.conf

cat >/etc/swift/account-server.conf < [DEFAULT]
devices = /srv/node
mount_check = false
bind_port = 6002
user = swift
log_facility = LOG_LOCAL2
bind_ip =0.0.0.0
workers = 2
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]
EOF
复制代码

8、创建/etc/swift/container-server.conf

cat >/etc/swift/container-server.conf < [DEFAULT]
devices = /srv/node
mount_check = false
bind_port = 6001
user = swift
log_facility = LOG_LOCAL3
bind_ip =0.0.0.0
workers = 2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]
EOF
复制代码

9、创建/etc/swift/object-server.conf

cat >/etc/swift/object-server.conf < [DEFAULT]
devices = /srv/node
mount_check = false
bind_port = 6000
user = swift
log_facility = LOG_LOCAL4
bind_ip = 0.0.0.0
workers = 2
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
EOF
复制代码

10、开启存储节点服务

swift-init object-server start
swift-init object-replicator start
swift-init object-updater start
swift-init object-auditor start
swift-init container-server start
swift-init container-replicator start
swift-init container-updater start
swift-init container-auditor start
swift-init account-server start
swift-init account-replicator start
swift-init account-auditor start
复制代码

四、测试存储的配置是否可以使用，此时的swift架构就是1个代理多个存储的部署，代理存在单点故障隐患。

1、到192.168.100.7服务器开启代理服务
2、修改控制节点192.168.3.97认证的keystone，配置Endpoints部分。（PS：ip：192.168.3.97为Openstack的控制节点ip地址，swift我采用keystone认证）
3、dashboard 和命令行测试，此处不再描述

五、新增一个代理节点Proxybak

1、按步骤一，安装配置Proxybak节点，完全复制/etc/swift 目录下内容到新Proxybak节点，配置过程不再写。

注意，swift.conf配置文件的hash值不要修改保持一致。修改proxy-server.conf 中的IP 设置为Proxybak的IP，Ring文件直接拷贝第一个代理节点的，此节点不需要在重新生成。

2、启动代理服务

3、检测，Proxybak的代理服务启动后，应该可以获取之前Proxy上传的所有数据，同时在Proxybak上传数据，在Proxy节点也可以看到。

到此也就实现了，多代理后端存储的功能，连接任何一个代理服务器都可以看到后端的存储内容。

为了保证Openstack云存储swift的代理冗余性和稳定性，我们可以采用Nginx实现代理的负载均衡。

Nginx实现swift代理的负载均衡参考这篇《用Nginx负载均衡Openstack对象存储swift》

引用：http://hiying.net/post-43.html