立即注册 登录
About云-梭伦科技 返回首页

xuanxufeng的个人空间 https://www.aboutyun.com/?53 [收藏] [复制] [分享] [RSS]

日志

Ceilometer与Gnocchi整合

已有 2961 次阅读2017-4-10 09:59


1 准备虚拟环境

yum install -y python-setuptools python-devel

easy_install virtualenv virtualenvwrapper pip

pip install Flask -i http://pypi.douban.com/simple

mkdir ~/workspaces

echo "export WORKON_HOME=~/workspaces" >> .bashrc

source /usr/bin/virtualenvwrapper.sh

 

2 分别安装Ceilometer(kilo)Goncchi(master)

源码安装ceilometer

(env)#git clone https://github.com/openstack/ceilometer.git

(env)#cd ceilometer

(env)#git checkout -b kilo remotes/origin/stable/kilo

(env)#python setup.py install

(env)#tox -e genconfig

(env)#mkdir /etc/ceilometer

(env)#cp etc/ceilometer/* /etc/ceilometer/

(env)# ls /etc/ceilometer/

api_paste.ini  event_definitions.yaml  pipeline.yaml      policy.json  ceilometer.conf  event_pipeline.yaml

 

同样方法安装python-ceilometerclient

(env)#git clone https://github.com/openstack/python-ceilometerclient

(env)#cd python-ceilometerclient/

(env)#git checkout -b kilo remotes/origin/stable/kilo

(env)#python setup.py install

 

源码安装gnocchi

请参考:http://blog.sina.com.cn/s/blog_6de3aa8a0102vgop.html

(env)#python setup.py install

(env)#tox -e genconfig

 

 

3 整合配置

gnocchi的配置基本按照http://blog.sina.com.cn/s/blog_6de3aa8a0102vgop.html即可。

ceilometer的配置如下:vim /etc/ceilometer/ceilometer.conf

[DEFAULT]

rabbit_host=192.168.10.120

rabbit_port=5672

rabbit_hosts=192.168.10.120:5672

rabbit_use_ssl=False

rabbit_userid=guest

rabbit_password=guest

rabbit_virtual_host=/

rabbit_ha_queues=False

notification_topics=notifications

rpc_backend=ceilometer.openstack.common.rpc.impl_kombu

pipeline_cfg_file=pipeline.yaml

dispatcher=gnocchi

debug=False

verbose=True

log_dir=/var/log/ceilometer

use_syslog=False

policy_file=policy.json

[alarm]

partition_rpc_topic=alarm_partition_coordination

evaluation_interval=60

record_history=True

evaluation_service=ceilometer.alarm.service.SingletonAlarmService

[api]

port=8777

host=0.0.0.0

[coordination]

backend_url=redis://192.168.10.120:6379

 

[database]

metering_connection = mysql://root:password@127.0.0.1/ceilometer?charset=utf8

event_connection = mysql://root:password@127.0.0.1/ceilometer?charset=utf8

alarm_connection = mysql://root:password@127.0.0.1/ceilometer?charset=utf8

[event]

definitions_cfg_file=event_definitions.yaml

drop_unmatched_notifications=true

[keystone_authtoken]

auth_host=192.168.10.120

auth_port=35357

auth_protocol=http

auth_uri=http://192.168.10.120:5000/

admin_user=ceilometer

admin_password=password

admin_tenant_name=services

[notification]

ack_on_event_error=True

store_events=True

[publisher]

metering_secret=password

[service_credentials]

os_username=ceilometer

os_password=password

os_tenant_name=services

os_auth_url=http://192.168.10.120:35357/v2.0

os_region_name=RegionOne

[alarms]

gnocchi_url = http://192.168.10.120:8041

[dispatcher_gnocchi]

filter_project = admin

filter_service_activity = False

archive_policy = low

url = http://192.168.10.120:8041

archive_policy_file = /etc/ceilometer/gnocchi_archive_policy_map.yaml

 

/etc/ceilometer/目录下创建gnocchi_archive_policy_map.yaml文件

# cat /etc/ceilometer/gnocchi_archive_policy_map.yaml

# This file is used to map a metric name to corresponding archive policy

# and used by the ceilometer dispatcher.

# Format: :

cpu_utils: "high"

disk.*: "low"

volume.*: "high"


4 配置其它组件服务实现ceilometer的监控功能

Compute

Image

Block Storage

Networking

Object Stroage

以上配置请参考官方文档。

http://docs.openstack.org/juno/install-guide/install/yum/content/ch_ceilometer.html

 

5 启动ceilometergnocchi的相关进程

首先创建archive_policy,可以参考以下脚本

# cat post_archive_policy.sh

RAW_TOKEN=`curl -s -X POST http://127.0.0.1:5000/v2.0/tokens -H "Content-Type: application/json"  -d '{"auth": {"tenantName": "'"$OS_TENANT_NAME"'", "passwordCredentials":{"username": "'"$OS_USERNAME"'", "password": "'"$OS_PASSWORD"'"}}}'`

#echo $RAW_TOKEN

TOKEN=`echo $RAW_TOKEN | python -c "import sys; import json; tok = json.loads(sys.stdin.read()); print tok['access']['token']['id'];"`

#echo $TOKEN

 

curl -H "Content-Type: application/json" \

     -H "X-Auth-Token: $TOKEN" \

     -X POST \

     -d '{  "name": "low",

            "back_window": 0,

            "definition": [

                {

                    "granularity": "1s",

                    "timespan": "1 hour"

                },

                {

                    "points": 48,

                    "timespan": "1 day"

                }

              ]

            }' \

       http://192.168.10.120:8041/v1/archive_policy  \

| python -m json.tool


若创建成功,则可以在数据库表中查询到。

 

启动各个服务

(env)#gnocchi-api

(env)#ceilometer-polling

(env)#ceilometer-collector

(env)#ceilometer-agent-central

(env)#ceilometer-agent-notification

(env)#ceilometer-agent-compute

(env)#ceilometer-api

(env)#ceilometer-agent-ipmi

(env)#ceilometer-alarm-evaluator

(env)#ceilometer-alarm-notifier

 

6 修订一些问题和bug

(1)      ceilometer-agent-compute报错如下:

2015-05-06 22:51:22.383 1002 TRACE ceilometer.compute.pollsters.net

2015-05-06 22:51:22.386 1002 ERROR ceilometer.compute.pollsters.net [-] Ignoring instance instance-00000018: 'NoneType' object has no attribute 'libvirtError'

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net Traceback (most recent call last):

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net   File "/root/workspaces/env/lib/python2.7/site-packages/ceilometer/compute/pollsters/net.py", line 97, in get_samples

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net     instance,

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net   File "/root/workspaces/env/lib/python2.7/site-packages/ceilometer/compute/pollsters/net.py", line 84, in _get_vnics_for_instance

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net     self._get_vnic_info(inspector, instance)

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net   File "/root/workspaces/env/lib/python2.7/site-packages/ceilometer/compute/virt/libvirt/inspector.py", line 126, in inspect_vnics

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net     domain = self._get_domain_not_shut_off_or_raise(instance)

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net   File "/root/workspaces/env/lib/python2.7/site-packages/ceilometer/compute/virt/libvirt/inspector.py", line 113, in _get_domain_not_shut_off_or_raise

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net     domain = self._lookup_by_uuid(instance)

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net   File "/root/workspaces/env/lib/python2.7/site-packages/ceilometer/compute/virt/libvirt/inspector.py", line 50, in decorator

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net     except libvirt.libvirtError as e:

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net AttributeError: 'NoneType' object has no attribute 'libvirtError'

2015-05-06 22:51:22.386 1002 TRACE ceilometer.compute.pollsters.net

解决办法:

说明ceilometer+gnocchi的虚拟环境中确实libvirt库,执行安装即可

(env) # easy_install pyton-libvirt 

另外需要注意的是上面配置文件中涉及到的目录或文件,当系统没有时需要手动创建,以免带来不必要的麻烦。

 

2)如果你发现各个表中都没有数据,resource表,metric表等全是空的(不像下图那样)。


ceilometer-collector进程报错,如下:

2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher    self.dispatcher_manager.map_method(self.method, payload)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/stevedore/extension.py", line 251, in map_method
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     method_name, *args, **kwds)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/stevedore/extension.py", line 224, in map
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     self._invoke_one_plugin(response.append, func, e, args, kwds)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/stevedore/extension.py", line 255, in _invoke_one_plugin
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     response_callback(func(e, *args, **kwds))
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/stevedore/extension.py", line 229, in _call_extension_method
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     return getattr(extension.obj, method_name)(*args, **kwds)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 238, in record_metering_data
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     ext, resource_id, metric_name, list(samples),
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 89, in log_and_ignore
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     func(self, *args, **kwargs)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 267, in _process_samples
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     resource_attributes = self._get_resource_attributes(
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 311, in _get_resource_attributes
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     (metric_name, self.get_archive_policy(metric_name))
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 311, in
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     (metric_name, self.get_archive_policy(metric_name))
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 142, in get_archive_policy
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     policy_match = self._match_metric(metric_name)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py", line 163, in _match_metric
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     # Support wild cards such as disk.*
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib64/python2.7/fnmatch.py", line 43, in fnmatch
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     return fnmatchcase(name, pat)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib64/python2.7/fnmatch.py", line 75, in fnmatchcase
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     res = translate(pat)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher   File "/root/workspaces/env/lib64/python2.7/fnmatch.py", line 87, in translate
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher     i, n = 0, len(pat)
2015-05-06 19:00:50.495 18146 TRACE oslo_messaging.notify.dispatcher TypeError: object of type 'int' has no len()

解决方法:

请参考https://review.openstack.org/#/c/180409/ 中我提的patch.

修改env/lib/python2.7/site-packages/gnocchi/ceilometer/dispatcher.py +158

将 for metric, policy in enumerate(self.gnocchi_archive_policy_data):

换为 for metric, policy in self.gnocchi_archive_policy_data.items():

因为在处理gnocchi_archive_policy_map.yaml文件时进行正则表达宽匹配时,函数fnmatch.fnmatch(metric_name, metric)传入的参数metric是整数据(0,1,2….),固没有len()方法,根源是使用了enumerate()函数,获得metric的值不是预期的。

 

3)当你创建虚拟机用的flavorID是自动生成类似于'9212d3f4-58e3-4a3a-8f2a-eec60f2e717a'时:

ceilometer-collector进程报错,如下:

oslo_messaging.notify.dispatcher ValueError: invalid literal for int() with base 10: '9212d3f4-58e3-4a3a-8f2a-eec60f2e717a'

解决方法:

请参考https://review.openstack.org/#/c/180380/ 中我提的patch.

修改env/lib/python2.7/site-packages/gnocchi/ceilometer/resources/instance.py文件第3337行,将int(metadata['instance_flavor_id'])换为metadata['instance_flavor_id']

       int(metadata["flavor"]["id"])换为metadata["flavor"]["id"]

并且,修改env/lib/python2.7/site-packages/gnocchi/rest/__init__.py +765

       "flavor_id": int 换为 "flavor_id": six.text_type

最后,将gnocchi数据库instance表中的flavor_id字段的类型改为VARCHAR 255,或者将gnocchi数据库中的表全部删去,重新执行gnocchi-dbsync


4)当你创建的volume或者image没有名称时(openstack命令行下是允许的),默认是None,其它字段也类似,这样的资源gnocchi无法记录监控数据,因为在创建resourcemeitric时报错创建失败。

原因是在env/lib/python2.7/site-packages/gnocchi/ceilometer/resources目录下image.py  instance.pynetwork.pyvolume.py这些文件中都有extra_attributes,在获得其值的时候没有将None转换为Unicode类型,其代码默认这些字段都是完整的,正常的string,非None的。所以如有必要可以加str()强制转换为Unicode来避免错误。


 


路过

雷人

握手

鲜花

鸡蛋

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 立即注册

关闭

推荐上一条 /2 下一条