OpenStack使用Bosh部署CloudFoundry(二)

问题导读

1.Micro Bosh的作用是什么？
2.如何部署部署Micro Bosh？
3.如何验证部署是否成功？






本文介绍如何在OpenStack环境中部署Micro Bosh，Micro Bosh其实就是Bosh，不过是将分布式的Bosh浓缩到一个机器节点上，用于小规模化部署分布式的Bosh环境，事实上，也可以直接使用Micro Bosh来部署CloudFoundry，不过Micro Bosh只适用于小规模部署的情况，大规模的生产环境如果使用Micro Bosh还是不太适合；

下边是具体的部署步骤：

1、下载Micro Bosh Stemcell
[mw_shl_code=bash,true]root@bosh-cli:~# mkdir -p ~/bosh-workspace/stemcells
root@bosh-cli:~# cd ~/bosh-workspace/stemcells
root@bosh-cli:~# wget http://bosh-jenkins-artifacts.s3.amazonaws.com/bosh-stemcell/openstack/bosh-stemcell-latest-openstack-kvm-ubuntu.tgz[/mw_shl_code]


或者直接使用Bosh下载：

[mw_shl_code=bash,true]root@bosh-cli:~# bosh public stemcells
+---------------------------------------------+
| Name                                        |
+---------------------------------------------+
| bosh-stemcell-1623-aws-xen-ubuntu.tgz       |
| bosh-stemcell-1623-aws-xen-centos.tgz       |
| light-bosh-stemcell-1623-aws-xen-ubuntu.tgz |
| light-bosh-stemcell-1623-aws-xen-centos.tgz |
| bosh-stemcell-1623-openstack-kvm-ubuntu.tgz |
| bosh-stemcell-1623-vsphere-esxi-ubuntu.tgz  |
| bosh-stemcell-1623-vsphere-esxi-centos.tgz  |
+---------------------------------------------+
To download use `bosh download public stemcell <stemcell_name>'. For full url use --full.
root@bosh-cli:~# bosh download public stemcell bosh-stemcell-1623-openstack-kvm-ubuntu.tgz [/mw_shl_code]

鉴于国际出口带宽的限制，下载可能需要耗费一点时间，强烈建议国内同学使用直接下载的方式并且最好使用迅雷等具有断点续传功能的下载工具下载，等待过程中可以继续下面的步骤。
说明：本环境中所有使用到Stemcell模板的地方都统一使用bosh-stemcell-latest-openstack-kvm-ubuntu.tgz。

2、创建Manifest文件

[mw_shl_code=bash,true]root@bosh-cli:~# mkdir -p ~/bosh-workspace/deployments/microbosh-openstack
root@bosh-cli:~# cd ~/bosh-workspace/deployments/microbosh-openstack
root@bosh-cli:~# vi micro_bosh.yml[/mw_shl_code]
配置内容可以参照：
[mw_shl_code=bash,true]---
name: microbosh-openstack
   
logging:
  level: DEBUG
   
network:
  type: dynamic
  vip: 10.68.19.130 # Floating IP
   
resources:
  persistent_disk: 16384
  cloud_properties:
    instance_type: v1.small
   
cloud:
  plugin: openstack
  properties:
    openstack:
      auth_url: http://10.68.19.61:5000/v2.0
      username: VF     # openstack username
      api_key: 1       # openstack api_key
      tenant: VF_CF    # openstack tenant
      region: myregion # openstack region
      default_security_groups: ["default"] # using default security groups
      default_key_name: vkey  # key name as cf created earlier
      private_key: ~/vkey.pem # pem file by uploading yourself
   
apply_spec:
  properties:
    director:
      max_threads: 3
    hm:
      resurrector_enabled: true
    ntp:
      - 0.north-america.pool.ntp.org
      - 1.north-america.pool.ntp.org [/mw_shl_code]

注意其中的编写说明，文件内不太适合使用中文，故英文说明。

3、部署Micro Bosh

设置Micro Bosh 部署文件：
[mw_shl_code=bash,true]root@bosh-cli:~# cd ~/bosh-workspace/deployments
root@bosh-cli:~/bosh-workspace/deployments# bosh micro deployment microbosh-openstack
Deployment set to '/root/bosh-workspace/deployments/microbosh-openstack/micro_bosh.yml'[/mw_shl_code]

使用上边下载好的Stemcell部署Micro Bosh：

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh micro deploy ~/bosh-workspace/stemcells/bosh-stemcell-latest-openstack-kvm-ubuntu.tgz [/mw_shl_code]

部署成功提示bosh target切换信息：
[mw_shl_code=bash,true]WARNING! Your target has been changed to `https://10.68.19.130:25555'!
Deployment set to '/root/bosh-workspace/deployments/microbosh-openstack/micro_bosh.yml'
Deployed `microbosh-openstack/micro_bosh.yml' to `https://microbosh-openstack:25555', took 00:07:08 to complete [/mw_shl_code]

4、登陆并创建账号

Target到Micro Bosh的director
[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments#bosh target https://10.68.19.130:25555 [/mw_shl_code]

初始账号：admin/admin
[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh login
Your username: admin
Enter password: *****
Logged in as `admin' [/mw_shl_code]

创建账号，并使用创建的账号登陆
[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh create user
Enter new username: venusource
Enter new password: **********
Verify new password: **********
User `venusource' has been created
root@bosh-cli:~/bosh-workspace/deployments# bosh login
Your username: venusource
Enter password: **********
Logged in as `venusource'[/mw_shl_code]


查看bosh状态

[mw_shl_code=bash,true]root@bosh-cli:~# bosh target https://10.68.19.130
Target set to `microbosh-openstack'
root@bosh-cli:~# bosh status
Config
             /root/.bosh_config
   
Director
  Name       microbosh-openstack
  URL        https://10.68.19.130:25555
  Version    1.5.0.pre.1478 (release:a8e43f41 bosh:a8e43f41)
  User       venusource
  UUID       5d8c682c-290a-4194-bfc6-60248567ceab
  CPI        openstack
  dns        enabled (domain_name: microbosh)
  compiled_package_cache disabled
  snapshots  disabled
   
Deployment
  Manifest   /root/bosh-workspace/deployments/bosh-openstack/bosh-openstack.yml[/mw_shl_code]


##########################################################

本文介绍如何使用Micro Bosh来部署Bosh，部署需要创建8个虚拟机来部署8个Bosh组件，所以，至少得准备好8个Fixed IP，后边使用Bosh的时候需要访问Director组件及配置DNS，所以准备至少2个Floating IP，本文中为（10.68.19.131和10.68.19.132），Bosh组件的详细介绍可以参看BOSH组件介绍。

下面是部署步骤：

1、上传Bosh Stemcell

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/stemcells# bosh upload stemcell bosh-stemcell-latest-openstack-kvm-ubuntu.tgz

Verifying stemcell...
File exists and readable                                     OK
Using cached manifest...
Stemcell properties                                          OK

Stemcell info
-------------
Name:    bosh-openstack-kvm-ubuntu
Version: 1478

Checking if stemcell already exists...
No

Uploading stemcell...

latest-bosh-s: 100% |ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| 251.3MB  11.3MB/s Time: 00:00:22

Director task 1

Update stemcell
  extracting stemcell archive (00:00:06)
  verifying stemcell manifest (00:00:00)
  checking if this stemcell already exists (00:00:00)
  uploading stemcell bosh-stemcell/776 to the cloud (00:01:53)
  save stemcell bosh-stemcell/776 (f207bfcd-f0cb-4ea2-b753-022e6234ebfa) (00:00:00)
Done                    5/5 00:01:59

Task 1 done
Started         2013-12-16 09:51:59 UTC
Finished        2013-12-16 09:53:58 UTC
Duration        00:01:59

Stemcell uploaded and created[/mw_shl_code]

2、下载打包Bosh Release

[mw_shl_code=bash,true]root@bosh-cli:~# cd ~/bosh-workspace
root@bosh-cli:~# git clone git://github.com/cloudfoundry/bosh.git[/mw_shl_code]

耗费时间较长，请耐心等待。
[mw_shl_code=bash,true]root@bosh-cli:~# cd ~/bosh-workspace/bosh
root@bosh-cli:~# apt-get install libxslt-dev libxml2-dev libmysql-ruby libmysqlclient-dev libpq-dev
root@bosh-cli:~# bundle install --local
root@bosh-cli:~# bundle exec rake release:create_dev_release[/mw_shl_code]


可以先修改~/bosh-workspace/bosh/Gemfile的source为淘宝源：

[mw_shl_code=bash,true]source 'http://ruby.taobao.org'[/mw_shl_code]
打包结束得到以下输出：
[mw_shl_code=bash,true]bosh create release --force
Syncing blobs...

Building DEV release
---------------------------------

....

Release version: 13.1-dev
Release manifest: /root/bosh-workspace/bosh/release/dev_releases/bosh-13.1-dev.yml[/mw_shl_code]

3、上传Bosh Release

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace# bosh upload release ~/bosh-workspace/bosh/release/dev_releases/bosh-13.1-dev.yml[/mw_shl_code]

成功后得到以下输出：

[mw_shl_code=bash,true][mw_shl_code=bash,true]....

Creating new jobs
blobstore/3.1-dev (00:00:00)
powerdns/3.1-dev (00:00:00)
redis/3 (00:00:00)
nats/5 (00:00:00)
postgres/4.1-dev (00:00:00)
director/11.2-dev (00:00:00)
health_monitor/5.1-dev (00:00:00)
registry/0.1-dev (00:00:00)
Done 8/8 00:00:00

Release has been created
bosh/13.1-dev (00:00:00)
Done 1/1 00:00:00

Task 2 done
Started 2013-12-16 10:06:26 UTC
Finished 2013-12-16 10:06:46 UTC
Duration 00:00:20

Release uploaded[/mw_shl_code][/mw_shl_code]

4、确认上传

[mw_shl_code=bash,true]root@bosh-cli:~# bosh stemcells

+---------------------------+---------+--------------------------------------+
| Name                      | Version | CID                                  |
+---------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu | 1478    | e64c6da5-f2ab-48f2-b03d-79c29d1acc43 |
+---------------------------+---------+--------------------------------------+

Stemcells total: 1
root@bosh-cli:~# bosh releases

+------+-----------+-------------+
| Name | Versions  | Commit Hash |
+------+-----------+-------------+
| bosh | 13.1-dev* | a8e43f41+   |
+------+-----------+-------------+
(*) Currently deployed
(+) Uncommitted changes

Releases total: 1[/mw_shl_code]

5、创建Manifest文件

创建目录

[mw_shl_code=bash,true]root@bosh-cli:~# mkdir -p ~/bosh-workspace/deployments/bosh-openstack
root@bosh-cli:~# cd ~/bosh-workspace/deployments/bosh-openstack[/mw_shl_code]
复制模板文件

[mw_shl_code=bash,true]root@bosh-cli:~# cp ~/bosh-workspace/bosh/release/examples/bosh-openstack-dynamic.yml bosh-openstack.yml
[/mw_shl_code]
如果OpenStack中采用的是quantum网络组件，则使用下边的模板文件

[mw_shl_code=bash,true]root@bosh-cli:~# cp ~/bosh-workspace/bosh/release/examples/bosh-openstack-manual.yml bosh-openstack.yml[/mw_shl_code]
本次安装中使用nova-network网络组件，所以采用bosh-openstack-dynamic.yml，修改其中标记有# CHANGE 的配置项，下边是本环境中的配置，大家可参考：

[mw_shl_code=bash,true]---
name: bosh-openstack
director_uuid: 5d8c682c-290a-4194-bfc6-60248567ceab # CHANGE

release:
  name: bosh
  version: latest

compilation:
  workers: 3
  network: default
  reuse_compilation_vms: true
  cloud_properties:
    instance_type: v1.compile # CHANGE

update:
  canaries: 1
  canary_watch_time: 3000-120000
  update_watch_time: 3000-120000
  max_in_flight: 4
  max_errors: 1

networks:
  - name: floating
    type: vip
    cloud_properties: {}
  - name: default
    type: dynamic
    cloud_properties: {}

resource_pools:
  - name: common
    network: default
    size: 8
    stemcell:
      name: bosh-openstack-kvm-ubuntu
      version: latest
    cloud_properties:
      instance_type: v1.micro # CHANGE

jobs:
  - name: nats
    template: nats
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]

  - name: redis
    template: redis
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]

  - name: postgres
    template: postgres
    instances: 1
    resource_pool: common
    persistent_disk: 16384
    networks:
      - name: default
        default: [dns, gateway]

  - name: powerdns
    template: powerdns
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]
      - name: floating
        static_ips:
          - 10.68.19.131 # CHANGE

  - name: blobstore
    template: blobstore
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]

  - name: director
    template: director
    instances: 1
    resource_pool: common
    persistent_disk: 16384
    networks:
      - name: default
        default: [dns, gateway]
      - name: floating
        static_ips:
          - 10.68.19.132 # CHANGE

  - name: registry
    template: registry
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]

  - name: health_monitor
    template: health_monitor
    instances: 1
    resource_pool: common
    networks:
      - name: default
        default: [dns, gateway]

properties:
  nats:
    address: 0.nats.default.bosh-openstack.microbosh
    user: nats
    password: nats

  redis:
    address: 0.redis.default.bosh-openstack.microbosh
    password: redis

  postgres: &bosh_db
    host: 0.postgres.default.bosh-openstack.microbosh
    user: postgres
    password: postgres
    database: bosh

  dns:
    address: 10.68.19.131 # CHANGE
    db: *bosh_db
    recursor: 10.68.19.130 # CHANGE

  blobstore:
    address: 0.blobstore.default.bosh-openstack.microbosh
    agent:
      user: agent
      password: agent
    director:
      user: director
      password: director

  director:
    name: bosh
    address: 0.director.default.bosh-openstack.microbosh
    db: *bosh_db

  registry:
    address: 0.registry.default.bosh-openstack.microbosh
    db: *bosh_db
    http:
      user: registry
      password: registry

  hm:
    http:
      user: hm
      password: hm
    director_account:
      user: admin
      password: admin
    resurrector_enabled: true

  ntp:
    - 0.north-america.pool.ntp.org
    - 1.north-america.pool.ntp.org

  openstack:
    auth_url: http://10.68.19.61:5000/v2.0 # CHANGE
    username: VF # CHANGE
    api_key: 1 # CHANGE
    tenant: VF_CF # CHANGE
    region: myregion # CHANGE
    default_security_groups: ["default"] # CHANGE
    default_key_name: vkey # CHANGE[/mw_shl_code]
注意：director_uuid是Micro Bosh的Director的UUID，可以使用bosh status查看并复制到Manifest文件中。

6、部署Bosh

设置Bosh的部署文件

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh deployment ~/bosh-workspace/deployments/bosh-openstack/bosh-openstack.yml
Deployment set to `/root/bosh-workspace/deployments/bosh-openstack/bosh-openstack.yml'[/mw_shl_code]
执行部署命令

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh deploy[/mw_shl_code]
部署时可能会遇到错误RateLimit报错，报错内容如下：

[mw_shl_code=bash,true]E, [2013-12-15T06:13:32.599434 #6477] [task:3] ERROR -- : OpenStack API Request Entity Too Large error. Check task debug log for details.
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:20:in `cloud_error'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:39:in `rescue in with_openstack'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:25:in `with_openstack'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/cloud.rb:512:in `block in set_vm_metadata'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/cloud.rb:511:in `set_vm_metadata'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/metadata_helper.rb:14:in `update_vm_metadata'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:225:in `block (2 levels) in compile_package'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:290:in `prepare_vm'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:224:in `block in compile_package'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock_helper.rb:89:in `block in with_compile_lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock.rb:58:in `lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock_helper.rb:88:in `with_compile_lock'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:217:in `compile_package'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:205:in `block (2 levels) in process_task'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/event_log.rb:58:in `track'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:203:in `block in process_task'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_formatter.rb:46:in `with_thread_name'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:199:in `process_task'
/var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:174:in `block (4 levels) in compile_packages'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:83:in `call'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:83:in `block (2 levels) in create_thread'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:67:in `loop'
/var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:67:in `block in create_thread'[/mw_shl_code]

解决办法：请查看《使用Bosh在OpenStack上部署CloudFoundry碰到的问题》，部署成功反馈：

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh deploy
Getting deployment properties from director...
Unable to get properties list from director, trying without it...
Compiling deployment manifest...
Cannot get current deployment information from director, possibly a new deployment
Please review all changes carefully
Deploying `bosh-openstack.yml' to `microbosh-openstack' (type 'yes' to continue): yes

Director task 89

Preparing deployment
  binding deployment (00:00:00)
  binding releases (00:00:00)

....

Updating job nats
  nats/0 (canary) (00:00:29)
Done                    1/1 00:00:29

Updating job redis
  redis/0 (canary) (00:00:24)
Done                    1/1 00:00:24

Updating job postgres
  postgres/0 (canary) (00:00:44)
Done                    1/1 00:00:44

Updating job powerdns
  powerdns/0 (canary) (00:00:25)
Done                    1/1 00:00:25

Updating job blobstore
  blobstore/0 (canary) (00:00:25)
Done                    1/1 00:00:25

Updating job director
  director/0 (canary) (00:00:58)
Done                    1/1 00:00:58

Updating job registry
  registry/0 (canary) (00:00:33)
Done                    1/1 00:00:33

Updating job health_monitor
  health_monitor/0 (canary) (00:00:28)
Done                    1/1 00:00:28

Task 3 done
Started         2013-12-17 02:41:15 UTC
Finished        2013-12-17 02:57:08 UTC
Duration        00:15:53

Deployed `bosh-openstack.yml' to `microbosh-openstack'[/mw_shl_code]
7、验证部署是否成功

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments/bosh-openstack# bosh vms
Deployment `bosh-openstack'

Director task 90

Task 90 done

+------------------+---------+---------------+--------------------------+
| Job/index        | State   | Resource Pool | IPs                      |
+------------------+---------+---------------+--------------------------+
| blobstore/0      | running | common        | 50.50.0.11               |
| director/0       | running | common        | 50.50.0.10, 10.68.19.132 |
| health_monitor/0 | running | common        | 50.50.0.13               |
| nats/0           | running | common        | 50.50.0.5                |
| postgres/0       | running | common        | 50.50.0.7                |
| powerdns/0       | running | common        | 50.50.0.9, 10.68.19.131  |
| redis/0          | running | common        | 50.50.0.6                |
| registry/0       | running | common        | 50.50.0.12               |
+------------------+---------+---------------+--------------------------+

VMs total: 8[/mw_shl_code]
如上，所有Job的状态都是running，则部署成功。
############################################################



CloudFoundry目前官方的版本为V2版，release最新发布为170，不过CloudFoundry在GitHub上的仓库代码贡献非常频繁，一般只要提出有BUG，很快就会有修改并提交，所以，最好的部署方式是直接下载源码，创建dev版的release进行部署，当然，也可以直接使用官方发布的release进行打包部署，本文中使用149.1-dev进行打包部署，如果要使用官方release，指定release路径即可，如：bosh create release release/cf-149.yml。
注意：本文中的示例Manifest仅适用于149+版本的release。

1、调整虚拟机存储配置

默认情况下blobstore/0虚拟机中/var/vcap/store的存储空间比较小，而后边上传用于部署CloudFoundry的Stemcell包和Release包时，会因为存储空间不足导致错误，所以这里先增加/var/vcap/store的挂载空间，防止发生错误。
具体步骤参看文章：《使用Bosh在OpenStack上部署CloudFoundry碰到的问题》。
说明：本环境配置文件中的所有通过Stemcell创建的虚拟机的登陆账户密码为“root/c1oudc0w”或者“vcap/c1oudc0w”
注意：即使配置挂载空间后，以后重启blobstore的虚拟机的时候一定记得再次挂载，或者可以修改为开机挂载，本人就是忘记该步骤，导致重启虚拟机后，后边部署cf-service-release时出现莫名其妙找不到package的错误，折磨几个小时。

2、编译Release包

依次执行以下命令从GitHub获取并更新代码

[mw_shl_code=bash,true]root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create release --force

....

Building jobs
-------------
Building health_manager_next...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 7.1-dev

Building dashboard...
  Final version:   Downloading `dashboard (10)'...

Building ccdb_postgres...
  Final version:   Downloading `ccdb_postgres (7)'...

Building gorouter...
  Final version:   Downloading `gorouter (2)'...

Building dea_next...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 12.1-dev

Building collector...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 6.1-dev

Building uaa...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 27.1-dev

Building syslog_aggregator...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 10.1-dev

Building login...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 11.1-dev

Building debian_nfs_server...
  Final version:   FOUND LOCAL

Building nats...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 10.1-dev

Building postgres...
  Final version:   Downloading `postgres (4)'...

Building cloud_controller_ng...
  Final version:   NOT FOUND
  Dev version:     NOT FOUND
  Generating...
  Generated version 7.1-dev

Building release
----------------

Generating manifest...
----------------------
Writing manifest...

Release summary
---------------
Packages
+---------------------+----------+-------------+
| Name                | Version  | Notes       |
+---------------------+----------+-------------+
| git                 | 1        |             |
| nginx               | 9        |             |
| ruby                | 8        |             |
| health_manager_next | 18       |             |
| dashboard           | 14       |             |
| libyaml             | 1        |             |
| rootfs_lucid64      | 1        |             |
| insight_agent       | 2        |             |
| ruby_next           | 3        |             |
| gorouter            | 4.1-dev  | new version |
| dea_next            | 16.1-dev | new version |
| collector           | 8.1-dev  | new version |
| uaa                 | 24.1-dev | new version |
| golang              | 2        |             |
| warden              | 26.1-dev | new version |
| mysqlclient         | 3        |             |
| syslog_aggregator   | 4        |             |
| libpq               | 5        |             |
| login               | 13.1-dev | new version |
| dea_jvm             | 4        |             |
| buildpack_cache     | 1.1-dev  | new version |
| debian_nfs_server   | 3        |             |
| nats                | 8        |             |
| postgres            | 4        |             |
| sqlite              | 3        |             |
| imagemagick         | 2        |             |
| cloud_controller_ng | 13.1-dev | new version |
| common              | 5        |             |
+---------------------+----------+-------------+

Jobs
+---------------------+----------+-------------+
| Name                | Version  | Notes       |
+---------------------+----------+-------------+
| health_manager_next | 7.1-dev  | new version |
| dashboard           | 10       |             |
| ccdb_postgres       | 7        |             |
| gorouter            | 2        |             |
| dea_next            | 12.1-dev | new version |
| collector           | 6.1-dev  | new version |
| uaa                 | 27.1-dev | new version |
| syslog_aggregator   | 10.1-dev | new version |
| login               | 11.1-dev | new version |
| debian_nfs_server   | 7        |             |
| nats                | 10.1-dev | new version |
| postgres            | 4        |             |
| cloud_controller_ng | 7.1-dev  | new version |
+---------------------+----------+-------------+

Jobs affected by changes in this release
+---------------------+----------+
| Name                | Version  |
+---------------------+----------+
| health_manager_next | 7.1-dev  |
| dea_next            | 12.1-dev |
| collector           | 6.1-dev  |
| uaa                 | 27.1-dev |
| syslog_aggregator   | 10.1-dev |
| login               | 11.1-dev |
| nats                | 10.1-dev |
| cloud_controller_ng | 7.1-dev  |
| gorouter            | 2        |
+---------------------+----------+

Release version: 149.1-dev
Release manifest: /root/src/cloudfoundry/cf-release/dev_releases/cf-release-149.1-dev.yml[/mw_shl_code]

3、连接director

[mw_shl_code=bash,true]root@bosh-cli:~/src/cloudfoundry/cf-release# bosh target https://10.68.19.132:25555
Target set to `bosh'
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login
Your username: admin
Enter password: *****
Logged in as `admin'
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create user
Enter new username: venusource
Enter new password: **********
Verify new password: **********
User `venusource' has been created
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login
Your username: venusource
Enter password: **********
Logged in as `venusource'[/mw_shl_code]

4、上传release和stemcell

上传Stemcell和部署Bosh的时候使用的是同一个Stemcell：

[mw_shl_code=bash,true]root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload stemcell ~/bosh-workspace/stemcells/<span style="font-family: Consolas, 'Courier New', Courier, mono, serif; line-height: 18px;">b</span>osh-stemcell-latest-openstack-kvm-ubuntu.tgz

Verifying stemcell...
File exists and readable                                     OK
Using cached manifest...
Stemcell properties                                          OK

Stemcell info
-------------
Name:    bosh-stemcell
Version: 776

Checking if stemcell already exists...
No

Uploading stemcell...

latest-bosh-s: 100% |oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| 251.3MB  11.0MB/s Time: 00:00:22

Director task 1

Update stemcell
  extracting stemcell archive (00:00:12)
  verifying stemcell manifest (00:00:00)
  checking if this stemcell already exists (00:00:00)
  uploading stemcell bosh-stemcell/776 to the cloud (00:02:05)
  save stemcell bosh-stemcell/776 (4e433167-acf2-41b0-b691-934684a63e52) (00:00:00)
Done                    5/5 00:02:17

Task 1 done
Started         2013-12-17 07:03:06 UTC
Finished        2013-12-17 07:05:23 UTC
Duration        00:02:17

Stemcell uploaded and created[/mw_shl_code]

上传Release到Bosh：

[mw_shl_code=bash,true]root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload release

....

Release has been created
  cf-release/149.1-dev (00:00:00)
Done                    1/1 00:00:00

Task 2 done
Started         2013-12-18 01:59:30 UTC
Finished        2013-12-18 02:02:20 UTC
Duration        00:02:50

Release uploaded[/mw_shl_code]
验证下Stemcell和Release是否正常：

[mw_shl_code=bash,true]root@bosh-cli:~# bosh releases

+------------+------------+-------------+
| Name       | Versions   | Commit Hash |
+------------+------------+-------------+
| cf-release | 149.1-dev* | 27784f08    |
+------------+------------+-------------+
(*) Currently deployed
(+) Uncommitted changes

Releases total: 3
root@bosh-cli:~# bosh stemcells

+---------------------------+---------+--------------------------------------+
| Name                      | Version | CID                                  |
+---------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu | 1478    | 316501e1-482d-472d-9fd9-b71a7b1223ee |
+---------------------------+---------+--------------------------------------+

Stemcells total: 1[/mw_shl_code]

5、创建Manifest文件

从GitHub中获取模板文件：


[mw_shl_code=bash,true]root@bosh-cli:~/src/cloudfoundry/cf-release# cd ~/bosh-workspace/deployments/
root@bosh-cli:~/bosh-workspace/deployments# vi cf-149.yml[/mw_shl_code]
下边给我本环境中配置后的Manifest.yml文件作为参考：
[mw_shl_code=bash,true]
---
name: cf
director_uuid: 785aa0a6-87c7-4452-a862-c118afc90b37

releases:
- name: cf-release
   version: latest

networks:
- name: floating
  type: vip
  cloud_properties: {}
- name: default
  type: dynamic
  cloud_properties:
    security_groups:
    - default

update:
  canaries: 1
  canary_watch_time: 30000-60000
  update_watch_time: 30000-60000
  max_in_flight: 4

compilation:
  workers: 6
  network: default
  reuse_compilation_vms: true
  cloud_properties:
    instance_type: v1.medium

resource_pools:
  - name: small
    network: default
    size: 4
    stemcell:
      name: bosh-openstack-kvm-ubuntu
      version: latest
    cloud_properties:
      instance_type: v1.small

  - name: medium
    network: default
    size: 0
    stemcell:
      name: bosh-openstack-kvm-ubuntu
      version: latest
    cloud_properties:
      instance_type: v1.medium

jobs:
  - name: data
    release: cf-release
    template:
      - postgres
      - debian_nfs_server
    instances: 1
    resource_pool: small
    persistent_disk: 4096
    networks:
    - name: default
      default:
      - dns
      - gateway
    properties:
      db: databases

  - name: core
    release: cf-release
    template:
      - syslog_aggregator
      - nats
      - health_manager_next
      - uaa
    instances: 1
    resource_pool: small
    networks:
    - name: default
      default:
      - dns
      - gateway

  - name: api
    release: cf-release
    template:
      - cloud_controller_ng
      - gorouter
    instances: 1
    resource_pool: small
    networks:
    - name: default
      default:
      - dns
      - gateway
    - name: floating
      static_ips:
      - 10.68.19.133
    properties:
      db: databases

  - name: dea
    release: cf-release
    template:
      - dea_next
    instances: 1
    resource_pool: small
    networks:
      - name: default
        default: [dns, gateway]

properties:
  cf:
    name: cf
    dns: vsc.com
    ip_addresses: ["10.68.19.133"]
    deployment_size: medium
    security_group: default
    persistent_disk: 4096
    common_password: c1oudc0w
    dea_server_ram: 4096

  domain: vsc.com
  system_domain: vsc.com.com
  system_domain_organization: system_domain
  app_domains:
    - vsc.com

  networks:
    apps: default
    management: default

  nats:
    address: 0.core.default.cf.bosh
    machines:
    - 0.core.default.cf.bosh
    port: 4222
    user: nats
    password: c1oudc0w
    authorization_timeout: 5

  router:
    address: 0.api.default.cf.bosh
    port: 8081
    status:
      port: 8080
      user: gorouter
      password: c1oudc0w

  dea: &dea
    memory_mb: 4096
    disk_mb: 10240
    directory_server_protocol: http

  dea_next: *dea

  syslog_aggregator:
    address: 0.core.default.cf.bosh
    port: 54321

  nfs_server:
    address: 0.data.default.cf.bosh
    network: "*.cf.bosh"
    idmapd_domain: vsc.com

  debian_nfs_server:
    no_root_squash: true

  databases: &databases
    db_scheme: postgres
    address: 0.data.default.cf.bosh
    port: 5524
    roles:
      - tag: admin
        name: ccadmin
        password: c1oudc0w
      - tag: admin
        name: uaaadmin
        password: c1oudc0w
    databases:
      - tag: cc
        name: ccdb
        citext: true
      - tag: uaa
        name: uaadb
        citext: true

  ccdb: &ccdb
    db_scheme: postgres
    address: 0.data.default.cf.bosh
    port: 5524
    roles:
      - tag: admin
        name: ccadmin
        password: c1oudc0w
    databases:
      - tag: cc
        name: ccdb
        citext: true

  ccdb_ng: *ccdb

  uaadb:
    db_scheme: postgresql
    address: 0.data.default.cf.bosh
    port: 5524
    roles:
      - tag: admin
        name: uaaadmin
        password: c1oudc0w
    databases:
      - tag: uaa
        name: uaadb
        citext: true

  cc_api_version: v2

  cc: &cc
    logging_level: debug
    external_host: api
    srv_api_uri: http://api.vsc.com
    cc_partition: default
    db_encryption_key: c1oudc0w
    bootstrap_admin_email: admin@vsc.com
    bulk_api_password: c1oudc0w
    uaa_resource_id: cloud_controller
    staging_upload_user: uploaduser
    staging_upload_password: c1oudc0w
    resource_pool:
      resource_directory_key: cc-resources
      # Local provider when using NFS
      fog_connection:
        provider: Local
    packages:
      app_package_directory_key: cc-packages
    droplets:
      droplet_directory_key: cc-droplets
    default_quota_definition: runaway

  ccng: *cc

  login:
    enabled: false

  uaa:
    url: http://uaa.vsc.com
    spring_profiles: postgresql
    no_ssl: true
    catalina_opts: -Xmx768m -XX:MaxPermSize=256m
    resource_id: account_manager
    jwt:
      signing_key: |
        -----BEGIN RSA PRIVATE KEY-----
        MIICXAIBAAKBgQDHFr+KICms+tuT1OXJwhCUmR2dKVy7psa8xzElSyzqx7oJyfJ1
        JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMXqHxf+ZH9BL1gk9Y6kCnbM5R6
        0gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBugspULZVNRxq7veq/fzwIDAQAB
        AoGBAJ8dRTQFhIllbHx4GLbpTQsWXJ6w4hZvskJKCLM/o8R4n+0W45pQ1xEiYKdA
        Z/DRcnjltylRImBD8XuLL8iYOQSZXNMb1h3g5/UGbUXLmCgQLOUUlnYt34QOQm+0
        KvUqfMSFBbKMsYBAoQmNdTHBaz3dZa8ON9hh/f5TT8u0OWNRAkEA5opzsIXv+52J
        duc1VGyX3SwlxiE2dStW8wZqGiuLH142n6MKnkLU4ctNLiclw6BZePXFZYIK+AkE
        xQ+k16je5QJBAN0TIKMPWIbbHVr5rkdUqOyezlFFWYOwnMmw/BKa1d3zp54VP/P8
        +5aQ2d4sMoKEOfdWH7UqMe3FszfYFvSu5KMCQFMYeFaaEEP7Jn8rGzfQ5HQd44ek
        lQJqmq6CE2BXbY/i34FuvPcKU70HEEygY6Y9d8J3o6zQ0K9SYNu+pcXt4lkCQA3h
        jJQQe5uEGJTExqed7jllQ0khFJzLMx0K6tj0NeeIzAaGCQz13oo2sCdeGRHO4aDh
        HH6Qlq/6UOV5wP8+GAcCQFgRCcB+hrje8hfEEefHcFpyKH+5g1Eu1k0mLrxK2zd+
        4SlotYRHgPCEubokb2S1zfZDWIXW3HmggnGgM949TlY=
        -----END RSA PRIVATE KEY-----
      verification_key: |
        -----BEGIN PUBLIC KEY-----
        MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHFr+KICms+tuT1OXJwhCUmR2d
        KVy7psa8xzElSyzqx7oJyfJ1JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMX
        qHxf+ZH9BL1gk9Y6kCnbM5R60gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBug
        spULZVNRxq7veq/fzwIDAQAB
        -----END PUBLIC KEY-----
    cc:
      client_secret: c1oudc0w
    admin:
      client_secret: c1oudc0w
    batch:
      username: batchuser
      password: c1oudc0w
    client:
      autoapprove:
        - cf
    clients:
      cf:
        override: true
        authorized-grant-types: password,implicit,refresh_token
        authorities: uaa.none
        scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write
        access-token-validity: 7200
        refresh-token-validity: 1209600
    scim:
      users:
      - admin|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin
      - services|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin[/mw_shl_code]
修改cf.yml文件
director_uuid：修改为Bosh Director的UUID，可以通过bosh status查看；
releases的name属性，修改为之前创建CloudFoundry的Release包时输入的包名；
10.58.19.133：修改为可用的Floating IP；
vsc.com：域名，CF平台不能通过IP直接访问使用，必须配置域名，可以自己任意写个域名，然后配置本地DNS Server进行解析即可；
反馈：


[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh deployment cf-149.yml[/mw_shl_code]

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh deploy
Getting deployment properties from director...
Unable to get properties list from director, trying without it...
Compiling deployment manifest...
Cannot get current deployment information from director, possibly a new deployment
Please review all changes carefully
Deploying `cf-149.yml' to `bosh' (type 'yes' to continue): yes

Director task 10

Preparing deployment
  binding deployment (00:00:00)
  binding releases (00:00:00)
  binding existing deployment (00:00:00)
  binding resource pools (00:00:00)
  binding stemcells (00:00:00)
  binding templates (00:00:01)
  binding properties (00:00:00)
  binding unallocated VMs (00:00:00)
  binding instance networks (00:00:00)
Done                    9/9 00:00:01

Preparing package compilation

Preparing DNS
  binding DNS (00:00:00)
Done                    1/1 00:00:00

Creating bound missing VMs
  small/0 (00:00:42)
  small/1 (00:00:57)
  small/2 (00:01:02)
  small/3 (00:01:10)
Done                    4/4 00:01:10

Binding instance VMs
  data/0 (00:00:01)
  dea/0 (00:00:01)
  core/0 (00:00:01)
  api/0 (00:00:01)
Done                    4/4 00:00:01

Preparing configuration
  binding configuration (00:00:02)
Done                    1/1 00:00:02

Updating job data
  data/0 (canary) (00:00:53)
Done                    1/1 00:00:53

Updating job core
  core/0 (canary) (00:01:08)
Done                    1/1 00:01:08

Updating job api
  api/0 (canary) (00:01:04)
Done                    1/1 00:01:04

Updating job dea
  dea/0 (canary) (00:03:20)
Done                    1/1 00:03:20

Task 10 done
Started         2013-12-18 11:20:53 UTC
Finished        2013-12-18 11:29:37 UTC
Duration        00:08:44

Deployed `cf-149.yml' to `bosh'[/mw_shl_code]
部署过程可能会因为OpenStack的VM虚拟机创建时间过长或PING虚拟机响应过长导致失败，只需要删除Deployments后再重新部署即可。
Deployment删除命令：

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh delete deployment cf[/mw_shl_code]
部署成功之后可以使用命令查看虚拟机运行状态：

[mw_shl_code=bash,true]root@bosh-cli:~/bosh-workspace/deployments# bosh vms
Deployment `cf'

Director task 191

Task 191 done

+-----------+---------+---------------+--------------------------+
| Job/index | State   | Resource Pool | IPs                      |
+-----------+---------+---------------+--------------------------+
| api/0     | running | small         | 50.50.0.16, 10.68.19.133 |
| core/0    | running | small         | 50.50.0.17               |
| data/0    | running | small         | 50.50.0.14               |
| dea/0     | running | small         | 50.50.0.15               |
+-----------+---------+---------------+--------------------------+

VMs total: 4[/mw_shl_code]
全部Job为Running状态则正常。

6、使用CF管理CloudFoundry

注意：请先配置DNS Server，对设置的虚拟域名（vsc.com）进行解析，参照文章：《配置DNS Server》和《配置OpenStack环境DNS》
6.1、安装CF命令行工具

[mw_shl_code=bash,true]gem install cf[/mw_shl_code]

6.2、切换CF指向为api.vsc.com

[mw_shl_code=bash,true]root@bosh-cli:~# cf target api.vsc.com
Setting target to http://api.vsc.com... OK

target: http://api.vsc.com
organization: system_domain[/mw_shl_code]

6.3、登陆到CF平台，本环境账号密码为admin/c1oudc0w

[mw_shl_code=bash,true]root@bosh-cli:~# cf login admin
target: http://api.vsc.com

Password> ********

Authenticating... OK
There are no spaces. You may want to create one with create-space.[/mw_shl_code]

6.4、创建组织
[mw_shl_code=bash,true]root@bosh-cli:~# cf create-org venusource
Creating organization venusource... OK
Switching to organization venusource... OK
There are no spaces. You may want to create one with create-space.[/mw_shl_code]

6.5、创建并切换空间

[mw_shl_code=bash,true]root@bosh-cli:~# cf create-space development
Creating space development... OK
Adding you as a manager... OK
Adding you as a developer... OK
Space created! Use `cf switch-space development` to target it.
root@bosh-cli:~# cf switch-space development
Switching to space development... OK

target: http://api.vsc.com
organization: venusource
space: development[/mw_shl_code]

6.6、下载测试Ruby工程

[mw_shl_code=bash,true]root@bosh-cli:~# git clone https://github.com/cloudfoundry- ... moapp_ruby_rack.git
[/mw_shl_code]


6.7、打包并发布测试工程

[mw_shl_code=bash,true]root@bosh-cli:~# cd cf_demoapp_ruby_rack/
root@bosh-cli:~/cf_demoapp_ruby_rack# rm manifest.yml
root@bosh-cli:~/cf_demoapp_ruby_rack# bundle
Resolving dependencies...
Using rack (1.5.2)
Using bundler (1.3.5)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
root@bosh-cli:~/cf_demoapp_ruby_rack# cf push
Name> hello

Instances> 1

1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 1

Creating hello... OK

1: hello
2: none
Subdomain> hello

1: vsc.com
2: none
Domain> mycloud.com

Creating route hello.mycloud.com... OK
Binding hello.mycloud.com to hello... OK

Create services for application?> n

Save configuration?> n

Uploading hello... OK
Starting hello... OK
-----> Downloaded app package (4.0K)

....

Checking hello...
Staging in progress...
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  1/1 instances: 1 running[/mw_shl_code]
OK
部署完毕！欢迎各位交流指正，谢谢！

版权属于：VF

原文地址：http://cloudnil.com/2014/05/15/deploy-cloudfoundry-5/

转载时必须以链接形式注明原始出处及本声明。