分享

nova-network问题,求助有经验的给点意见

qq243206114 发表于 2015-7-22 22:44:51 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 0 10566
回帖奖励 1 云币      回复本帖可获得 1 云币奖励! 每人限 1 次
openstack juno 单节点 all in one 环境,网络使用nova-network的FlatDHCPManager模式,两个网卡,em1作为管理网络与public_interface,em2作为flat_interface,使用fixed ip 一切正常,关联上floating ip之后ping不通网关以及外网,实在找不到问题,希望有经验的能给点思路

网络信息如下:
em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 78:2b:cb:64:c7:ab brd ff:ff:ff:ff:ff:ff
    inet 172.16.1.215/16 brd 172.16.255.255 scope global em1
       valid_lft forever preferred_lft forever
    inet 172.16.1.169/32 scope global em1
       valid_lft forever preferred_lft forever
    inet6 fe80::7a2b:cbff:fe64:c7ab/64 scope link
       valid_lft forever preferred_lft forever
em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br100 state UP qlen 1000
    link/ether 78:2b:cb:64:c7:ad brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7a2b:cbff:fe64:c7ad/64 scope link
       valid_lft forever preferred_lft forever
br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 78:2b:cb:64:c7:ad brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global br100
       valid_lft forever preferred_lft forever
    inet6 fe80::d829:4bff:fe43:505e/64 scope link
       valid_lft forever preferred_lft forever
vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UNKNOWN qlen 500
    link/ether fe:16:3e:68:62:0e brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fe68:620e/64 scope link
       valid_lft forever preferred_lft forever
vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UNKNOWN qlen 500
    link/ether fe:16:3e:f6:23:b1 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc16:3eff:fef6:23b1/64 scope link
       valid_lft forever preferred_lft forever


网桥信息如下:
[root@allinone ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
br100           8000.782bcb64c7ad       no                   em2
                                                                                    vnet0
                                                                                    vnet1


nova配置文件如下:
/etc/nova/nova.conf
network_api_class = nova.network.api.API
security_group_api = nova
firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver
network_manager = nova.network.manager.FlatDHCPManager
network_size = 254
allow_same_net_traffic = False
multi_host = True
send_arp_for_ha = True
share_dhcp_address = True
force_dhcp_release = True
flat_network_bridge = br100
flat_interface = em2
public_interface = em1

创建fixed ip
[root@allinone ~]# nova network-create demo-net --bridge br100 --multi-host T  --fixed-range-v4 10.0.0.0/24
[root@allinone ~]# nova net-list
+--------------------------------------+----------+-------------+
| ID                                   | Label    | CIDR        |
+--------------------------------------+----------+-------------+
| 97899710-1c95-41ef-a249-779e42ce5f88 | demo-net | 10.0.0.0/24 |
+--------------------------------------+----------+-------------+


创建并关联floating ip
[root@allinone ~]# nova-manage floating create --ip_range=172.16.1.160/30
[root@allinone ~]# nova floating-ip-create
[root@allinone ~]# nova floatinf-ip-associate demo 172.16.1.169

[root@allinone ~]# nova-manage floating list
b1b2741c02d140faba93b68c0f8f250d        172.16.1.169    de1c7879-58ee-4d91-899e-0febb61d4867    nova    em1
None    172.16.1.170    None    nova    em1


[root@allinone ~]# nova list
+--------------------------------------+------+--------+------------+-------------+---------------------------------+
| ID                                   | Name | Status | Task State | Power State | Networks                        |
+--------------------------------------+------+--------+------------+-------------+---------------------------------+
| de1c7879-58ee-4d91-899e-0febb61d4867 | demo | ACTIVE | -          | Running     | demo-net=10.0.0.2, 172.16.1.169 |
| cf4fb41b-c2b2-47b1-ad00-a09d20cae2b6 | test | ACTIVE | -          | Running     | demo-net=10.0.0.3               |
+--------------------------------------+------+--------+------------+-------------+---------------------------------+


首先使用fixed ip都没有问题
[root@allinone ~]# ssh root@10.0.0.3
root@10.0.0.3's password:
[root@test ~]# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=1.73 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.590 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.513 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.533 ms
^C
--- 10.0.0.2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3383ms
rtt min/avg/max/mdev = 0.513/0.843/1.737/0.517 ms
[root@test ~]# ping 172.16.1.1   
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_seq=1 ttl=127 time=0.641 ms
64 bytes from 172.16.1.1: icmp_seq=2 ttl=127 time=0.554 ms
64 bytes from 172.16.1.1: icmp_seq=3 ttl=127 time=0.523 ms
64 bytes from 172.16.1.1: icmp_seq=4 ttl=127 time=0.533 ms
^C
--- 172.16.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3444ms
rtt min/avg/max/mdev = 0.523/0.562/0.641/0.054 ms
[root@test ~]# ping www.baidu.com
PING www.a.shifen.com (180.97.33.107) 56(84) bytes of data.
64 bytes from 180.97.33.107: icmp_seq=1 ttl=54 time=9.00 ms
64 bytes from 180.97.33.107: icmp_seq=2 ttl=54 time=7.55 ms
64 bytes from 180.97.33.107: icmp_seq=3 ttl=54 time=7.55 ms
64 bytes from 180.97.33.107: icmp_seq=4 ttl=54 time=7.41 ms
^C
--- www.a.shifen.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3989ms
rtt min/avg/max/mdev = 7.413/7.883/9.009/0.661 ms


实例demo关联了floating ip 之后,就不能ping通网关和外网
[root@allinone ~]# ssh cirros@172.16.1.169
cirros@172.16.1.169's password:
$ ping www.baidu.com
PING www.baidu.com (180.97.33.107): 56 data bytes

无法响应ping,查看public interface,有ICMP requese,但是没有 ICMP reply
[root@allinone ~]# tcpdump -i em1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
05:31:18.208515 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 47, length 64
05:31:19.208663 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 48, length 64
05:31:20.208805 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 49, length 64
05:31:21.208945 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 50, length 64
05:31:22.209154 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 51, length 64
05:31:23.209276 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 52, length 64
05:31:24.209409 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 53, length 64
05:31:25.209517 IP allinone > 180.97.33.107: ICMP echo request, id 13313, seq 54, length 64


ping网关也是一样,没有ICMP reply
$ ping 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em1, link-type EN10MB (Ethernet), capture size 65535 bytes
05:37:20.488352 IP allinone > 172.16.1.1: ICMP echo request, id 13569, seq 99, length 64
05:37:21.488510 IP allinone > 172.16.1.1: ICMP echo request, id 13569, seq 100, length 64
05:37:22.488595 IP allinone > 172.16.1.1: ICMP echo request, id 13569, seq 101, length 64
05:37:23.488730 IP allinone > 172.16.1.1: ICMP echo request, id 13569, seq 102, length 64
05:37:24.488848 IP allinone > 172.16.1.1: ICMP echo request, id 13569, seq 103, length 64

ping宿主机跟同一网段都能通
$ ping 172.16.1.215
PING 172.16.1.215 (172.16.1.215): 56 data bytes
64 bytes from 172.16.1.215: seq=0 ttl=64 time=0.275 ms
64 bytes from 172.16.1.215: seq=1 ttl=64 time=0.270 ms
64 bytes from 172.16.1.215: seq=2 ttl=64 time=0.239 ms
64 bytes from 172.16.1.215: seq=3 ttl=64 time=0.180 ms
64 bytes from 172.16.1.215: seq=4 ttl=64 time=0.250 ms
64 bytes from 172.16.1.215: seq=5 ttl=64 time=0.232 ms
$ ping 172.16.1.3
PING 172.16.1.3 (172.16.1.3): 56 data bytes
64 bytes from 172.16.1.3: seq=0 ttl=63 time=0.625 ms
64 bytes from 172.16.1.3: seq=1 ttl=63 time=0.404 ms
64 bytes from 172.16.1.3: seq=2 ttl=63 time=0.395 ms
64 bytes from 172.16.1.3: seq=3 ttl=63 time=0.371 ms

附上iptables关键链的内容,我觉得没什么问题
*filter
-A nova-api-metadat-INPUT -d 172.16.1.215/32 -p tcp -m tcp --dport 8775 -j ACCEPT
-A nova-compute-FORWARD -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT
-A nova-compute-INPUT -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -j ACCEPT
-A nova-compute-inst-1 -m state --state INVALID -j DROP
-A nova-compute-inst-1 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-1 -j nova-compute-provider
-A nova-compute-inst-1 -s 10.0.0.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-1 -p tcp -m tcp --dport 22 -j ACCEPT
-A nova-compute-inst-1 -p icmp -j ACCEPT
-A nova-compute-inst-1 -j nova-compute-sg-fallback
-A nova-compute-inst-2 -m state --state INVALID -j DROP
-A nova-compute-inst-2 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-2 -j nova-compute-provider
-A nova-compute-inst-2 -s 10.0.0.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-2 -p tcp -m tcp --dport 22 -j ACCEPT
-A nova-compute-inst-2 -p icmp -j ACCEPT
-A nova-compute-inst-2 -j nova-compute-sg-fallback
-A nova-compute-local -d 10.0.0.2/32 -j nova-compute-inst-1
-A nova-compute-local -d 10.0.0.3/32 -j nova-compute-inst-2
-A nova-compute-sg-fallback -j DROP
-A nova-filter-top -j nova-network-local
-A nova-filter-top -j nova-compute-local
-A nova-filter-top -j nova-api-metadat-local
-A nova-network-FORWARD -i br100 -j ACCEPT
-A nova-network-FORWARD -o br100 -j ACCEPT
-A nova-network-INPUT -i br100 -p udp -m udp --dport 67 -j ACCEPT
-A nova-network-INPUT -i br100 -p tcp -m tcp --dport 67 -j ACCEPT
-A nova-network-INPUT -i br100 -p udp -m udp --dport 53 -j ACCEPT
-A nova-network-INPUT -i br100 -p tcp -m tcp --dport 53 -j ACCEPT

*nat
-A nova-compute-snat -j nova-compute-float-snat
-A nova-network-OUTPUT -d 172.16.1.169/32 -j DNAT --to-destination 10.0.0.2
-A nova-network-POSTROUTING -s 10.0.0.0/24 -d 172.16.1.215/32 -j ACCEPT
-A nova-network-POSTROUTING -s 10.0.0.0/24 -d 10.0.0.0/24 -m conntrack ! --ctstate DNAT -j ACCEPT
-A nova-network-POSTROUTING -s 10.0.0.2/32 -m conntrack --ctstate DNAT -j SNAT --to-source 172.16.1.169
-A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.16.1.215:8775
-A nova-network-PREROUTING -d 172.16.1.169/32 -j DNAT --to-destination 10.0.0.2
-A nova-network-float-snat -s 10.0.0.2/32 -d 10.0.0.2/32 -j SNAT --to-source 172.16.1.169
-A nova-network-float-snat -s 10.0.0.2/32 -o em1 -j SNAT --to-source 172.16.1.169
-A nova-network-snat -j nova-network-float-snat
-A nova-network-snat -s 10.0.0.0/24 -o em1 -j SNAT --to-source 172.16.1.215
-A nova-postrouting-bottom -j nova-network-snat
-A nova-postrouting-bottom -j nova-compute-snat
-A nova-postrouting-bottom -j nova-api-metadat-snat

没找到任何评论,期待你打破沉寂

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条