openstack DVR环境组网
如上图:
- 2计算节点 + 1 网络节点 +1控制节点
- DVR配置计算节点无floating ip
- 主要分析计算节点的ovs 流表策略
- ubuntun 16.04 + openstack newton版本
计算节点compute的ovs流表
在此安装环境下计算节点共有3个ovs bridge(br-ex,br-int,br-tun)
- br-ex:负责计算节点连接外网使用,需在dvr环境下配置floating ip vm数据流量才走此桥出外网
- br-int:负责连接虚拟机所接入的linux bridge、router namespace
- br-tun: 隧道所有的虚拟交换机,通过此隧道连接网络节点(dhcp、router、snat)、计算节点(虚拟机、router);完成大二层的需求
在对流表分析前,我们先了解网络拓扑,并获取接口和接口索引对应关系;
查看compute br-int接口和接口索引映射关系
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37root@compute:~# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00005278ead35849
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
1(patch-tun): addr:8e:c4:54:26:0f:46
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
2(qvo06ec6270-33): addr:0a:7f:a0:b5:1f:b8
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
3(qr-6474fe2d-84): addr:00:00:00:00:0e:00
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
4(qvo2118dc24-92): addr:6e:1b:a5:df:d7:77
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
5(int-br-ex): addr:aa:83:f4:ae:aa:41
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
6(qr-d2e6fcc5-c7): addr:00:00:00:00:0e:00
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
LOCAL(br-int): addr:52:78:ea:d3:58:49
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
root@compute:~#查看compute br-tun接口和接口索引映射关系
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23root@compute:~# ovs-ofctl show br-tun
OFPT_FEATURES_REPLY (xid=0x2): dpid:00005222d0a63e4f
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
1(patch-int): addr:ae:2b:47:4b:15:6a
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
2(vxlan-0a1e0aa0): addr:06:2d:8c:b2:d2:64
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
3(vxlan-0a1e0a9a): addr:4e:7d:65:8d:09:09
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
LOCAL(br-tun): addr:52:22:d0:a6:3e:4f
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
root@compute:~#
接口对应关系已经标注在openvswitch DVR环境组网图上,见上图
分析 br-int流表
分析结果如图:
步骤解析如下:
icmp6 协议交给table24处理
1 | root@compute:~# ovs-ofctl dump-flows br-int |
2和4接口arp交给24表格处理
1 | cookie=0xbc602bf73a053ff8, duration=2944.215s, table=0, n_packets=15, n_bytes=630, idle_age=547, priority=10,arp,in_port=2 actions=resubmit(,24) |
5接口流入的数据包全部丢弃
1 | cookie=0xbc602bf73a053ff8, duration=2968.234s, table=0, n_packets=4699, n_bytes=1348102, idle_age=1, priority=2,in_port=5 actions=drop |
2和4口其它协议数据包统统交给25表格处理
1 | cookie=0xbc602bf73a053ff8, duration=2944.232s, table=0, n_packets=927, n_bytes=106878, idle_age=1, priority=9,in_port=2 actions=resubmit(,25) |
其它数据包像正常交换机数据一样转发数据
1 | cookie=0xbc602bf73a053ff8, duration=2968.576s, table=0, n_packets=2332, n_bytes=240272, idle_age=1, priority=0 actions=NORMAL |
23表格默认全部丢弃数据包
1 | cookie=0xbc602bf73a053ff8, duration=2968.580s, table=23, n_packets=0, n_bytes=0, idle_age=2968, priority=0 actions=drop |
表24 处理ipv6 nd
1 | cookie=0xbc602bf73a053ff8, duration=2944.226s, table=24, n_packets=0, n_bytes=0, idle_age=2944, priority=2,icmp6,in_port=2,icmp_type=136,nd_target=fe80::f816:3eff:fef3:31f5 actions=NORMAL |
表24 接口2或4 收到 172.16.1.3 1.1.1.11 的arp信息交给表25处理
1 | cookie=0xbc602bf73a053ff8, duration=2944.219s, table=24, n_packets=15, n_bytes=630, idle_age=547, priority=2,arp,in_port=2,arp_spa=172.16.1.3 actions=resubmit(,25) |
表24 如果上述未处理完成,将剩余数据包丢去
1 | cookie=0xbc602bf73a053ff8, duration=2968.573s, table=24, n_packets=0, n_bytes=0, idle_age=2968, priority=0 actions=drop |
表25 接口2或则4 源mac是fa:16:3e:f3:31:f5 fa:16:3e:3c:d2:56 的来的数据包正常处理
1 | cookie=0xbc602bf73a053ff8, duration=2944.242s, table=25, n_packets=926, n_bytes=106092, idle_age=1, priority=2,in_port=2,dl_src=fa:16:3e:f3:31:f5 actions=NORMAL |
分析 br-tun流表
分析结果如图:
步骤解析如下:
内网接口 交给table2处理
1 | root@compute:~# ovs-ofctl dump-flows br-tun |
外网接口交给table4处理
1 | cookie=0xb9ef2c735af9d49c, duration=2963.688s, table=0, n_packets=154, n_bytes=12924, idle_age=208, priority=1,in_port=2 actions=resubmit(,4) |
其它:1
cookie=0xb9ef2c735af9d49c, duration=2988.034s, table=0, n_packets=0, n_bytes=0, idle_age=2988, priority=0 actions=drop
单播交给表20处理
1 | cookie=0xb9ef2c735af9d49c, duration=2988.027s, table=2, n_packets=209, n_bytes=20555, idle_age=208, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) |
多播交给表22处理
1 | cookie=0xb9ef2c735af9d49c, duration=2988.009s, table=2, n_packets=90, n_bytes=10366, idle_age=210, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22) |
其它:1
cookie=0xb9ef2c735af9d49c, duration=2988.006s, table=3, n_packets=0, n_bytes=0, idle_age=2988, priority=0 actions=drop
从外网过来的数据包,去除tunid添加对应的vlanid后将数据包交给table10处理
1 | cookie=0xb9ef2c735af9d49c, duration=2975.899s, table=4, n_packets=206, n_bytes=19127, idle_age=208, priority=1,tun_id=0x44 actions=mod_vlan_vid:1,resubmit(,10) |
其它:1
2cookie=0xb9ef2c735af9d49c, duration=2988.005s, table=4, n_packets=0, n_bytes=0, idle_age=2988, priority=0 actions=drop
cookie=0xb9ef2c735af9d49c, duration=2987.998s, table=6, n_packets=0, n_bytes=0, idle_age=2987, priority=0 actions=drop
学习mac等相关信息,将信息放到表20,将数据包从pach-int发出交给内网处理
1 | cookie=0xb9ef2c735af9d49c, duration=2987.994s, table=10, n_packets=354, n_bytes=32608, idle_age=208, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb9ef2c735af9d49c,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:OXM_OF_IN_PORT[]),output:1 |
发往目的mac为fa:16:3e:3e:69:70 tag为1 (ip为172.16.1.9虚拟机数据包)被去除vlan 加上tunid,交给compute1 连接的隧道接口
1 | cookie=0xb9ef2c735af9d49c, duration=2949.737s, table=20, n_packets=66, n_bytes=6977, idle_age=2871, priority=2,dl_vlan=1,dl_dst=fa:16:3e:3e:69:70 actions=strip_vlan,load:0x44->NXM_NX_TUN_ID[],output:3 |
发给172.16.1.2 dhcp 服务namespace数据包去除vlan后通过隧道交给网络节点
1 | cookie=0xb9ef2c735af9d49c, duration=2937.625s, table=20, n_packets=1, n_bytes=42, idle_age=2886, priority=2,dl_vlan=1,dl_dst=fa:16:3e:cb:c0:9e actions=strip_vlan,load:0x44->NXM_NX_TUN_ID[],output:2 |
发给1.1.1.2 dhcp 服务namespace数据包去除vlan后通过隧道交给网络节点
1 | cookie=0xb9ef2c735af9d49c, duration=2937.136s, table=20, n_packets=1, n_bytes=42, idle_age=2879, priority=2,dl_vlan=2,dl_dst=fa:16:3e:b0:bd:78 actions=strip_vlan,load:0x5b->NXM_NX_TUN_ID[],output:2 |
发给1.1.1.12的snat中网关数据包去除vlan后通过隧道交给网络节点
1 | cookie=0xb9ef2c735af9d49c, duration=2926.229s, table=20, n_packets=0, n_bytes=0, idle_age=2958, priority=2,dl_vlan=2,dl_dst=fa:16:3e:6c:ab:eb actions=strip_vlan,load:0x5b->NXM_NX_TUN_ID[],output:2 |
发往172.16.1.4的snat中网关数据包去除vlan后通过隧道交给网络节点
1 | cookie=0xb9ef2c735af9d49c, duration=2898.976s, table=20, n_packets=1, n_bytes=42, idle_age=2889, priority=2,dl_vlan=1,dl_dst=fa:16:3e:6b:31:95 actions=strip_vlan,load:0x44->NXM_NX_TUN_ID[],output:2 |
学习到的表项
1
2cookie=0xb9ef2c735af9d49c, duration=210.599s, table=20, n_packets=2, n_bytes=196, hard_timeout=300, idle_age=209, hard_age=208, priority=1,vlan_tci=0x0002/0x0fff,dl_dst=fa:16:3e:1f:95:2c actions=load:0->NXM_OF_VLAN_TCI[],load:0x5b->NXM_NX_TUN_ID[],output:2
cookie=0xb9ef2c735af9d49c, duration=210.597s, table=20, n_packets=2, n_bytes=264, hard_timeout=300, idle_age=208, hard_age=208, priority=1,vlan_tci=0x0001/0x0fff,dl_dst=fa:16:3e:2c:c3:0d actions=load:0->NXM_OF_VLAN_TCI[],load:0x44->NXM_NX_TUN_ID[],output:2未命中表20其它策略,统统交给流表22处理
1
cookie=0xb9ef2c735af9d49c, duration=2987.986s, table=20, n_packets=5, n_bytes=418, idle_age=210, priority=0 actions=resubmit(,22)
表22去除vlan 附加上tunid将数据包从隧道发出
1 | cookie=0xb9ef2c735af9d49c, duration=2926.237s, table=22, n_packets=18, n_bytes=2034, idle_age=210, priority=1,dl_vlan=2 actions=strip_vlan,load:0x5b->NXM_NX_TUN_ID[],output:2,output:3 |
表22其它:1
cookie=0xb9ef2c735af9d49c, duration=2987.979s, table=22, n_packets=54, n_bytes=6376, idle_age=2959, priority=0 actions=drop
分析br-ex流表
1 | root@compute:~# ovs-ofctl dump-flows br-ex |
如上流表:默认2三层转发策略,且丢弃从外网收到的数据包