openvSwitch

openvSwitch简介

openvSwitch 是SDN常用的虚拟交换机,其将普通交换机的数据平面和控制平面相分离,SDN交换机只负责数据的转发,而控制指令则由更上一级的控制器下发

OpenvSwitch(虚拟交换机)常用模块

  • ovs-vswitchd

主要模块、实现交换功能,含有支持流交换的Linux内核模块,基于流交换
它和上层controller通信采用OPENFLOW协议,与ovsdb-server通信采用OVSDB协议,和内核通信采用netlink通信
支持多个独立datapatch(网桥)

1
2
root      2225     1  0 20:05 ?        00:00:00 ovs-vswitchd: monitoring pid 2226 (healthy)
root 2226 2225 0 20:05 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
  • ovsdb-server

虚拟交换机配置信息(接口、交换内容、VLAN等)存放的数据库服务,ovs-vswitchd会根据ovsdb-server中的配置工作。
ovsdb-server 和ovs-vswitchd 使用OVSDB(JSON-RPC)方式通信

1
2
root      2201     1  0 20:05 ?        00:00:00 ovsdb-server: monitoring pid 2202 (healthy)
root 2202 2201 0 20:05 ? 00:00:00 ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/var/run/openvswitch/db.sock --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir --log-file=/var/log/openvswitch/ovsdb-server.log --pidfile=/var/run/openvswitch/ovsdb-server.pid --detach --monitor

openvSwitch编译

编译环境:ubuntun16.04

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
cat build.sh

apt-get install build-essential libssl-dev linux-headers-$(uname -r)
apt-get install graphviz autoconf automake bzip2 debhelper dh-autoreconf libssl-dev libtool openssl procps python-all python-qt4 python-twisted-conch python-zopeinterface python-six dkms module-assistant ipsec-tools racoon libc6-dev module-init-tools netbase python-argparse uuid-runtime -y
git clone https://github.com/openvswitch/ovs.git
cd ovs
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make
make install
make modules_install
/sbin/modprobe openvswitch
ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \
--private-key=db:Open_vSwitch,SSL,private_key \
--certificate=db:Open_vSwitch,SSL,certificate \
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \
--pidfile --detach
ovs-vsctl --no-wait init
ovs-vswitchd --pidfile --detach

make modules_install 错误
ubbuntun16.04报错如下,该错误忽略即可

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
root@controller-VirtualBox:~/ovs# make modules_install
cd datapath/linux && make modules_install
make[1]: Entering directory '/root/ovs/datapath/linux'
make -C /lib/modules/4.8.0-36-generic/build M=/root/ovs/datapath/linux modules_install
make[2]: Entering directory '/usr/src/linux-headers-4.8.0-36-generic'
INSTALL /root/ovs/datapath/linux/openvswitch.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
INSTALL /root/ovs/datapath/linux/vport-geneve.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
INSTALL /root/ovs/datapath/linux/vport-gre.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
INSTALL /root/ovs/datapath/linux/vport-lisp.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
INSTALL /root/ovs/datapath/linux/vport-stt.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
INSTALL /root/ovs/datapath/linux/vport-vxlan.ko
At main.c:158:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
DEPMOD 4.8.0-36-generic
make[2]: Leaving directory '/usr/src/linux-headers-4.8.0-36-generic'
depmod `sed -n 's/#define UTS_RELEASE "\([^"]*\)"/\1/p' /lib/modules/4.8.0-36-generic/build/include/generated/utsrelease.h`
make[1]: Leaving directory '/root/ovs/datapath/linux'

建立ovs接口连接两个namespace组成二层网络

环境搭建拓扑

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

br0
+--------------------------------------+
+--+ +--+
+---+ | tap1 tap2| +---+
| +--+ +--+ |
| | | |
| +--------------------------------------+ |
| |
| |
| |
| |
+------------------+ +-------------------+
| tap1 | | tap2 |
|192.168.1.102/24 | | 192.168.1.102/24 |
| | | |
| | | |
| | | |
| namespace ns1 | | namespace ns1 |
| | | |
+------------------+ +-------------------+

实现脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
ip netns add ns1
ip netns add ns2

ovs-vsctl add-br br0
ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=internal

ip link set tap1 netns ns1
ip netns exec ns1 ip link set dev tap1 up

ovs-vsctl add-port br0 tap2 -- set Interface tap2 type=internal
ip link set tap2 netns ns2
ip netns exec ns2 ip link set dev tap2 up

ip netns exec ns1 ip addr add 192.168.1.102/24 dev tap1
ip netns exec ns2 ip addr add 192.168.1.101/24 dev tap2

ip netns exec ns1 ip link set lo up
ip netns exec ns2 ip link set lo up
ip netns exec ns1 ping -c 4 192.168.1.101
ip netns exec ns1 ping -c 4 192.168.1.102

建立vlan二层网络

环境搭建拓扑

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22


br0 trunk vlan tag 10,11 br1
+------------------------+ +------------------------+
| | tag10 tag10 | |
| trunk_br0 +-----------------------+trunk_br1 |
| +-----------------------+ |
| | tag11 tag11 | |
|tap1 tap2 | | tap3 |
+------------------------+ +------------------------+
|tag 10 tag11| tag10|
| | |
| | |
192.168.1.101/24 | | 192.168.1.102/24 | 192.168.1.103/24
+-------+ +-------+ +-------+
| tap1 | | tap2 | |tap3 |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
+-------+ +-------+ +-------+
ns1 ns2 ns3

环境实现脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
ip netns add ns1
ip netns add ns2

ovs-vsctl add-br br0
ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=internal

ovs-vsctl set Port tap1 tag=10

ip link set tap1 netns ns1
ip netns exec ns1 ip link set dev tap1 up

ovs-vsctl add-port br0 tap2 -- set Interface tap2 type=internal
ovs-vsctl set Port tap2 tag=11

ip link set tap2 netns ns2
ip netns exec ns2 ip link set dev tap2 up

ip netns exec ns1 ip addr add 192.168.1.101/24 dev tap1
ip netns exec ns2 ip addr add 192.168.1.102/24 dev tap2

ip netns exec ns1 ip link set lo up
ip netns exec ns2 ip link set lo up

ovs-vsctl add-br br1
ovs-vsctl add-port br1 tap3 -- set Interface tap3 type=internal

ovs-vsctl add-port br0 trunk_br0 trunks=10,11 -- set Interface trunk_br0 type=patch options:peer=trunk_br1
ovs-vsctl add-port br1 trunk_br1 trunks=10,11 -- set Interface trunk_br1 type=patch options:peer=trunk_br0

ip netns add ns3
ip link set tap3 netns ns3
ip netns exec ns3 ip addr add 192.168.1.103/24 dev tap3
ip netns exec ns3 ip link set dev tap3 up
ovs-vsctl set Port tap3 tag=10


ip netns exec ns3 ping -c 4 192.168.1.101
ip netns exec ns3 ping -c 4 192.168.1.102

说明:

  • br0和br1两个交换机之间连接使用的是patch口,在创建时候需要指明peer(对端口)选项

    1
    2
    ovs-vsctl add-port br0 trunk_br0 trunks=10,11  -- set Interface trunk_br0 type=patch options:peer=trunk_br1
    ovs-vsctl add-port br1 trunk_br1 trunks=10,11 -- set Interface trunk_br1 type=patch options:peer=trunk_br0
  • br0和br1两个交换机之间连接在trunk口附加上tag10和tag11

结论

ns3:tap3:vlan10 能ping通ns1:tap1:vlan10 因为ns3和ns1属于同一个vlan;同时无法ping通ns2

ovs vlan报文转发原理探究

环境搭建拓扑

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


first_ns second_ns third_ns
+-----------+ +-----------+ +-----------+
| | | | | |
| | | | | |
| | | | | |
| first_br | second_br | third_br|
+-----------+ +-----------+ +-----------+
10.0.0.4/24 10.0.0.5/24 | 10.0.0.6/24
| | |
| | |
|tag 10 | 无 tag | trunk 11,12
+------------------------------------------+
| first_br second_br third_br
| |
| br0 |
| |
+------------------------------------------+
| tag 10
|
|
|
|
|10.0.0.1/24
+------------+
| |
| |
| |
| |
+------------+
ns1

搭建网络脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 ovs-vsctl add-br br0

ovs-vsctl add-port br0 first_br -- set Interface first_br type=internal
ovs-vsctl set Port first_br tag=10

ip netns add first
ip link set first_br netns first
ip netns exec first ip addr add 10.0.0.4/24 dev first_br
ip netns exec first ip link set dev first_br up

ip netns add ns1
ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=internal
ovs-vsctl set Port tap1 tag=10
ip link set tap1 netns ns1
ip netns exec ns1 ip link set lo up
ip netns exec ns1 ip link set dev tap1 up
ip netns exec ns1 ip addr add 10.0.0.1/24 dev tap1


ovs-vsctl add-port br0 second_br -- set Interface second_br type=internal

ip netns add second
ip link set second_br netns second
ip netns exec second ip addr add 10.0.0.5/24 dev second_br
ip netns exec second ip link set dev second_br up

ovs-vsctl add-port br0 third_br trunks=11,12 -- set Interface third_br type=internal

ip netns add third
ip link set third_br netns third
ip netns exec third ip addr add 10.0.0.6/24 dev third_br
ip netns exec third ip link set dev third_br up

实验过程:
进入netns1,一直ping 10.0.0.4,在netns first、second、third分别抓包

实验记录

  • first抓取报文
    1
    2
    3
    4
    5
    root@controller-VirtualBox:~# ip netns exec first tcpdump  -n -e -i first_br arp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on first_br, link-type EN10MB (Ethernet), capture size 262144 bytes
    15:47:54.636790 9a:03:f1:61:48:9d > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.4 tell 10.0.0.1, length 28
    15:47:54.636808 4e:cc:d6:5a:53:f4 > 9a:03:f1:61:48:9d, ethertype ARP (0x0806), length 42: Reply 10.0.0.4 is-at 4e:cc:d6:5a:53:f4, length 28

抓到arp广播包

  • second抓取报文
    1
    2
    3
    4
    root@controller-VirtualBox:~# ip netns exec second tcpdump  -n -e -i second_br arp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on second_br, link-type EN10MB (Ethernet), capture size 262144 bytes
    15:49:40.345271 9a:03:f1:61:48:9d > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 10, p 0, ethertype ARP, Request who-has 10.0.0.4 tell 10.0.0.1, length 28

抓到arp广播包

  • third抓取报文
    1
    2
    3
    root@controller-VirtualBox:~# ip netns exec third tcpdump  -n -e -i third_br arp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on third_br, link-type EN10MB (Ethernet), capture size 262144 bytes

没有抓到arp广播包

结论

  • trunk port
    (1)这个port不配置tag,配置trunks,如果trunks为空,则所有的VLAN都trunk,也就意味着对于所有的VLAN的包,本身带什么VLAN ID,就是携带者什么VLAN ID,
    (2)如果没有设置VLAN,就属于VLAN 0,全部允许通过。
    (3)如果trunks不为空,则仅仅带着这些VLAN ID的包通过。
  • access port
    (1)这个port配置tag,从这个port进来的包会被打上这个tag,
    (2)从其他的trunk port中进来的本身就带有VLAN ID的包,如果VLAN ID等于tag,则会从这个port发出,
    (3)从其他的access port上来的包,如果tag相同,也会被forward到这个port。
    (4)从access port发出的包不带VLAN ID。
    (5)如果一个本身带VLAN ID的包到达access port,即便VLAN ID等于tag,也会被抛弃。

ovs bonding链路冗余

实验拓扑

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
  192.168.0.101/24
+---------+ +-------------------------------+
| | | |
| tap0+------------+tap0 br0 |
| | ++ |
| | | +-------------+ |
| | +---------|bond0|---------------+
+---------+ +-------------+
ns1 br0_tap0 | | br0_tap1
| |
| |
br1_tap0 | | br1_tap1
192.168.0.102/24 +--------------+
+---------+ +---------|bond1|---------------+
| +------------+ +--------------+ |
| | | |
| | | br1 |
| | | |
| | +-------------------------------+
+---------+
ns2

环境搭建脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
ovs-vsctl add-br br0
ovs-vsctl add-br br1
ip link add br0_tap0 type veth peer name br1_tap0
ip link add br0_tap1 type veth peer name br1_tap1
ip link set br0_tap0 up
ip link set br0_tap1 up
ip link set br1_tap0 up
ip link set br1_tap1 up

ovs-vsctl add-bond br0 bond0 br0_tap0 br0_tap1
ovs-vsctl add-bond br1 bond1 br1_tap0 br1_tap1


ip netns add ns1
ip netns add ns2
ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=internal
ip link set tap1 netns ns1
ip netns exec ns1 ip link set dev tap1 up
ip netns exec ns1 ip addr add 192.168.1.101/24 dev tap1


ovs-vsctl add-port br1 tap2 -- set Interface tap2 type=internal
ip link set tap2 netns ns2
ip netns exec ns2 ip link set dev tap2 up
ip netns exec ns2 ip addr add 192.168.1.102/24 dev tap2


ovs-vsctl set Port bond0 lacp=active
ovs-vsctl set Port bond1 lacp=active

环境搭建完成后查看bond

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
root@controller-VirtualBox:~# ovs-appctl bond/show
---- bond1 ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
lacp_status: negotiated
lacp_fallback_ab: false
active slave mac: 9e:d9:94:98:26:85(br1_tap0)

slave br1_tap0: enabled
active slave
may_enable: true

slave br1_tap1: enabled
may_enable: true

---- bond0 ----
bond_mode: active-backup
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
lacp_status: negotiated
lacp_fallback_ab: false
active slave mac: 6a:d8:f8:f3:2b:6d(br0_tap0)

slave br0_tap0: enabled
active slave
may_enable: true

slave br0_tap1: enabled
may_enable: true

查看结论:

默认建立的bond为主备模式(bond_mode: active-backup)
br1_tap0和br0_tap0 流量走这对口

抓包结果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
抓取 br0_tap0
root@controller-VirtualBox:~# tcpdump -n -e -i br0_tap0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0_tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:12:45.022854 6a:d8:f8:f3:2b:6d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:12:45.023409 9e:d9:94:98:26:85 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:13:15.024627 6a:d8:f8:f3:2b:6d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:13:15.025299 9e:d9:94:98:26:85 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:13:45.025411 6a:d8:f8:f3:2b:6d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:13:45.025724 9e:d9:94:98:26:85 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110

在ns1 ping ns2,流量只走在主机接口上(br1_tap0和br0_tap0)
root@controller-VirtualBox:~# tcpdump -n -e -i br0_tap0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0_tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:15:29.980585 d2:25:e0:f4:59:83 > 3a:df:e7:d3:68:84, ethertype IPv4 (0x0800), length 98: 192.168.1.101 > 192.168.1.102: ICMP echo request, id 6006, seq 4, length 64
17:15:29.980608 3a:df:e7:d3:68:84 > d2:25:e0:f4:59:83, ethertype IPv4 (0x0800), length 98: 192.168.1.102 > 192.168.1.101: ICMP echo reply, id 6006, seq 4, length 64
17:15:31.003922 d2:25:e0:f4:59:83 > 3a:df:e7:d3:68:84, ethertype IPv4 (0x0800), length 98: 192.168.1.101 > 192.168.1.102: ICMP echo request, id 6006, seq 5, length 64

抓取br1_tap0

1
2
3
4
5
6
7
root@controller-VirtualBox:~# tcpdump -n -e -i br1_tap0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br1_tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:14:15.027630 6a:d8:f8:f3:2b:6d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:14:15.028468 9e:d9:94:98:26:85 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:14:45.028801 6a:d8:f8:f3:2b:6d > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110
17:14:45.029541 9e:d9:94:98:26:85 > 01:80:c2:00:00:02, ethertype Slow Protocols (0x8809), length 124: LACPv1, length 110

另外补充

  • 查看lacpovs-appctl lacp/show

  • active-backup 主-备 无法提升吞吐

  • balance-slb, 根据包的 source MAC + vlan tag來均衡流量

  • banlnce-tcp, 根据包的 L2/L3/L4 header来均衡流量

    banlance-tcp必须让硬件交换机设置802.3ad,balance-slb则设不设均可,设了流量提高比较大。

  • ovs-vsctl set Port bond0 bond_mode=balance-slb

  • 观察流量命令 cat /proc/net/dev

  • LACP是链路汇聚控制协议(具体研究待定)

openvswitch概念补充

几个重要的概念

  • Bridge: Bridge 代表一个以太网交换机(Switch),一个主机中可以创建一个或者多个 Bridge 设备。
  • Port: 端口与物理交换机的端口概念类似,每个 Port 都隶属于一个 Bridge。
  • Interface: 连接到 Port 的网络接口设备。在通常情况下,Port 和 Interface 是一对一的关系, 只有在配置 Port 为 bond 模式后,Port 和 Interface 是一对多的关系。
  • Controller: OpenFlow 控制器。OVS 可以同时接受一个或者多个 OpenFlow 控制器的管理。
  • datapath: 在 OVS 中,datapath 负责执行数据交换,也就是把从接收端口收到的数据包在流表中进行匹配,并执行匹配到的动作。
  • Flow table: 每个 datapath 都和一个“flow table”关联,当 datapath 接收到数据之后, OVS 会在 flow table 中查找可以匹配的 flow,执行对应的操作, 例如转发数据到另外的端口。

运行原理

内核模块实现了多个“数据路径(DataPath)”(类似于网桥),每个都可以有多个“vports”(类似于桥内的端口)。每个数据路径也通过关联流表(flow table)来设置操作,而这些流表中的流都是用户空间在报文头和元数据的基础上映射的关键信息,一般的操作都是将数据包转发到另一个vport。当一个数据包到达一个vport,内核模块所做的处理是提取其流的关键信息并在流表中查找这些关键信息。当有一个匹配的流时它执行对应的操作。如果没有匹配,它会将数据包送到用户空间的处理队列中(作为处理的一部分,用户空间可能会设置一个流用于以后遇到相同类型的数据包可以在内核中执行操作)。细节如下图所示:

openvswitch

ovs管理组件

  • ovs-dpctl:一个工具,用来配置交换机内核模块,可以控制转发规则。
  • ovs-vsctl:主要是获取或者更改ovs-vswitchd的配置信息,此工具操作的时候会更新ovsdb-server中的数据库。
  • ovs-appctl:主要是向OVS守护进程发送命令的,一般用不上。 a utility that sends commands to running Open vSwitch daemons (ovs-vswitchd)
  • ovsdbmonitor:GUI工具来显示ovsdb-server中数据信息。(Ubuntu下是可以使用apt-get安装,可以远程获取OVS数据库和OpenFlow的流表)
  • ovs-controller:一个简单的OpenFlow控制器
  • ovs-ofctl:用来控制OVS作为OpenFlow交换机工作时候的流表内容。
  • ovs-pki:OpenFlow交换机创建和管理公钥框架;
  • ovs-tcpundump:tcpdump的补丁,解析OpenFlow的消息;
  • brocompat.ko : Linux bridge compatibility module
  • openvswitch.ko : Open vSwitch switching datapath

ovs流表实验

网络拓扑

ovs-flow

拓扑实现脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
ovs-vsctl add-br ovs-switch

ovs-vsctl add-port ovs-switch p0 -- set Interface p0 ofport_request=100
ovs-vsctl set Interface p0 type=internal

ip netns add ns0
ip link set p0 netns ns0
ip netns exec ns0 ip addr add 192.168.1.100/24 dev p0
ip netns exec ns0 ifconfig p0 promisc up

ovs-vsctl add-port ovs-switch p1 -- set Interface p1 ofport_request=101
ovs-vsctl set Interface p1 type=internal

ip netns add ns1
ip link set p1 netns ns1
ip netns exec ns1 ip addr add 192.168.1.101/24 dev p1
ip netns exec ns1 ifconfig p1 promisc up

ovs-vsctl add-port ovs-switch p2 -- set Interface p2 ofport_request=102
ovs-vsctl set Interface p2 type=internal

ip netns add ns2
ip link set p2 netns ns2
ip netns exec ns2 ip addr add 192.168.1.102/24 dev p2
ip netns exec ns2 ifconfig p2 promisc up

ovs-ofctl show ovs-switch 显示虚拟交换机流表的统计信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
root@controller-VirtualBox:~# ovs-ofctl show ovs-switch
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000aaa801224e40
n_tables:254, n_buffers:0
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
100(p0): addr:fc:7f:00:00:36:b8
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
101(p1): addr:56:a7:a1:27:a9:b8
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
102(p2): addr:d2:af:d5:d2:c9:52
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
LOCAL(ovs-switch): addr:aa:a8:01:22:4e:40
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0

ovs-dpctl show 查看内核接口状态

1
2
3
4
5
6
7
8
9
10
root@controller-VirtualBox:~# ovs-dpctl show
system@ovs-system:
lookups: hit:334 missed:22 lost:0
flows: 0
masks: hit:371 total:0 hit/pkt:1.04
port 0: ovs-system (internal)
port 1: ovs-switch (internal)
port 2: p0 (internal)
port 3: p1 (internal)
port 4: p2 (internal)

ovs-ofctl dump-flows ovs-switch查看ovs-switch的流表

接口p1发出去的icmp发包源地址变成10.10.10.10

ovs-ofctl del-flows ovs-switch “in_port=100”删除in_port=100的流表

接口p1发出去的icmp报文源地址均变成10.10.10.10

1
ovs-ofctl add-flow ovs-switch "priority=1 idle_timeout=0,in_port=100,actions=mod_nw_src:10.10.10.10,normal"

所有的icmp定向到p2(in_port=102)

1
ovs-ofctl add-flow ovs-switch idle_timeout=0,dl_type=0x0800,nw_proto=1,actions=output:102

在该组网下数据包转发过程测试

产生数据包源 p0(06:fc:2c:fd:84:f5) -> p1(56:a7:a1:27:a9:b8)

1
2
3
4
5
6
7
8
9
10
11
12
root@controller-VirtualBox:~# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8 -generate
Flow: in_port=100,vlan_tci=0x0000,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000

bridge("ovs-switch")
--------------------
0. priority 0
NORMAL
-> no learned MAC for destination, flooding

Final flow: unchanged
Megaflow: recirc_id=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000
Datapath actions: 1,3,4

数据包转发过程:
上述产生的数据包最终被广播到接口1,3,4
port 1: ovs-switch (internal)
port 2: p0 (internal)
port 3: p1 (internal)
port 4: p2 (internal)

配置接口p1为tag101后 产生数据包源 p0(06:fc:2c:fd:84:f5) -> p1(56:a7:a1:27:a9:b8)

1
2
3
4
5
6
7
8
9
10
11
12
13
root@controller-VirtualBox:~# ovs-vsctl set Port p1 tag=101

root@controller-VirtualBox:~# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8 -generate
Flow: in_port=100,vlan_tci=0x0000,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000

bridge("ovs-switch")
--------------------
0. priority 0
NORMAL
-> no learned MAC for destination, flooding

Final flow: unchanged
Megaflow: recirc_id=0,in_port=100,vlan_tci=0x0000/0x1fff,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000

数据包转发过程:
上述产生数据包最终被广播到接口1,4 因为接口2已经配置了tag101和产生数据包不在同一个广播域

修改从p0口发出的数据包强制加上vlantag 101, 产生数据包p0(06:fc:2c:fd:84:f5) -> p1(56:a7:a1:27:a9:b8)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
ovs-ofctl add-flow ovs-switch "priority=3,in_port=100,dl_vlan=0xffff,actions=mod_vlan_vid:101,normal"

root@controller-VirtualBox:~# ovs-appctl ofproto/trace ovs-switch in_port=100,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8 -generate
Flow: in_port=100,vlan_tci=0x0000,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000

bridge("ovs-switch")
--------------------
0. in_port=100,vlan_tci=0x0000, priority 3
mod_vlan_vid:101
NORMAL
-> learned that 06:fc:2c:fd:84:f5 is on port p0 in VLAN 101
-> forwarding to learned port

Final flow: in_port=100,dl_vlan=101,dl_vlan_pcp=0,vlan_tci1=0x0000,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000
Megaflow: recirc_id=0,in_port=100,vlan_tci=0x0000,dl_src=06:fc:2c:fd:84:f5,dl_dst=56:a7:a1:27:a9:b8,dl_type=0x0000
Datapath actions: 3

数据包转发过程
产生的流从p0->p1 ,数据包被完整转发到port 3 ,也就是说明该数据包被完整添加上了tag101

产生一条p1->p0流量,该流量携带tag101,发送给接口p0

1
2
3
4
5
6
7
8
9
10
11
12
root@controller-VirtualBox:~# ovs-appctl ofproto/trace ovs-switch in_port=101,dl_dst=06:fc:2c:fd:84:f5,dl_src=56:a7:a1:27:a9:b8 -generate
Flow: in_port=101,vlan_tci=0x0000,dl_src=56:a7:a1:27:a9:b8,dl_dst=06:fc:2c:fd:84:f5,dl_type=0x0000

bridge("ovs-switch")
--------------------
0. priority 0
NORMAL
-> forwarding to learned port

Final flow: unchanged
Megaflow: recirc_id=0,in_port=101,vlan_tci=0x0000/0x1fff,dl_src=56:a7:a1:27:a9:b8,dl_dst=06:fc:2c:fd:84:f5,dl_type=0x0000
Datapath actions: push_vlan(vid=101,pcp=0),2

ovs配合使用Controller之一 Floodlight

操作系统环境:ubuntun 16.04
ovs版本信息:(Open vSwitch) 2.5.2
10.30.10.145/24

依赖包安装

1
2
apt-get install build-essential ant maven python-dev
apt-get install build-essential openjdk-8-jdk ant maven python-dev

下载floodlight

1
2
3
4
5
6
7
8
git clone git://github.com/floodlight/floodlight.git
cd floodlight/
git submodule init
git submodule update
ant
java -jar target/floodlight.jar

# nohup java -jar target/floodlight.jar > floodlight.log 2>&1 &

在被管理的ovs上虚拟机上执行加入命令

ovs-vsctl set-controller ovs-switch tcp:10.30.10.145:6653
ovs-vsctl set Bridge ovs-switch fail-mode=secure

其它说明:
(1)10.30.10.145为floodlight controller的安装服务地址
(2)一堆教程说端口是6633,新版本发生了变化更改为了6653,具体变化版本号未研究
(3)采用的是secure,不停的连接Controller

当 OVS 交换机连接到 Floodlight 控制器后,理论上所有的流表规则应该交给控制器来建立。由于 OVS 交换机和控制器之间是通过网络通讯来传递数据的,所以网络连接失败会影响到 Flow 的建立,为了处理该问题提供两种模式:

  • standlone: 默认模式。如果 OVS 交换机超过三次无法正常连接到 OpenFlow 控制器,OVS 交换机自己会负责建立流表。在这种模式下,OVS 和常见的 L2 交换机相似。与此同时,OVS 也会继续尝试连接控制器,一旦网络连接恢复,OVS 会再次切换到使用控制器进行流表管理。
  • secure: 在 secure 模式下,如果 OVS 无法正常连接到 OpenFlow 控制器,OVS 会不停的尝试与控制器重新建立连接,而不会自己负责建立流表。

访问测试

http://10.30.10.145:8080/ui/pages/index.html
floodlight

备注说明

博文的篇幅太长有碍观瞻,深入研究内容请见后续博文openvSwitch XX。