openstack Security group

Security group(安全组)概念

安全组是通过Linux IPtables实现的,安全组对象是虚拟网卡,由L2 Agent(neutron_openvswitch_agent、neutron_linuxbridfe_agent)实现,在计算节点上通过iptables规则限制虚拟网卡流量进出。其容易和FW混淆,FW主要是异常流量隔离,一般负责跨子网流量不管虚拟网卡间通讯流量。

neutron port-list 列出属于租户的所有接口

1
2
3
4
5
6
7
8
9
10
11
12
root@controller:~# . demo-openrc
root@controller:~# neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+--------------------------------------------+
| 44a3abfa-1203-48d8-b83c-0c8657853db9 | | fa:16:3e:71:34:ed | {"subnet_id": "f746bd76-5bb3-4432-8fde- |
| | | | 66ccf8c59da9", "ip_address": "172.16.1.7"} |
| d879832c-1ec1-429e-8e4f-2e11506435a4 | | fa:16:3e:2e:09:4b | {"subnet_id": "f746bd76-5bb3-4432-8fde- |
| | | | 66ccf8c59da9", "ip_address": "172.16.1.1"} |
| f804cbf3-c4e8-4f66-ade5-87661591ecca | | fa:16:3e:f2:22:5d | {"subnet_id": "f746bd76-5bb3-4432-8fde- |
| | | | 66ccf8c59da9", "ip_address": "172.16.1.2"} |
+--------------------------------------+------+-------------------+--------------------------------------------+

openstack连接拓扑图(在此基础上研究安全组概念)

self_service

查看计算节点INPUT链

总结图:
compute_input

1
2
3
4
5
6
7
8
9
root@compute:~# iptables --line-numbers -vnL INPUT
Chain INPUT (policy ACCEPT 36472 packets, 8475K bytes)
num pkts bytes target prot opt in out source destination
1 33322 7781K nova-compute-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0
2 36472 8475K neutron-linuxbri-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53
4 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53
5 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67
6 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67

上述3、4、5、6策略放通dns和dhcp流量使用

根据策略1继续查看 nova-compute-INPUT

1
2
3
root@compute:~# iptables --line-numbers -vnL nova-compute-INPUT
Chain nova-compute-INPUT (1 references)
num pkts bytes target prot opt in out source destination

根据策略2继续查看 neutron-linuxbri-INPUT

1
2
3
Chain neutron-linuxbri-INPUT (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 neutron-linuxbri-o44a3abfa-1 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap44a3abfa-12 --physdev-is-bridged /* Direct incoming traffic from VM to the security group chain. */

linux桥和vm互联接口为tap44a3abfa-12,vm发出流量,查找INPUT链最终会查找neutron-linuxbri-o44a3abfa-1

根据策略1 继续查看neutron-linuxbri-o44a3abfa-1

1
2
3
4
5
6
7
8
9
10
11
root@compute:~# iptables --line-numbers -vnL  neutron-linuxbri-o44a3abfa-1
Chain neutron-linuxbri-o44a3abfa-1 (2 references)
num pkts bytes target prot opt in out source destination
1 2 648 RETURN udp -- * * 0.0.0.0 255.255.255.255 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
2 904 106K neutron-linuxbri-s44a3abfa-1 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 RETURN udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
4 0 0 DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:67 udp dpt:68 /* Prevent DHCP Spoofing by VM. */
5 865 103K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
6 39 2573 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
8 0 0 neutron-linuxbri-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */

DHCP广播地址、vm发包的ip和mac必须是vm分配的

neutron-linuxbri-s44a3abfa-1 检查流量源ip和mac是否为已分配(防止伪装地址攻击)

1
2
3
4
5
6
root@compute:~# iptables --line-numbers -vnL  neutron-linuxbri-s44a3abfa-1
Chain neutron-linuxbri-s44a3abfa-1 (1 references)
num pkts bytes target prot opt in out source destination
1 904 106K RETURN all -- * * 172.16.1.7 0.0.0.0/0 MAC FA:16:3E:71:34:ED /* Allow traffic from defined IP/MAC pairs. */
2 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* Drop traffic without an IP/MAC allow rule. */
root@compute:~#

该链表主要检查从vm发出来的数据包,是否是openstack所分配的IP和MAC,如果不匹配,禁止通过。此处是为了防止利用VM上进行一些伪装地址的攻击。

查看计算节点OUTPUT链

总结图:
compute_output

1
2
3
4
5
6
7
8
root@compute:~# iptables --line-numbers -vnL OUTPUT
Chain OUTPUT (policy ACCEPT 37285 packets, 8946K bytes)
num pkts bytes target prot opt in out source destination
1 140K 33M nova-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
2 34104 8216K nova-compute-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0
3 37285 8946K neutron-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
4 37285 8946K neutron-linuxbri-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT udp -- * virbr0 0.0.0.0/0 0.0.0.0/0 udp dpt:68

策略1从OUTPUT跳转到nova-filter-top跳转nova-compute-local

1
2
3
4
5
6
7
8
root@compute:~# iptables --line-numbers -vnL  nova-filter-top
Chain nova-filter-top (2 references)
num pkts bytes target prot opt in out source destination
1 39921 9619K nova-compute-local all -- * * 0.0.0.0/0 0.0.0.0/0
root@compute:~# iptables --line-numbers -vnL nova-compute-local
Chain nova-compute-local (1 references)
num pkts bytes target prot opt in out source destination
nova-compute-local 目前没有什么策略

策略2从OUTPUT跳转到nova-compute-OUTPUT

1
2
3
root@compute:~# iptables --line-numbers -vnL  nova-compute-OUTPUT
Chain nova-compute-OUTPUT (1 references)
num pkts bytes target prot opt in out source destination

nova-compute-OUTPUT没有什么策略

策略3从OUTPIUT跳转到neutron-filter-top跳转到neutron-linuxbri-local

1
2
3
4
5
6
7
8
root@compute:~# iptables --line-numbers -vnL  neutron-filter-top
Chain neutron-filter-top (2 references)
num pkts bytes target prot opt in out source destination
1 164K 38M neutron-linuxbri-local all -- * * 0.0.0.0/0 0.0.0.0/0

root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-local
Chain neutron-linuxbri-local (1 references)
num pkts bytes target prot opt in out source destination

neutron-linuxbri-local目前没有什么策略

策略4从OUTPUT跳转到neutron-linuxbri-OUTPUT

1
2
3
root@compute:~# iptables --line-numbers -vnL  neutron-linuxbri-OUTPUT
Chain neutron-linuxbri-OUTPUT (1 references)
num pkts bytes target prot opt in out source destination

neutron-linuxbri-OUTPUT目前没有什么策略

策略5

桥向68端口发udp包策略默认放开

查看计算节点FORWARD链

总结图:
compute_forward

1
2
3
4
5
6
7
8
9
10
11
12
root@compute:~# iptables --line-numbers -vnL FORWARD
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 neutron-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
2 0 0 neutron-linuxbri-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 nova-filter-top all -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 nova-compute-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
6 0 0 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
7 0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
8 0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
9 0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable

策略1从FORWARD跳转neutron-filter-top跳转到 neutron-linuxbri-local

1
2
3
4
5
6
7
8
9
root@compute:~# iptables --line-numbers -vnL neutron-filter-top
Chain neutron-filter-top (2 references)
num pkts bytes target prot opt in out source destination
1 165K 38M neutron-linuxbri-local all -- * * 0.0.0.0/0 0.0.0.0/0
root@compute:~#
root@compute:~#
root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-local
Chain neutron-linuxbri-local (1 references)
num pkts bytes target prot opt in out source destination

nova-compute-local目前没有什么策略

策略2从FORWARD跳转到neutron-linuxbri-FORWARD

1
2
3
4
5
6
root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-FORWARD
Chain neutron-linuxbri-FORWARD (1 references)
num pkts bytes target prot opt in out source destination
1 1212 112K neutron-linuxbri-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap44a3abfa-12 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
2 906 106K neutron-linuxbri-sg-chain all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap44a3abfa-12 --physdev-is-bridged /* Direct traffic from the VM interface to the security group chain. */
root@compute:~#

经过接口tap44a3abfa-12流量全部交给链表neutron-linuxbri-sg-chain处理

链表neutron-linuxbri-sg-chain策略

1
2
3
4
5
6
root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-sg-chain
Chain neutron-linuxbri-sg-chain (2 references)
num pkts bytes target prot opt in out source destination
1 1212 112K neutron-linuxbri-i44a3abfa-1 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap44a3abfa-12 --physdev-is-bridged /* Jump to the VM specific chain. */
2 906 106K neutron-linuxbri-o44a3abfa-1 all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap44a3abfa-12 --physdev-is-bridged /* Jump to the VM specific chain. */
3 2604 267K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0
  • 策略1:经过tap44a3abfa-12发给vm转发流量均由neutron-linuxbri-i44a3abfa-1 处理
  • 策略2:vm发出到直连桥流量均由neutron-linuxbri-o44a3abfa-1 处理
neutron-linuxbri-i44a3abfa-1策略
1
2
3
4
5
6
7
8
9
10
root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-i44a3abfa-1
Chain neutron-linuxbri-i44a3abfa-1 (1 references)
num pkts bytes target prot opt in out source destination
1 1194 110K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
2 2 729 RETURN udp -- * * 172.16.1.2 0.0.0.0/0 udp spt:67 udp dpt:68
3 8 480 RETURN tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
4 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv4d81fb7c3-1083-4203-8d6c- src
5 6 504 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0
6 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
7 2 648 neutron-linuxbri-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */

DHCP、ssh、icmp、以及已经建立好会话状态的后续连接流量可发给vm,其它流量一律丢弃

neutron-linuxbri-o44a3abfa-1策略
1
2
3
4
5
6
7
8
9
10
11
root@compute:~# iptables --line-numbers -vnL neutron-linuxbri-o44a3abfa-1
Chain neutron-linuxbri-o44a3abfa-1 (2 references)
num pkts bytes target prot opt in out source destination
1 2 648 RETURN udp -- * * 0.0.0.0 255.255.255.255 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
2 904 106K neutron-linuxbri-s44a3abfa-1 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 RETURN udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 /* Allow DHCP client traffic. */
4 0 0 DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:67 udp dpt:68 /* Prevent DHCP Spoofing by VM. */
5 865 103K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */
6 39 2573 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */
8 0 0 neutron-linuxbri-sg-fallback all -- * * 0.0.0.0/0 0.0.0.0/0 /* Send unmatched traffic to the fallback chain. */

DHCP返会流量、vm本机发出去流量均可以正常转发

策略3 从FORWARD跳转到neutron-filter-top跳转到nova-compute-local

1
2
3
4
5
6
7
8
9
root@compute:~# iptables --line-numbers -vnL nova-filter-top
Chain nova-filter-top (2 references)
num pkts bytes target prot opt in out source destination
1 4344 1066K nova-compute-local all -- * * 0.0.0.0/0 0.0.0.0/0
root@compute:~#
root@compute:~# iptables --line-numbers -vnL nova-compute-local
Chain nova-compute-local (1 references)
num pkts bytes target prot opt in out source destination
root@compute:~#

neutron-linuxbri-local 目前没有什么策略

策略4 从FORWARD跳转交给neutron-linuxbri-FORWARD处理

1
2
3
4
5
6
root@compute:~# iptables --line-numbers -vnL nova-compute-FORWARD
Chain nova-compute-FORWARD (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 ACCEPT all -- brqe47041c9-00 * 0.0.0.0/0 0.0.0.0/0
2 0 0 ACCEPT all -- * brqe47041c9-00 0.0.0.0/0 0.0.0.0/0
3 0 0 DROP all -- * brqe47041c9-00 0.0.0.0/0 0.0.0.0/0

寻找计算节点vm连接的接口的策略

1
2
3
4
5
6
7
8
9
10
11
12
13
14
root@compute:~# brctl show
bridge name bridge id STP enabled interfaces
brqe47041c9-00 8000.7a0a01214421 no tap44a3abfa-12
vxlan-68
virbr0 8000.000000000000 yes
root@compute:~#
root@compute:~#
root@compute:~# iptables -S|grep tap44a3abfa-12
-A neutron-linuxbri-FORWARD -m physdev --physdev-out tap44a3abfa-12 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-linuxbri-sg-chain
-A neutron-linuxbri-FORWARD -m physdev --physdev-in tap44a3abfa-12 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-linuxbri-sg-chain
-A neutron-linuxbri-INPUT -m physdev --physdev-in tap44a3abfa-12 --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-linuxbri-o44a3abfa-1
-A neutron-linuxbri-sg-chain -m physdev --physdev-out tap44a3abfa-12 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-linuxbri-i44a3abfa-1
-A neutron-linuxbri-sg-chain -m physdev --physdev-in tap44a3abfa-12 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-linuxbri-o44a3abfa-1
root@compute:~#

tap44a3abfa-12接口的FORWARD流量均扔给neutron-linuxbri-sg-chain处理,访问虚拟机流量扔给neutron-linuxbri-i44a3abfa-1,从虚拟机出来流量扔给neutron-linuxbri-o44a3abfa-1
从vm发出流量到 tap44a3abfa-12接口的INPUT扔给neutron-linuxbri-o44a3abfa-1链处理

待续