为什么NDP不能在虚拟以太网接口上为来自外部的数据包工作？

Question

我正在移动我的一些服务(SMTP、IMAP等)为了更好地隔离，我需要建立一些从外部重定向到容器的连接。在这个虚拟机上，我从提供者那里只得到了16个IPv6地址，所以我决定使用iptables将一些端口重定向到容器中。对于IPv4，它可以正常工作，但我不能让它在IPv6中工作。我已经解决了这个问题，因为我失去了虚拟以太网的对等MAC的邻居表条目，我不知道如何从那里开始。为了简化这个问题，我做了一个只有socat的测试容器，在端口5555上执行echo。我给它提供了一个主机和容器之间的虚拟以太网接口，它有一个专用的IPv4地址和一个ULA IPv6地址。

root@bonclok:~# machinectl status listenbox
listenbox(8c403969837546a7ad6a342da35fdf49)
           Since: Thu 2020-09-17 10:42:50 CEST; 4s ago
          Leader: 7319 ((sd-stubinit))
         Service: systemd-nspawn; class container
            Root: /chroot/listenbox
           Iface: ve-listenbox
         Address: 10.30.0.1
                  fdc9:c654:8207:17cb::1%91
                  fe80::ecaa:95ff:fe9a:1cff%91
            Unit: systemd-nspawn@listenbox.service
                  ├─payload
                  │ ├─7319 (sd-stubinit)
                  │ └─7324 socat -ly -d -d -d TCP6-LISTEN:5555,fork PIPE
                  └─supervisor
                    └─7317 /usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --network-veth -U --set

Sep 17 10:42:50 bonclok.hasiok.net systemd[1]: Started Container listenbox.
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I socat by Gerhard Rieger and contributors - see www.dest-unreach.org
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I This product includes software developed by the OpenSSL Project for use in t
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I This product includes software written by Tim Hudson (tjh@cryptsoft.com)
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I setting option "fork" to 1
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I socket(10, 1, 6) -> 6
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: I starting accept loop
Sep 17 10:42:50 bonclok.hasiok.net socat[7324]: N listening on AF=10 [0000:0000:0000:0000:0000:0000:0000:0000]:5555

下面是虚拟接口两端的配置

root@bonclok:~# ip a sh dev ve-listenbox # on host
91: ve-listenbox@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 9e:81:a0:6a:93:be brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet 10.30.0.2/24 brd 10.30.0.255 scope global ve-listenbox
       valid_lft forever preferred_lft forever
    inet6 fdc9:c654:8207:17cb::2/64 scope global
       valid_lft forever preferred_lft forever
root@bonclok:~# nsenter -a -t 7324 ip a sh dev host0 # in container
2: host0@if91: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ee:aa:95:9a:1c:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.30.0.1/24 scope global host0
       valid_lft forever preferred_lft forever
    inet6 fdc9:c654:8207:17cb::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::ecaa:95ff:fe9a:1cff/64 scope link

和航路表：

root@bonclok:~# ip -6 r|grep ve-listenbox # on host
fdc9:c654:8207:17cb::/64 dev ve-listenbox proto kernel metric 256 pref medium
root@bonclok:~# nsenter -a -t 7324 ip -6 r # in container
::1 dev lo proto kernel metric 256 pref medium
fdc9:c654:8207:17cb::/64 dev host0 proto kernel metric 256 pref medium
fe80::/64 dev host0 proto kernel metric 256 pref medium
default via fdc9:c654:8207:17cb::2 dev host0 metric 1024 pref medium

我已经通过iptable重定向了端口555

root@bonclok:~# ip6tables -n -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DNAT       tcp     !fdc9:c654:8207:17cb::1  ::/0                 tcp dpt:5555 to:[fdc9:c654:8207:17cb::1]:5555

现在，当我从主机连接到fdc9 9:c 654:8207:17cb：：1时，它就工作了。但是，如果我从外部连接，它只能在最近从主机连接并且有一个当前的邻居表条目时才能工作。我已经在接口上运行了tcpdump，并且当我从主机连接时：

root@bonclok:~# socat - TCP:[fdc9:c654:8207:17cb::1]:5555

我看到ICMPv6数据包来回移动和TCP握手：

root@bonclok:~# tcpdump -t -n -vvv -i ve-listenbox
tcpdump: listening on ve-listenbox, link-type EN10MB (Ethernet), capture size 262144 bytes
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::ecaa:95ff:fe9a:1cff > ff02::2: [icmp6 sum ok] ICMP6, router solicitation, length 16
          source link-address option (1), length 8 (1): ee:aa:95:9a:1c:ff
            0x0000:  eeaa 959a 1cff
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fdc9:c654:8207:17cb::2 > ff02::1:ff00:1: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fdc9:c654:8207:17cb::1
          source link-address option (1), length 8 (1): 9e:81:a0:6a:93:be
            0x0000:  9e81 a06a 93be
IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) fdc9:c654:8207:17cb::1 > fdc9:c654:8207:17cb::2: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fdc9:c654:8207:17cb::1, Flags [solicited, override]
          destination link-address option (2), length 8 (1): ee:aa:95:9a:1c:ff
            0x0000:  eeaa 959a 1cff
IP6 (flowlabel 0x2e8e4, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::2.38176 > fdc9:c654:8207:17cb::1.5555: Flags [S], cksum 0xbc13 (incorrect -> 0xe27d), seq 3531392912, win 64800, options [mss 1440,sackOK,TS val 2219762528 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0x78e11, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::1.5555 > fdc9:c654:8207:17cb::2.38176: Flags [S.], cksum 0xbc13 (incorrect -> 0x151d), seq 3651211421, ack 3531392913, win 64260, options [mss 1440,sackOK,TS val 1555204218 ecr 2219762528,nop,wscale 7], length 0
IP6 (flowlabel 0x2e8e4, hlim 64, next-header TCP (6) payload length: 32) fdc9:c654:8207:17cb::2.38176 > fdc9:c654:8207:17cb::1.5555: Flags [.], cksum 0xbc0b (incorrect -> 0x3cdf), seq 1, ack 1, win 507, options [nop,nop,TS val 2219762528 ecr 1555204218], length 0

我看到了邻居表的条目。

root@bonclok:~# ip ne sh dev ve-listenbox
fdc9:c654:8207:17cb::1 lladdr ee:aa:95:9a:1c:ff REACHABLE

然后，在关闭本地连接并从外部连接后：

piotras@red:~$ telnet -6 2a03:b0c0:3:f0::36:7000 5555
Trying 2a03:b0c0:3:f0::36:7000...
Connected to 2a03:b0c0:3:f0::36:7000.

我在eth0上看到原始数据包：

root@bonclok:~# tcpdump -n -t -vvv -i eth0 tcp port 5555
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP6 (flowlabel 0xcad87, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x7dcc (correct), seq 3406297189, win 64800, options [mss 1440,sackOK,TS val 2466613775 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xcc342, hlim 63, next-header TCP (6) payload length: 40) 2a03:b0c0:3:f0::36:7000.5555 > 2a02:2780:9000:50:fb:ff:fe02:2.43314: Flags [S.], cksum 0x2ded (incorrect -> 0x701a), seq 1345388437, ack 3406297190, win 64260, options [mss 1440,sackOK,TS val 290303657 ecr 2466613775,nop,wscale 7], length 0
IP6 (flowlabel 0xcad87, hlim 51, next-header TCP (6) payload length: 32) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > 2a03:b0c0:3:f0::36:7000.5555: Flags [.], cksum 0x97c6 (correct), seq 1, ack 1, win 507, options [nop,nop,TS val 2466613797 ecr 290303657], length 0

并在VE接口上翻译数据包。

IP6 (flowlabel 0xcad87, hlim 50, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > fdc9:c654:8207:17cb::1.5555: Flags [S], cksum 0x6bc7 (correct), seq 3406297189, win 64800, options [mss 1440,sackOK,TS val 2466613775 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xcc342, hlim 64, next-header TCP (6) payload length: 40) fdc9:c654:8207:17cb::1.5555 > 2a02:2780:9000:50:fb:ff:fe02:2.43314: Flags [S.], cksum 0x3ff2 (incorrect -> 0x5e15), seq 1345388437, ack 3406297190, win 64260, options [mss 1440,sackOK,TS val 290303657 ecr 2466613775,nop,wscale 7], length 0
IP6 (flowlabel 0xcad87, hlim 50, next-header TCP (6) payload length: 32) 2a02:2780:9000:50:fb:ff:fe02:2.43314 > fdc9:c654:8207:17cb::1.5555: Flags [.], cksum 0x85c1 (correct), seq 1, ack 1, win 507, options [nop,nop,TS val 2466613797 ecr 290303657], length 0

但是过了一段时间，邻居表条目就过期了。

root@bonclok:~# ip ne sh dev ve-listenbox
fdc9:c654:8207:17cb::1  INCOMPLETE

在上没有更多的流量了。当我试图重新连接时，我看到eth0上的SYN数据包被无法到达的ICMP目的地响应。

IP6 (flowlabel 0x09321, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x53f1 (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468133554 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xa154e, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x4feb (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468134584 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0x7c56a, hlim 51, next-header TCP (6) payload length: 40) 2a02:2780:9000:50:fb:ff:fe02:2.43532 > 2a03:b0c0:3:f0::36:7000.5555: Flags [S], cksum 0x480b (correct), seq 3900363577, win 64800, options [mss 1440,sackOK,TS val 2468136600 ecr 0,nop,wscale 7], length 0
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000
IP6 (flowlabel 0xd1899, hlim 64, next-header ICMPv6 (58) payload length: 88) 2a03:b0c0:3:f0::36:7000 > 2a02:2780:9000:50:fb:ff:fe02:2: [icmp6 sum ok] ICMP6, destination unreachable, unreachable address 2a03:b0c0:3:f0::36:7000

但是在虚拟以太网上，NDP查询没有发送到VE接口上.iptables规则的计数增加了，因此它肯定是NATing --到端口5555的新传入连接，但是由于没有邻居表条目，所以它不知道如何发送已翻译的数据包。

为什么当有来自外部的数据包时，它不尝试重新查询，但是当连接来自主机时，它却很高兴地重新查询？我还是不明白。

当我向邻居表添加静态条目时，它可以工作，但我不喜欢这样。

我做错了什么？

Answer 1

听起来你需要一个NDP代理。为此，我使用ndppd。它的配置非常简单，一旦您输入了外部接口和IPv6块的规则，就应该可以工作了。