mjhjmtu

I want http traffic to be forwarded to 192.168.99.100:32150.

I use:

sudo iptables -A OUTPUT -t nat -o lo -p tcp --dport 80 -j DNAT --to-destination 192.168.99.100:32150

I expect that telnet 127.0.0.1 80 connects to 192.168.99.100:32150.
For some reason it just hangs forever (TRACE logs below). I know that 192.168.99.100:32150 is listening because telnet 192.168.99.100 32150 does connect successfully.

Surprisingly, it does work as expected when the DNAT destination endpoint is 192.168.1.3:32150 (wlp3s0f0 interface) instead of 192.168.99.100:32150 (vboxnet0 interface). It seems that for some reason DNAT does not work if the destination interface is vboxnet0 but it does work for wlp3s0f0.

What could possibly cause such behaviour? Is my rule incorrect?

uname -a:

Linux xxx-X550LC 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

ifconfig:

conjureup0 Link encap:Ethernet HWaddr de:6c:af:10:43:16 
 inet addr:10.99.0.1 Bcast:0.0.0.0 Mask:255.255.255.0
 inet6 addr: fe80::dc6c:afff:fe10:4316/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:373 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000 
 RX bytes:0 (0.0 B) TX bytes:67696 (67.6 KB)

docker0 Link encap:Ethernet HWaddr 02:42:4f:04:7a:6c 
 inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
 inet6 addr: fe80::42:4fff:fe04:7a6c/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:198 errors:0 dropped:0 overruns:0 frame:0
 TX packets:570 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0 
 RX bytes:52783 (52.7 KB) TX bytes:90304 (90.3 KB)

enp2s0f1 Link encap:Ethernet HWaddr bc:ee:7b:19:f6:ff 
 UP BROADCAST MULTICAST MTU:1500 Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000 
 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

lo Link encap:Local Loopback 
 inet addr:127.0.0.1 Mask:255.0.0.0
 inet6 addr: ::1/128 Scope:Host
 UP LOOPBACK RUNNING MTU:65536 Metric:1
 RX packets:16471 errors:0 dropped:0 overruns:0 frame:0
 TX packets:16471 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1 
 RX bytes:103617080 (103.6 MB) TX bytes:103617080 (103.6 MB)

vboxnet0 Link encap:Ethernet HWaddr 0a:00:27:00:00:00 
 inet addr:192.168.99.1 Bcast:192.168.99.255 Mask:255.255.255.0
 inet6 addr: fe80::800:27ff:fe00:0/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
 TX packets:270 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000 
 RX bytes:0 (0.0 B) TX bytes:50604 (50.6 KB)

veth86607c2 Link encap:Ethernet HWaddr 76:05:f3:47:87:0f 
 inet6 addr: fe80::7405:f3ff:fe47:870f/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:198 errors:0 dropped:0 overruns:0 frame:0
 TX packets:624 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0 
 RX bytes:55555 (55.5 KB) TX bytes:96370 (96.3 KB)

wlp3s0f0 Link encap:Ethernet HWaddr 48:5a:b6:03:28:7f 
 inet addr:192.168.1.3 Bcast:192.168.1.255 Mask:255.255.255.0
 inet6 addr: fe80::e6a5:9e76:34f8:fe45/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:657207 errors:0 dropped:0 overruns:0 frame:0
 TX packets:370496 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000 
 RX bytes:883579787 (883.5 MB) TX bytes:38367961 (38.3 MB)

sudo iptables -L -t nat -n:

Chain PREROUTING (policy ACCEPT)
target prot opt source destination 
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target prot opt source destination 

Chain OUTPUT (policy ACCEPT)
target prot opt source destination 
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 to:192.168.99.100:32150

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination 
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 
MASQUERADE tcp -- 172.17.0.2 172.17.0.2 tcp dpt:5000
MASQUERADE all -- 10.99.0.0/24 !10.99.0.0/24 

Chain DOCKER (2 references)
target prot opt source destination 
RETURN all -- 0.0.0.0/0 0.0.0.0/0 
DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:5000 to:172.17.0.2:5000

tailf /var/log/kern.log | grep TRACE: (hangs on the DNAT rule and repeats):

Dec 22 20:14:41 xxx-X550LC kernel: [ 8848.624645] TRACE: raw:OUTPUT:policy:2 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47705 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209B710000000001030307) UID=1000 GID=1000 
Dec 22 20:14:41 xxx-X550LC kernel: [ 8848.624659] TRACE: mangle:OUTPUT:policy:1 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47705 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209B710000000001030307) UID=1000 GID=1000 
Dec 22 20:14:41 xxx-X550LC kernel: [ 8848.624667] TRACE: nat:OUTPUT:rule:2 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47705 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209B710000000001030307) UID=1000 GID=1000 
Dec 22 20:14:42 xxx-X550LC kernel: [ 8849.622139] TRACE: raw:OUTPUT:policy:2 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47706 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209C6B0000000001030307) UID=1000 GID=1000 
Dec 22 20:14:42 xxx-X550LC kernel: [ 8849.622169] TRACE: mangle:OUTPUT:policy:1 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47706 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209C6B0000000001030307) UID=1000 GID=1000 
Dec 22 20:14:42 xxx-X550LC kernel: [ 8849.622180] TRACE: nat:OUTPUT:rule:2 IN= OUT=lo SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=47706 DF PROTO=TCP SPT=51306 DPT=80 SEQ=1122574632 ACK=0 WINDOW=43690 RES=0x00 SYN URGP=0 OPT (0204FFD70402080A00209C6B0000000001030307) UID=1000 GID=1000 

The missing pieces to make it work were:

sudo sysctl -w net.ipv4.conf.all.route_localnet=1
sudo iptables -t nat -A POSTROUTING -j MASQUERADE

However, I do not understand why it did work when the destination was 192.168.1.3:32150.