mjhjmtu

When using NFSv4, I expected that all traffic would be on port 2049. This seemed to be the behavior in RHEL6. However, in RHEL7, when a client makes a connection to a server on port 2049, the server will then make a connection back to the client with a source port in the min_resvport <= x <= max_resvport range to a destination port on the client in the ephemeral port range. Using tcpdump, this second connection does not seem to pass any data, but once per minute a keep alive is sent (I think). After a period of 2-5 minutes, that second connection is closed by the server.

For fun, I used iptables on the client side to block that second connection. After a minute or so of SYN retries, the server gave up, but then it appeared the original 2049 connection took over doing the periodic keep alive. Also, during this time, the NFS mount seemed to work just fine regardless of the success of that second connection.

The mount command used was:

mount -t nfs4 -o noresvport,nodev,nosuid,noexec host:dir mountpoint

So what is this second connection? And is there a way via configuration files to prevent it from making this second connection?

tcpdump of the above mount command. Server is 10.3.3.11 and client is 10.3.0.99. You can see the second connection using a source port of 940.

14:31:51.366467 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [S], seq 3520698884, win 29200, options [mss 1460,sackOK,TS val 2836344526 ecr 0,nop,wscale 7], length 0
14:31:51.366860 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [S.], seq 621440608, ack 3520698885, win 28960, options [mss 1460,sackOK,TS val 2832333931 ecr 2836344526,nop,wscale 7], length 0
14:31:51.366901 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 1, win 229, options [nop,nop,TS val 2836344526 ecr 2832333931], length 0
14:31:51.367323 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1:45, ack 1, win 229, options [nop,nop,TS val 2836344526 ecr 2832333931], length 44: NFS request xid 3288096941 null
14:31:51.367777 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 45, win 227, options [nop,nop,TS val 2832333932 ecr 2836344526], length 0
14:31:51.367797 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1:29, ack 45, win 227, options [nop,nop,TS val 2832333932 ecr 2836344526], length 28: NFS reply xid 3288096941 reply ok 24 null
14:31:51.367808 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 29, win 229, options [nop,nop,TS val 2836344527 ecr 2832333932], length 0
14:31:51.368378 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 45:225, ack 29, win 229, options [nop,nop,TS val 2836344527 ecr 2832333932], length 180: NFS request xid 3304874157 getattr fh 0,0/35
14:31:51.369007 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 29:93, ack 225, win 235, options [nop,nop,TS val 2832333933 ecr 2836344527], length 64: NFS reply xid 3304874157 reply ok 60 getattr NON 1 ids 0/338650970 sz 1879048192
14:31:51.369228 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 225:329, ack 93, win 229, options [nop,nop,TS val 2836344528 ecr 2832333933], length 104: NFS request xid 3321651373 getattr fh 0,0/36
14:31:51.369704 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 93:141, ack 329, win 235, options [nop,nop,TS val 2832333934 ecr 2836344528], length 48: NFS reply xid 3321651373 reply ok 44 getattr [|nfs]
14:31:51.369731 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [S], seq 1398204113, win 29200, options [mss 1460,sackOK,TS val 2832333934 ecr 0,nop,wscale 7], length 0
14:31:51.369759 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [S.], seq 2188378938, ack 1398204114, win 28960, options [mss 1460,sackOK,TS val 2836344529 ecr 2832333934,nop,wscale 7], length 0
14:31:51.369992 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 1, win 229, options [nop,nop,TS val 2832333934 ecr 2836344529], length 0
14:31:51.370034 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [P.], seq 1:73, ack 1, win 229, options [nop,nop,TS val 2832333934 ecr 2836344529], length 72
14:31:51.370049 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836344529 ecr 2832333934], length 0
14:31:51.370115 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [P.], seq 1:29, ack 73, win 227, options [nop,nop,TS val 2836344529 ecr 2832333934], length 28
14:31:51.370325 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832333935 ecr 2836344529], length 0
14:31:51.370640 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 329:441, ack 141, win 229, options [nop,nop,TS val 2836344530 ecr 2832333934], length 112: NFS request xid 3338428589 getattr fh 0,0/24
14:31:51.371704 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 141:361, ack 441, win 235, options [nop,nop,TS val 2832333936 ecr 2836344530], length 220: NFS reply xid 3338428589 reply ok 216 getattr NON 3 ids 0/10 sz 0
14:31:51.371820 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 441:557, ack 361, win 237, options [nop,nop,TS val 2836344531 ecr 2832333936], length 116: NFS request xid 3355205805 getattr fh 0,0/22
14:31:51.372273 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 361:457, ack 557, win 235, options [nop,nop,TS val 2832333937 ecr 2836344531], length 96: NFS reply xid 3355205805 reply ok 92 getattr NON 2 ids 0/9 sz 0
14:31:51.372727 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 557:677, ack 457, win 237, options [nop,nop,TS val 2836344532 ecr 2832333937], length 120: NFS request xid 3371983021 getattr fh 0,0/22
14:31:51.372999 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 457:569, ack 677, win 235, options [nop,nop,TS val 2832333937 ecr 2836344532], length 112: NFS reply xid 3371983021 reply ok 108 getattr NON 2 ids 0/9 sz 0
14:31:51.373105 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 677:793, ack 569, win 237, options [nop,nop,TS val 2836344532 ecr 2832333937], length 116: NFS request xid 3388760237 getattr fh 0,0/22
14:31:51.373371 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 569:665, ack 793, win 235, options [nop,nop,TS val 2832333938 ecr 2836344532], length 96: NFS reply xid 3388760237 reply ok 92 getattr NON 2 ids 0/9 sz 0
14:31:51.373488 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 793:913, ack 665, win 237, options [nop,nop,TS val 2836344533 ecr 2832333938], length 120: NFS request xid 3405537453 getattr fh 0,0/22
14:31:51.373930 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 665:777, ack 913, win 235, options [nop,nop,TS val 2832333938 ecr 2836344533], length 112: NFS reply xid 3405537453 reply ok 108 getattr NON 2 ids 0/9 sz 0
14:31:51.374008 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 913:1029, ack 777, win 237, options [nop,nop,TS val 2836344533 ecr 2832333938], length 116: NFS request xid 3422314669 getattr fh 0,0/22
14:31:51.374258 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 777:853, ack 1029, win 235, options [nop,nop,TS val 2832333939 ecr 2836344533], length 76: NFS reply xid 3422314669 reply ok 72 getattr NON 2 ids 0/9 sz 0
14:31:51.374456 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1029:1145, ack 853, win 237, options [nop,nop,TS val 2836344534 ecr 2832333939], length 116: NFS request xid 3439091885 getattr fh 0,0/22
14:31:51.374739 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 853:949, ack 1145, win 235, options [nop,nop,TS val 2832333939 ecr 2836344534], length 96: NFS reply xid 3439091885 reply ok 92 getattr NON 2 ids 0/9 sz 0
14:31:51.374817 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1145:1265, ack 949, win 237, options [nop,nop,TS val 2836344534 ecr 2832333939], length 120: NFS request xid 3455869101 getattr fh 0,0/22
14:31:51.375060 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 949:1149, ack 1265, win 235, options [nop,nop,TS val 2832333939 ecr 2836344534], length 200: NFS reply xid 3455869101 reply ok 196 getattr NON 2 ids 0/9 sz 0
14:31:51.375150 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1265:1393, ack 1149, win 245, options [nop,nop,TS val 2836344534 ecr 2832333939], length 128: NFS request xid 3472646317 getattr fh 0,0/22
14:31:51.375430 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1149:1277, ack 1393, win 243, options [nop,nop,TS val 2832333940 ecr 2836344534], length 128: NFS reply xid 3472646317 reply ok 124 getattr NON 3 ids 0/3 sz 0
14:31:51.375614 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1393:1533, ack 1277, win 254, options [nop,nop,TS val 2836344535 ecr 2832333940], length 140: NFS request xid 3489423533 getattr fh 0,0/22
14:31:51.376419 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1277:1505, ack 1533, win 252, options [nop,nop,TS val 2832333941 ecr 2836344535], length 228: NFS reply xid 3489423533 reply ok 224 getattr NON 4 ids 0/15 sz 0
14:31:51.376696 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1533:1673, ack 1505, win 262, options [nop,nop,TS val 2836344536 ecr 2832333941], length 140: NFS request xid 3506200749 getattr fh 0,0/22
14:31:51.377258 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1505:1733, ack 1673, win 260, options [nop,nop,TS val 2832333941 ecr 2836344536], length 228: NFS reply xid 3506200749 reply ok 224 getattr NON 4 ids 0/15 sz 0
14:31:51.377489 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1673:1789, ack 1733, win 270, options [nop,nop,TS val 2836344537 ecr 2832333941], length 116: NFS request xid 3522977965 getattr fh 0,0/22
14:31:51.377878 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1733:1829, ack 1789, win 260, options [nop,nop,TS val 2832333942 ecr 2836344537], length 96: NFS reply xid 3522977965 reply ok 92 getattr NON 2 ids 0/9 sz 0
14:31:51.377971 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1789:1909, ack 1829, win 270, options [nop,nop,TS val 2836344537 ecr 2832333942], length 120: NFS request xid 3539755181 getattr fh 0,0/22
14:31:51.378306 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1829:1941, ack 1909, win 260, options [nop,nop,TS val 2832333943 ecr 2836344537], length 112: NFS reply xid 3539755181 reply ok 108 getattr NON 2 ids 0/9 sz 0
14:31:51.378449 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 1909:2025, ack 1941, win 270, options [nop,nop,TS val 2836344538 ecr 2832333943], length 116: NFS request xid 3556532397 getattr fh 0,0/22
14:31:51.378756 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 1941:2017, ack 2025, win 260, options [nop,nop,TS val 2832333943 ecr 2836344538], length 76: NFS reply xid 3556532397 reply ok 72 getattr NON 2 ids 0/9 sz 0
14:31:51.378932 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2025:2141, ack 2017, win 270, options [nop,nop,TS val 2836344538 ecr 2832333943], length 116: NFS request xid 3573309613 getattr fh 0,0/22
14:31:51.379180 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2017:2113, ack 2141, win 260, options [nop,nop,TS val 2832333944 ecr 2836344538], length 96: NFS reply xid 3573309613 reply ok 92 getattr NON 2 ids 0/9 sz 0
14:31:51.379277 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2141:2261, ack 2113, win 270, options [nop,nop,TS val 2836344538 ecr 2832333944], length 120: NFS request xid 3590086829 getattr fh 0,0/22
14:31:51.380350 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2113:2313, ack 2261, win 260, options [nop,nop,TS val 2832333945 ecr 2836344538], length 200: NFS reply xid 3590086829 reply ok 196 getattr NON 2 ids 0/9 sz 0
14:31:51.420441 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2313, win 279, options [nop,nop,TS val 2836344580 ecr 2832333945], length 0
14:32:51.419303 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832393984 ecr 2836344529], length 0
14:32:51.419334 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836404578 ecr 2832333935], length 0
14:32:51.576487 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2313, win 279, options [nop,nop,TS val 2836404736 ecr 2832333945], length 0
14:32:51.576650 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2261:2357, ack 2313, win 279, options [nop,nop,TS val 2836404736 ecr 2832333945], length 96: NFS request xid 3606864045 getattr [|nfs]
14:32:51.576838 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 2261, win 260, options [nop,nop,TS val 2832394141 ecr 2836344580], length 0
14:32:51.577113 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2313:2361, ack 2357, win 260, options [nop,nop,TS val 2832394141 ecr 2836404736], length 48: NFS reply xid 3606864045 reply ok 44 getattr [|nfs]
14:32:51.577136 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2361, win 279, options [nop,nop,TS val 2836404736 ecr 2832394141], length 0
14:33:51.579310 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832454144 ecr 2836404578], length 0
14:33:51.579343 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836464738 ecr 2832333935], length 0
14:33:51.736500 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2361, win 279, options [nop,nop,TS val 2836464896 ecr 2832394141], length 0
14:33:51.736907 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 2357, win 260, options [nop,nop,TS val 2832454301 ecr 2836404736], length 0
14:33:51.736990 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2357:2453, ack 2361, win 279, options [nop,nop,TS val 2836464896 ecr 2832454301], length 96: NFS request xid 3623641261 getattr [|nfs]
14:33:51.737364 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2361:2409, ack 2453, win 260, options [nop,nop,TS val 2832454302 ecr 2836464896], length 48: NFS reply xid 3623641261 reply ok 44 getattr [|nfs]
14:33:51.737430 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2409, win 279, options [nop,nop,TS val 2836464897 ecr 2832454302], length 0
14:34:51.739332 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832514304 ecr 2836464738], length 0
14:34:51.739362 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836524898 ecr 2832333935], length 0
14:34:51.896515 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2409, win 279, options [nop,nop,TS val 2836525056 ecr 2832454302], length 0
14:34:51.896898 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2453:2549, ack 2409, win 279, options [nop,nop,TS val 2836525056 ecr 2832454302], length 96: NFS request xid 3640418477 getattr [|nfs]
14:34:51.896935 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 2453, win 260, options [nop,nop,TS val 2832514461 ecr 2836464897], length 0
14:34:51.897285 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2409:2457, ack 2549, win 260, options [nop,nop,TS val 2832514462 ecr 2836525056], length 48: NFS reply xid 3640418477 reply ok 44 getattr [|nfs]
14:34:51.897308 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2457, win 279, options [nop,nop,TS val 2836525056 ecr 2832514462], length 0
14:35:51.899293 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832574464 ecr 2836524898], length 0
14:35:51.899338 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836585058 ecr 2832333935], length 0
14:35:52.056506 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2457, win 279, options [nop,nop,TS val 2836585216 ecr 2832514462], length 0
14:35:52.057032 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 2549, win 260, options [nop,nop,TS val 2832574621 ecr 2836525056], length 0
14:35:52.057205 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2549:2645, ack 2457, win 279, options [nop,nop,TS val 2836585216 ecr 2832574621], length 96: NFS request xid 3657195693 getattr [|nfs]
14:35:52.057602 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2457:2505, ack 2645, win 260, options [nop,nop,TS val 2832574622 ecr 2836585216], length 48: NFS reply xid 3657195693 reply ok 44 getattr [|nfs]
14:35:52.057632 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2505, win 279, options [nop,nop,TS val 2836585217 ecr 2832574622], length 0
14:36:52.059300 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 29, win 229, options [nop,nop,TS val 2832634624 ecr 2836585058], length 0
14:36:52.059333 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [.], ack 73, win 227, options [nop,nop,TS val 2836645218 ecr 2832333935], length 0
14:36:52.216516 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2505, win 279, options [nop,nop,TS val 2836645376 ecr 2832574622], length 0
14:36:52.216916 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [.], ack 2645, win 260, options [nop,nop,TS val 2832634781 ecr 2836585217], length 0
14:36:52.217030 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [P.], seq 2645:2741, ack 2505, win 279, options [nop,nop,TS val 2836645376 ecr 2832634781], length 96: NFS request xid 3673972909 getattr [|nfs]
14:36:52.217474 IP 10.3.3.11.2049 > 10.3.0.99.47977: Flags [P.], seq 2505:2553, ack 2741, win 260, options [nop,nop,TS val 2832634782 ecr 2836645376], length 48: NFS reply xid 3673972909 reply ok 44 getattr [|nfs]
14:36:52.217497 IP 10.3.0.99.47977 > 10.3.3.11.2049: Flags [.], ack 2553, win 279, options [nop,nop,TS val 2836645377 ecr 2832634782], length 0
14:36:52.315321 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [F.], seq 73, ack 29, win 229, options [nop,nop,TS val 2832634880 ecr 2836645218], length 0
14:36:52.315446 IP 10.3.0.99.48044 > 10.3.3.11.940: Flags [F.], seq 29, ack 74, win 227, options [nop,nop,TS val 2836645475 ecr 2832634880], length 0
14:36:52.315905 IP 10.3.3.11.940 > 10.3.0.99.48044: Flags [.], ack 30, win 229, options [nop,nop,TS val 2832634880 ecr 2836645475], length 0

Output of rpcinfo on the server:

[root@iafw1 ~]# rpcinfo -p
 program vers proto port service
 100000 4 tcp 111 portmapper
 100000 3 tcp 111 portmapper
 100000 2 tcp 111 portmapper
 100000 4 udp 111 portmapper
 100000 3 udp 111 portmapper
 100000 2 udp 111 portmapper
 100005 1 udp 20048 mountd
 100005 1 tcp 20048 mountd
 100005 2 udp 20048 mountd
 100005 2 tcp 20048 mountd
 100005 3 udp 20048 mountd
 100005 3 tcp 20048 mountd
 100003 3 tcp 2049 nfs
 100003 4 tcp 2049 nfs
 100227 3 tcp 2049 nfs_acl
 100003 3 udp 2049 nfs
 100003 4 udp 2049 nfs
 100227 3 udp 2049 nfs_acl
 100021 1 udp 43355 nlockmgr
 100021 3 udp 43355 nlockmgr
 100021 4 udp 43355 nlockmgr
 100021 1 tcp 40652 nlockmgr
 100021 3 tcp 40652 nlockmgr
 100021 4 tcp 40652 nlockmgr
 100024 1 udp 60369 status
 100024 1 tcp 45690 status

Adding output of netstat on server:

[root@iafw1 ~]# netstat -tulip
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
lan0 1500 132296683 0 0 0 100476488 0 0 0 BMRU
lan0:10 1500 - no statistics available - BMRU
lan0:11 1500 - no statistics available - BMRU
lo 65536 618307922 0 0 0 618307922 0 0 0 LRU

And another netstat command on server. You can see the normal NFSv4 connection on port 2049 and the second mystery connection on port 982 (note, this is different from the tcpdump port of 940 because it's a different mount). The '-' in the last column means it's owned by the kernel (vs. user-space process).

[root@iafw1 ~]# netstat -anp | grep 10.3.0.99
tcp 0 0 10.3.3.12:22 10.3.0.99:42554 ESTABLISHED 29450/sshd: root@pt 
tcp 0 0 10.3.3.11:982 10.3.0.99:48044 ESTABLISHED - 
tcp 0 0 10.3.3.11:2049 10.3.0.99:47979 ESTABLISHED -