mjhjmtu

arch linux with lts kernel

systemd networkd is configured to bond two NICs into LACP trunk with smart switch (DGS-1210-28). But it fails after OS startup - only one NIC is used for bond connection. As shown in /proc/net/bonding/*

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 1000
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: **:**:**:**:**:ad
Active Aggregator Info:
 Aggregator ID: 2
 Number of ports: 1
 Actor Key: 9
 Partner Key: 3
 Partner Mac Address: **:**:**:**:**:6e

Slave Interface: enp7s0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: **:**:**:**:**:67
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
 system priority: 65535
 system mac address: **:**:**:**:**:ad
 port key: 0
 port priority: 255
 port number: 1
 port state: 69
details partner lacp pdu:
 system priority: 65535
 system mac address: 00:00:00:00:00:00
 oper key: 1
 port priority: 255
 port number: 1
 port state: 1

Slave Interface: enp2s0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: **:**:**:**:**:ef
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
 system priority: 65535
 system mac address: **:**:**:**:**:ad
 port key: 9
 port priority: 255
 port number: 2
 port state: 61
details partner lacp pdu:
 system priority: 32768
 system mac address: **:**:**:**:**:6e
 oper key: 3
 port priority: 128
 port number: 15
 port state: 61

notice that bond have AggregatorID=2, and unused (churned) NIC have AggregatorID=1, which is weird. I'd expect bond to have AggregatorID=1

This can be "fixed" by physical port reconnect for churned NIC. After this both NICs are used by bond. This can be seen in kernel log:

>> OS startup

[ ]: r8169 0000:07:00.0 enp7s0: renamed from eth1
[ ]: r8169 0000:02:00.0 enp2s0: renamed from eth0
[ ]: Trunk0: Enslaving enp7s0 as a backup interface with a down link
[ ]: Trunk0: Enslaving enp2s0 as a backup interface with a down link
[ ]: r8169 0000:07:00.0 enp7s0: Link is Up - 1Gbps/Full - flow control off
[ ]: Trunk0: link status up again after 0 ms for interface enp7s0
[ ]: Trunk0: Warning: No 802.3ad response from the link partner for any adapters in the bond
[ ]: 8021q: adding VLAN 0 to HW filter on device Trunk0
[ ]: Trunk0: link status up again after 0 ms for interface enp7s0
[ ]: Trunk0: link status definitely down for interface enp2s0, disabling it
[ ]: Trunk0: Warning: No 802.3ad response from the link partner for any adapters in the bond
[ ]: r8169 0000:02:00.0 enp2s0: Link is Up - 1Gbps/Full - flow control off
[ ]: Trunk0: link status definitely up for interface enp2s0, 1000 Mbps full duplex
[ ]: Trunk0: first active interface up!
[ ]: Trunk0: Warning: No 802.3ad response from the link partner for any adapters in the bond

>> churned NIC physical disconnect

[ ]: r8169 0000:07:00.0 enp7s0: Link is Down
[ ]: Trunk0: link status definitely down for interface enp7s0, disabling it
[ ]: Trunk0: first active interface up!

>> churned NIC physical connect

[ ]: r8169 0000:07:00.0 enp7s0: Link is Up - 1Gbps/Full - flow control off
[ ]: Trunk0: link status definitely up for interface enp7s0, 1000 Mbps full duplex

notice that NIC "enp7s0" initialized first, but gets churned.

Config files:

enp-any.network

[Match]
Name=enp*

[Network]
Bond=Trunk0

Trunk0.netdev

[NetDev]
Name=Trunk0
Kind=bond

[Bond]
Mode=802.3ad
TransmitHashPolicy=layer3+4
MIIMonitorSec=1s
LACPTransmitRate=slow

Trunk0.network

[Match]
Name=Trunk0

[Network]
VLAN=***
VLAN=***
VLAN=***
LinkLocalAddressing=no
BindCarrier=enp2s0 enp7s0