[Pkg-xen-devel] Bug#1042842: network interface names wrong in domU (>10 interfaces)

Tue Aug 8 15:59:03 BST 2023

Hi,

On 8/8/23 15:22, Valentin Kleibel wrote:
>> On [0], you can read "In both cases the device naming is subject to the 
>> usual guest or backend domain facilities for renaming network devices".
>> It says "naming/renaming", but you can assume "detecting".
>>
>>> I also checked which net_ids udev knows about and the only things that 
>>> pop up are:
>>> ID_NET_NAMING_SCHEME=v247
>>> ID_NET_NAME_MAC=enx00163efd832b
>>> ID_OUI_FROM_DATABASE=Xensource, Inc.

What I do is stuff like this:

-$ cat /etc/udev/rules.d/70-persistent-vifname.rules

SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/0",
NAME="vlan2"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/1",
NAME="vlan3"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/2",
NAME="vlan4"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/3",
NAME="vlan6"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/4",
NAME="vlan9"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/5",
NAME="vlan10"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/6",
NAME="vlan11"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/7",
NAME="vlan12"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/8",
NAME="vlan13"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/9",
NAME="vlan14"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/10",
NAME="vlan15"
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{nodename}=="device/vif/11",
NAME="vlan16"

The vif/X always matches the order in which you define the interfaces
inside the guest config file.

After starting to build router VMs (well before the whole interface
naming madness was a thing), it took only the first time when we wanted
to throw away a vlan, to realize that all the ethX numbers would shift 1
up, and from then on, I've always been using this so set my own style
predictable names (whenever there's more than one, otherwise it's just
eth0).

>> Is it from dom0 or domU ?
>> Are you using "net.ifnames=0" on the domU kernel command line ?
>> "v247" looks like systemd "predictive naming scheme" (eth -> enX).
>>  From bookworm on, domUs vifs get named enXN (enX0, enX1, ...).
>> Read on :
>> https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#xen-network
> 
> This is from the domU, running bullseye with a bookworm dom0.
> 
>> See how ethN interfaces get messed up, like in your setup, but 
>> predictable names would work, as you can see in "altname enXN" :
>> eth1 (:01) -> enX1
>> eth2 (:10) -> enX10
>> eth3 (:02) -> enX2

But yeah, so, even while not depending on whatever order it gets
initialized, and still having it function correctly, this is still just
pretty annoying... If I'm doing stuff around here, and just quickly want
to look up things (e.g. messing around with vlan15 settings), and
quickly type ip a instead of having to spend more time typing ip a show
dev vlan15 jadijadi, I still every time get this short "WTF huh, argh",
raises arms, does table flip, grmbl grbml feeling for a split second.

2: vlan2: <BROADCAST,MULT[...]
    [...]
    altname enX0
3: vlan3: <BROADCAST,MULT[...]
    [...]
    altname enX1
4: vlan15: <BROADCAST,MULT[...]
    [...]
    altname enX10
5: vlan16: <BROADCAST,MULT[...]
    [...]
    altname enX11
6: vlan4: <BROADCAST,MULT[...]
    [...]
    altname enX2
7: vlan6: <BROADCAST,MULT[...]
    [...]
    altname enX3
8: vlan9: <BROADCAST,MULT[...]
    [...]
    altname enX4
9: vlan10: <BROADCAST,MULT[...]
    [...]
    altname enX5
10: vlan11: <BROADCAST,MULT[...]
    [...]
    altname enX6
11: vlan12: <BROADCAST,MULT[...]
    [...]
    altname enX7
12: vlan13: <BROADCAST,MULT[...]
    [...]
    altname enX8
13: vlan14: <BROADCAST,MULT[...]
    [...]
    altname enX9

First, there's the split second start of "wait, what, I just added an IP
address, where did the whole interface go?!" and then the "Oh, right, of
course, it's the silly thing again.", and then it costs an extra second
looking at the screen like a confused Jeffrey Lebowski to find the right
line.

> I could not get our bullseye domU to show the "predictable names" even 
> though i tried installing the bullseye-backports kernel 6.1.
> After you wrote this i installed udev 252.5 from backports and it now 
> uses the correct enXn interface names, even with kernel 5.10.
> 
>> So, my answer does not tell you if something changed in Xen itself, only 
>> in Debian.
>> But I guess it relates to what Xen devs told us : vifs detection order 
>> cannot be relied upon, that's why "predictable names" were invented.
>> The vif detection part is related to the domains kernels, not Xen itself 
>> (at least that's what I understood).
>>
>> Using eth0 nowadays is a bit like using /dev/sda for hard drives, it's 
>> considered legacy as it may create problems in some setups, like yours 
>> (ie. for disks, it's recommended to use UUIDs or /dev/disk/by-*).
>>
>> I hope this answers your question.
> 
> Thank you, yes it does.
> 
> In our case the dom0 was updated to bookworm while the domU is still 
> running bullseye.
> -> updated Xen so the vif detection order changed (which we relied on)

I didn't read the other mailthread on the xen list fully yet. But, I
think it's shouldn't be very hard to find the code changes and see if
it's deterministic and can just be fixed. Simply just to decrease the
totally unnecessary amount of silliness.

> -> the predictable network names for Xen don't work with bullseye
> 
> So my new resolution for bullseye domUs on a bookworm dom0 is to install 
> udev from backports and change the domUs network config to use the new 
> enXn naming scheme instead of ethn.

Or the "device/vif/X" way...

So, anyway, did someone already did some test "just because we can" to
see how much network interfaces you can get added for fun, and if the
pattern keeps looking the same, also with enX4 enX40 .. enX49 enX5 etc?
:D enX1 enX10 enX100 .. enX109 enX11 enX110 argh o_O

Have fun,
Hans