RE: How to setup a multi vlan connection to an (Ubuntu) XEN-server !!??

["louis.forums@xxxxxxxxx" <lsrbreda@xxxxxxxxx>]

Simon, here some explanation and reaction to your comments

Lets start with the fact that I have to deal with two aspects:
1) Interfacing the host with the network
2) interfacing the host with the xen-hypervisor
The first problem to solve was how to interface the host with the network. 

The host is connected to the network via vlan's (or untagged interface). Those 
vlans are there for security reasons. And traffic should NEVER  be allowed to 
pass from vlan to another one in an endpoint like a server. Unless of course 
the server is used as firewall or managed switch. That is the reason that I 
absolutely require that traffic which enters via vlan-x is also answered via 
vlan-x.

As a consequence things like:
- a server wide default gateway is in principle a no go (I can live with it if 
it is assured by other means that traffic cannot leave a vlan)
- priority settings and routes OK but only withing the scope of a particular 
vlan (I do not use that here)
- as far as I know I need routes for incoming traffic and routing-policy for 
outgoing traffic. If I do not use both e.g. the answers to incoming traffic 
will be send through the host wide default gateway, where I want the answers 
send back via the same vlan as the incoming traffic arrived.

Related to MAC-addresses:
- of course I know the mac-addresses of the interfaces, however I do not see 
any reason to use them here on IP-level, ......... apart from one reason. My 
original intention was to change the interface names via "set-name", but apart 
that the syntax is bizarre (first that name and then the definition (strange), 
it did not work when using  match name <interface name> ...... and it is clear 
that match on mac does work (ihmo it is a bug)

Related to IP-addresses:
- I partly agree. I completely agree that the IP-address of a VM should be 
defined in the VM. However, the network is also there in favor of the host and 
DOM0. And at least the host (ubuntu) does need an IP. 
- for testing the vlans it is handy to have an ip assigned as well 
    
The trunk does not need an address, we agree and also on the fact that I should 
use an RFC1918 ranges address

Related to routing in conjunction to DHCP
- The first NIC is used to setup the system and for emergency purposes. The NIC 
is connected to a normal PC-lan and it is most logical to have the IP assigned 
via DNCP. However in this case traffic arriving via this interfaces should be 
answered via this interface as well. So ideally I would define the routes and 
routing-policy for this interface as well ... 
- however that seems to be impossible since the address is not static, unless I 
can do something with the "on-link" keyword. 
- note that I tried that and was not successful (perhaps I did something wrong 
??) 
   
Related to naming:
- I completely agree that it is better to give interfaces, but also e.g. vlans 
sensible names. Not sure that is possible for .e.g. vlans

Policy based routing
- You could use that to send specific destinations to a particular interface, 
apart if that is a server or a network switch function, I do not need that here
- the reason I use it is that as far as I know is to force the host/vm answers 
back into the vlan from which the answer arrived

Related to Bridges
- I do not know yet I need them. That is because I do not yet know yet how to 
connect one or more vm's, or dom0, to the host its network as defined in the 
actual stage
- perhaps I can simply connect one or more vm's or dom0 to an interface or 
vlan, however perhaps I need to use a bridge for that. My feeling is that a 
bridge is the equivalent of an unmanged switch ... only required if you want to 
connect multiple (logical) devices (vm's !?) to an interface / vlan.   


Louis


-----Original Message-----
From: Xen-users <xen-users-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of Simon
Sent: Saturday, October 8, 2022 10:17 PM
To: xen-users@xxxxxxxxxxxxxxxxxxxx
Subject: Re: How to setup a multi vlan connection to an (Ubuntu) XEN-server !!??

Sorry, a bit of rambling ...


louis.forums@xxxxxxxxx <lsrbreda@xxxxxxxxx> wrote:

> - you are using macaddresses. Why?

MAC addresses are used by bridges for a number of functions.
For STP, in the absence of specific priority settings, the MAC addresses will 
determine the network topology.
I would assume that if you want to attach an IP address to the bridge and use 
it as an interface from the host then it will need a MAC address.
There are probably other reasons I’m not aware of - quite likely the network 
stack needs it even if the host isn’t using it as an interface.

> - you do not assign ip-addresses. Why?

The host does not need an IP address in any plan/bridge it isn’t using for it’s 
own networking.

> - you do not specify vlan related gateways and routing tables .... ??

Again, not needed if the host is not actively using that interface for its own 
traffic

> - is het necessary to add a bridge if there is only one vm using a 
> certain vlan

I believe so. In any case, it’s the simplest way to do it. The host can create 
the trunk interface, VLANs, and bridges at boot time, then they are ready for 
the the guests to attach to as and when required. Note that the guest side of 
the networking is not configured until the guest starts some time after the 
host network config is set up. Also, the guest side will be deleted if/when the 
guest is shut down.


> ## Issues left ##
> # 1) So I did not define a server wide default gateway ....
> # .... however that is not true if enp5s0 becomes active ....
> # .... I simply do not know how to assign a routing and routing-policy 
> to # an dhcp provided address

I notice you have default routes configured on multiple interfaces. Normally 
this is incorrect - you’d only want a single default route, and potentially 
other specific routes via specific interfaces. Is this the reason for policy 
based routing (see below) ?
Are you forced to use DHCP for this ? If not, I would suggest static config 
would be preferable.

> # 2) The trunk is a container for vlans, it does not have an address ...
> # .... however I had to give it a dummy address to make this file work

So that’s a problem with the configuration tool because - as you point out - it 
absolutely does not need an IP address **unless** you wish to use it directly 
as a host interface. I would stick to using an address from the RFC1918 ranges 
BTW.


> The attached a jaml config file gives being an example for a server having 
> three interfaces. One of the interfaces is the vlan trunk. In the attached 
> file one vlan, but of course it is no problem to add other vlans or to add 
> IPV6 (I assume).

I am curious why you are applying policy based routing - either there's some 
detail (probably not important to the discussion) you’ve omitted, or possibly 
it’s not needed. In many years, I’ve only needed policy based (source address) 
routing in one network - and that was a campus network where I had to handle 2 
providers, separate IP ranges, and phased migration, hence needing to route via 
a specific interface according the IP address used by a connected client.
Ah, I see https://netplan.io/examples gives something like your config (key 
difference - example has different routing table numbers), but from the 
explanation I cannot fathom out what it’s saying ! I’m used to working with 
systems that handle what I think the page is saying “out of the box”, but since 
I don’t understand what problem is being fixed, I can’t see if this is relevant 
or not.

While on routing ...
The “on-link” statements are not needed as your router addresses are within the 
same subnets as the interface addresses. on-link is specifically for the case 
where this is not so - where normally the system would ignore the route as the 
router address is notionally unreachable. I think all you need is :
>           routes:
>               - to: default
>                 via: n.n.n.n
(https://netplan.io/examples)


BTW - my personal opinion is that you should really look into whether you can 
give the interfaces “sensible” names. Which enp3sf1 is not the worst of the 
naming schemes, you will really curse it the first time you change the host 
networking. As it is, you will need to find every instance of the old interface 
name in every config file its used in - it won’t just be in this file. And if 
you want to move a guest to a different host, you’ll need to change its config 
to suit the host’s interface names.
I’ve always done this via dev rules - and while some consider it heresy to use 
the MAC address as an interface identifier, it’s literally one line per 
interface and only one place to update if anything changes (replacement NIC, 
add/remove a card that renumbers other cards, move to new hardware, whatever).

Ah, I see from 
https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html 
that it’s possible to do it with a match statement in the config file.
> In the generic case, they can be selected by match: rules on desired 
> properties, such as name/name pattern, MAC address, driver, or device paths.


https://netplan.io/examples gives a worked example :
>   ethernets:
>       mainif:
>           match:
>               macaddress: "de:ad:be:ef:ca:fe"
>           set-name: mainif

And mainif is used elsewhere in the config.


I also note that your config does not define any bridges.


Simon