SOLVED: [Xen-users] xen 2.0.6 and wifi problems

[Michal Ludvig <michal@xxxxxxxx>]

Michal Ludvig wrote:

> I'm running xen 2.0.6 with linux 2.6.8 in dom0 and domU. The machine has
> two NICs - WiFi on network 192.168.157.0/25 and "wired" on
> 192.168.157.128/25. Dom0 uses .21 (dev wifi0) and .221 (dev wire0) and
> can access router on over both channels on .1/.129. However when I start
> up domU and assign addresses .41 to the interface bridged on wifi and
> .241 to the other, only the wired network works. No luck with wifi.

I guess other people may face this problem as well so just for the
record here is some discussion and solution.

* The culprit is not Xen but the wireless card which refuses to send out
packets with different MAC address than its own. But as Xen deploys
ethernet bridge to provide network for domU kernels the outgoing packets
have a virtual source MAC address different from the real one of the
WiFi card. You don't need Xen to observe this problem - simple ethernet
bridge constructed with brctl between wired and wireless segments will
not work with many wifi cards either. See "It doesn't work with my
Wireless card" at [1].

* The solution obviously is to change the source MAC address to the real
one on all outgoing packets. At first I tried to set it in
vif=['mac=...'] config option for domU host, but it didn't work. Better
solution turned out to be deployment of 'ebtables' [2] - netfilter
infrastructure similar to 'iptables' but working with MAC addresses
instead of IP addresses. Given that domU MAC address is
00:60:00:00:00:01 and wifi card MAC address is 00:60:aa:bb:cc:dd the
simplest ebfilter rule that made it work is:

ebtables -t nat -A POSTROUTING -s 00:60:00:00:00:01 \
              -j snat --to-source 00:60:aa:bb:cc:dd

You may want to add some more checks, e.g. for source interface name or
alike. This is the bare minimum.

* Surprisingly enough there doesn't have to be a corresponding reversed
'dnat' rule like. Ping from outside to domU triggers this packet
sequence (00:02:11:22:33:44 is my workstation MAC address):

01:56:03.972501 00:02:11:22:33:44 > ff:ff:ff:ff:ff:ff,
        ethertype ARP (0x0806), length 42:
                arp who-has 192.168.157.41 tell 192.168.157.20
01:56:03.975205 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,
        ethertype ARP (0x0806), length 60:
                arp reply 192.168.157.41 is-at 00:60:00:00:00:01

Interesting point is the ARP reply. Although it comes from wifi real MAC
address, inside in the packet it carries the domU virtual MAC.
Subsequently my workstation created an entry in its ARP table with the
virtual(!) address of domU and started sending all further communication
to that MAC:

01:56:03.975231 00:02:11:22:33:44 > 00:60:00:00:00:01,
        ethertype IPv4 (0x0800), length 98:
                IP 192.168.157.20 > 192.168.157.41:
                        icmp 64: echo request seq 1
01:56:03.982348 00:60:aa:bb:cc:dd > 00:02:11:22:33:44,
        ethertype IPv4 (0x0800), length 98:
                IP 192.168.157.41 > 192.168.157.20:
                        icmp 64: echo reply seq 1

As I said in my first posting the packets destined to the virtual MAC
are successfully delivered to domU, just the outgoing ones are blocked.
That's why it all works like a charm with just a single ebtables snat rule.

Q.O.D.

BTW I wonder if there are any caveats in this setup. Linux kernel seems
to always update its ARP table from the non-snat'ed contents of the ARP
packet, not from the snat'ed ethernet headers and it all looks just fine...

[1] - http://bridge.sourceforge.net/faq.html
[2] - http://ebtables.sourceforge.net/

Enjoy!

Michal Ludvig
-- 
* Personal homepage: http://www.logix.cz/michal
* Novy Zeland: http://kiwi.logix.cz







_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users