Xen project Mailing List

On Sun, Jun 06, 2010 at 11:54:11AM -0700, Boris Derzhavets wrote:
> Virt-install hangs attempting to retrieve updates.img from Apache Mirror
> setup at Dom0
> with kernel 2.6.32.15
> It doesn't happen with 2.6.32.10 (12,14 final).
> Environment Xen 4.0 Dom0 on top Ubuntu 10.04 Server. Libvirt is 0.8.0 .
> Attempting to virt-install F13 PV DomU in vnc mode.
>
> I believe that builds are consistent.
> Runtime behavior is the same for .10, .14, .15. Seems to be communicating
> problem between DomU and Dom0 during HTTP download.
>

I'm seeing PV domU network issues aswell.. when running Xen 4.0.0 on Fedora 13.
I'll have to dig more into it..

-- Pasi

> Boris.
>
> --- On Sun, 6/6/10, Jeremy Fitzhardinge <jeremy@xxxxxxxx> wrote:
>
> From: Jeremy Fitzhardinge <jeremy@xxxxxxxx>
> Subject: Re: [Xen-users] Re: [Xen-devel] ARP problems with xen 4.0 with
> pvops kernel 2.6.32.15
> To: "Boris Derzhavets" <bderzhavets@xxxxxxxxx>
> Cc: luis.silva@xxxxxxxxxxxxx, xen-devel@xxxxxxxxxxxxxxxxxxx,
> xen-users@xxxxxxxxxxxxxxxxxxx
> Date: Sunday, June 6, 2010, 12:43 PM
>
> On 06/06/2010 03:19 AM, Boris Derzhavets wrote:
> > Network issues when working with DomUs in 2.6.32.14 and finally been
> > fixed,
> > seem to appear again in 2.6.32.15. Reverting to back to xen/stable -
> > 2.6.32.10
> > works as a fix again.
> >
>
> There are no substantial differences between 2.6.32.14 and .15. If
> there are any differences in behaviour between them, then I'd suspect
> some inconsistency from boot to boot, or in your kernel build process.
>
> J
>
> >
> > Boris
> >
> > --- On *Thu, 6/3/10, Luís Silva /<[1]luis.silva@xxxxxxxxxxxxx>/*
> wrote:
> >
> >
> >    From: Luís Silva <[2]luis.silva@xxxxxxxxxxxxx>
> >    Subject: Re: [Xen-users] Re: [Xen-devel] ARP problems with xen 4.0
> >    with pvops kernel
> >    To: "Boris Derzhavets" <[3]bderzhavets@xxxxxxxxx>
> >    Cc: "Jeremy Fitzhardinge" <[4]jeremy@xxxxxxxx>,
> >    [5]xen-devel@xxxxxxxxxxxxxxxxxxx, [6]xen-users@xxxxxxxxxxxxxxxxxxx
> >    Date: Thursday, June 3, 2010, 6:20 AM
> >
> >    Hello,
> >
> >    Thanks for the suggestion, xen/stable works ok for me. Only
> >    problem is that I have to disable offload do get dhcp to work on
> >    domU, but the problem I described before doesn't exist in this
> >    kernel. Later today I'm going to try a previous build I have based
> >    on stable-2.6.32.x (2.6.32.13) to check if it already had this
> >    problem or not and I'll post the results.
> >
> >    Luís
> >
> >    On Wed, 2010-06-02 at 12:26 -0700, Boris Derzhavets wrote:
> >>    Could you,please, build and try 2.6.32.10 ( xen/stable) ?
> >>
> >>    Boris.
> >>
> >>    --- On *Wed, 6/2/10, Luís Silva
> **/<[7]luis.silva@xxxxxxxxxxxxx>/*
> >>    wrote:
> >>
> >>
> >>    From: Luís Silva <[8]luis.silva@xxxxxxxxxxxxx>
> >>    Subject: [Xen-users] Re: [Xen-devel] ARP problems with xen
> >>    4.0 with pvops kernel
> >>    To: "Jeremy Fitzhardinge" <[9]jeremy@xxxxxxxx>
> >>    Cc: [10]xen-devel@xxxxxxxxxxxxxxxxxxx,
> [11]xen-users@xxxxxxxxxxxxxxxxxxx
> >>    Date: Wednesday, June 2, 2010, 2:53 PM
> >>
> >>    Hello,
> >>
> >>    On Wed, 2010-06-02 at 09:06 -0700, Jeremy Fitzhardinge wrote:
> >>>    On 06/02/2010 01:47 AM, Luís Silva wrote:
> >>>    > Hello,
> >>>    >
> >>>    > I'm using the latest stable-2.6.32.x. I already tried
> "ethtool -K
> >>>    > <bridge> tx off", but that didn't make any difference.
> Also, this only
> >>>    > happen with pv, in hvm mode all works ok and the domU sees
> the arp
> >>>    > messages...
> >>>
> >>>    Yes, ARP is a new twist on network problems. I'm guessing
> you're using
> >>>    hvm without stubdoms, which means that its networking
> originates from
> >>>    qemu within dom0, whereas PV and HVM+stubdom comes via
> netback.
> >>>
> >>>
> >>    Yes, when I mentioned hvm I was talking about hvm without
> >>    stubdoms. I haven't tried those yet.
> >>>    But aside from that, I'm stumped. Are you running any
> firewalls on
> >>>    either side?   Can you try disabling all the offloads (tx,
> rx, gso, tso)
> >>>    on all the relevent interfaces (bridge, netback, within the
> guest) and
> >>>    see if that changes anything?
> >>>
> >>>    J
> >>>
> >>>
> >>
> >>    Ok, this is the bridge interface:
> >>
> >>    brctl show
> >>    bridge name bridge id STP enabled interfaces
> >>    virbr0 8000.feffffffffff no vif1.0
> >>
> >>    ifconfig virbr0
> >>    virbr0 Link encap:Ethernet HWaddr c2:ef:67:2b:a4:23
> >>    inet addr:192.168.120.254 Bcast:192.168.120.255
> Mask:255.255.255.0
> >>    inet6 addr: fe80::c0ef:67ff:fe2b:a423/64 Scope:Link
> >>    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >>    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >>    TX packets:25 errors:0 dropped:0 overruns:0
> carrier:0
> >>    collisions:0 txqueuelen:0
> >>    RX bytes:0 (0.0
> >> B)
> >> TX bytes:4662 (4.6 KB)
> >>
> >>
> >>
> >>    I'm not using firewall other than the rules defined by
> >>    libvirt. DomU has no firewall and the rules in dom0 are only
> >>    these (virbr0 is natted to the outside, virbr1 is routed. The
> >>    result is the same in either one of them):
> >>
> >>    sudo iptables -L -n -v
> >>    Chain INPUT (policy ACCEPT 241K packets, 53M bytes)
> >> pkts bytes target    prot opt in    out    source
>    destination
> >>    0    0 ACCEPT    udp -- virbr1 *    0.0.0.0/0
> 0.0.0.0/0    udp dpt:53
> >>    0    0 ACCEPT    tcp -- virbr1 *    0.0.0.0/0
> 0.0.0.0/0    tcp dpt:53
> >>
> >>
> >>    0    0 ACCEPT    udp -- virbr1 *    0.0.0.0/0
> 0.0.0.0/0    udp dpt:67
> >>    0    0 ACCEPT    tcp -- virbr1 *    0.0.0.0/0
> 0.0.0.0/0    tcp dpt:67
> >>    8   515 ACCEPT    udp -- virbr0 *    0.0.0.0/0
> 0.0.0.0/0    udp dpt:53
> >>    0    0
> >>
> >> ACCEPT    tcp -- virbr0 *    0.0.0.0/0
> 0.0.0.0/0    tcp dpt:53
> >>    0    0 ACCEPT    udp -- virbr0 *    0.0.0.0/0
> 0.0.0.0/0    udp dpt:67
> >>    0    0 ACCEPT    tcp -- virbr0 *    0.0.0.0/0
> 0.0.0.0/0    tcp dpt:67
> >>
> >>    Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> >> pkts bytes target
> >> prot
> >> opt in    out    source    destination
> >>    0    0 ACCEPT    all -- * virbr1 0.0.0.0/0
> 192.168.121.0/24
> >>    0    0 ACCEPT    all -- virbr1 *
>    192.168.121.0/24    0.0.0.0/0
> >>    0    0 ACCEPT    all -- virbr1 virbr1 0.0.0.0/0
>
> >>
> >> 0.0.0.0/0
> >>    0    0 REJECT    all -- * virbr1 0.0.0.0/0
> 0.0.0.0/0    reject-with icmp-port-unreachable
> >>    0    0 REJECT    all -- virbr1 *    0.0.0.0/0
> 0.0.0.0/0    reject-with icmp-port-unreachable
> >> 13 3448 ACCEPT    all -- * virbr0 0.0.0.0/0
> 192.168.120.0/24
> >> state
> >> RELATED,ESTABLISHED
> >> 16 1374 ACCEPT    all -- virbr0 *
>    192.168.120.0/24    0.0.0.0/0
> >>    0    0 ACCEPT    all -- virbr0 virbr0 0.0.0.0/0
> 0.0.0.0/0
> >>    0    0 REJECT    all -- * virbr0 0.0.0.0/0
> 0.0.0.0/0    reject-with icmp-port-unreachable
> >>    0    0 REJECT    all --
> >> virbr0
> >> *    0.0.0.0/0 0.0.0.0/0    reject-with
> icmp-port-unreachable
> >>
> >>    Chain OUTPUT (policy ACCEPT 233K packets, 27M bytes)
> >> pkts bytes target    prot opt in    out    source
>    destination
> >>
> >>
> >>
> >>
> >>    And these are the various offload parameters as set at boot:
> >>
> >>    Offload parameters for virbr0:
> >>    rx-checksumming: on
> >>    tx-checksumming: on
> >>    scatter-gather: on
> >>    tcp-segmentation-offload: on
> >>    udp-fragmentation-offload: on
> >>    generic-segmentation-offload: on
> >>    generic-receive-offload: off
> >>    large-receive-offload: off
> >>
> >>    Offload parameters for vif1.0:
> >>    rx-checksumming: on
> >>    tx-checksumming: on
> >>    scatter-gather: on
> >>    tcp-segmentation-offload: on
> >>    udp-fragmentation-offload: off
> >>    generic-segmentation-offload: on
> >>    generic-receive-offload: off
> >>    large-receive-offload: off
> >>
> >>    Offload parameters for eth0:
> >>    rx-checksumming: on
> >>    tx-checksumming: on
> >>    scatter-gather: on
> >>    tcp-segmentation-offload: on
> >>    udp-fragmentation-offload: off
> >>    generic-segmentation-offload: off
> >>    generic-receive-offload: off
> >>    large-receive-offload: off
> >>
> >>
> >>
> >>    To disable all checksuming I run the following commands:
> >>    dom0:
> >>
> >>    sudo ethtool -K virbr0 tx off sg off tso off gso off gro off
> >>    sudo ethtool -K vif1.0 tx off sg off tso off gso off gro off
> >>
> >>
> >>    domU
> >>
> >>    sudo ethtool -K eth0 tx off sg off tso off gso off gro off
> >>
> >>
> >>
> >>    This managed to get all parameter to off in the mentioned
> >>    interfaces, but unfortunately the result is the same. The arp
> >>    requests get to vif1.0, but not to eth0 on the domU.
> >>
> >>    sudo tcpdump -i vif1.0 -n -vv arp
> >>    tcpdump: WARNING: vif1.0: no IPv4 address assigned
> >>    tcpdump: listening on vif1.0, link-type EN10MB (Ethernet),
> capture size 96 bytes
> >>    19:43:51.233378 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>    19:43:52.233164 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>    19:43:53.233166 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>    19:43:54.684214 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>    19:43:55.684218 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>    19:43:56.684232 ARP, Ethernet (len 6), IPv4 (len 4), Request
> who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>
> >>
> >>
> >>    I hope this information is enough. If I can provide anything
> >>    else to help debug or test, please just ask! ;)
> >>
> >>    Thanks in advance,
> >>    Luís
> >>
> >>>    >
> >>>    > Thanks,
> >>>    > Luís
> >>>    >
> >>>    > On Tue, 2010-06-01 at 18:20 -0700, Jeremy Fitzhardinge
> wrote:
> >>>    >> On 06/01/2010 05:38 PM, Luís Silva wrote:
> >>>    >> > Hello,
> >>>    >> >
> >>>    >> > Finally I managed to get a xen 4.0 working on ubuntu
> 10.04 with pvops
> >>>    >> > kernel and libvirt. However I am having some problems
> with
> >>>    >> > networking... after initial installation with
> netinstall image in hvm
> >>>    >> > mode, when I transform the vm in xen pv (via pygrub
> with the current
> >>>    >> > ubuntu kernel), networking startEd to act weird...
> >>>    >> >
> >>>    >> > Basically I'm not using a network script from xen. I
> define a bridge
> >>>    >> > (manually or via libvirt, the result is the same) and I
> use vif-bridge
> >>>    >> > to connect the vif to it. But now the weird part comes:
> I can
> >>>    >> > communicate from domU to dom0, but not the other way
> >>> around,
> >>> unless I
> >>>    >> > keep a ping running from domU to dom0... That's right,
> weird... while
> >>>    >> > trying the ping from dom0 to domU, I used tcpdump both
> on the bridge,
> >>>    >> > on the vif and on the eth0 in the domU. The arp packets
> never get to
> >>>    >> > domU, but they appear both in the bridge and the vif
> sniff's...
> >>>    >>
> >>>    >> What version of kernel are you using in dom0 and domU?
> There was a
> >>>    >> netback bug which caused problems with dom0<->domU
> communication, but it
> >>>    >> has been fixed for a while in 2.6.32 (but only recently
> in .31). The
> >>>    >> workaround is to disable tx checksum offload on your
> bridge (ethtool -K
> >>>    >> <bridge> tx off).
> >>>    >>
> >>>    >>    J
> >>>    >>
> >>>    >> >
> >>>    >> > Here is the bridge:
> >>>    >> > ifconfig virbr0
> >>>    >> > virbr0 Link encap:Ethernet HWaddr
> fe:ff:ff:ff:ff:ff
> >>>    >> >
> >>>
> >>> inet addr:192.168.120.254 Bcast:192.168.120.255
> Mask:255.255.255.0
> >>>    >> >    inet6 addr: fe80::7cee:4bff:fe82:e63f/64
> Scope:Link
> >>>    >> >    UP BROADCAST RUNNING MULTICAST MTU:1500
> Metric:1
> >>>    >> >    RX packets:16 errors:0 dropped:0 overruns:0
> frame:0
> >>>    >> >    TX packets:226 errors:0 dropped:0 overruns:0
> carrier:0
> >>>    >> >    collisions:0 txqueuelen:0
> >>>    >> >    RX bytes:952 (952.0 B) TX bytes:13953 (13.9
> KB)
> >>>    >> >
> >>>    >> >
> >>>    >> > brctl show
> >>>    >> > bridge name bridge id STP enabled
> interfaces
> >>>    >> > virbr0 8000.feffffffffff no vif5.0
> >>>    >> >
> >>>    >> >
> >>>    >> > tcpdump -i virbr0 -vv -n
> >>>    >> > tcpdump: listening on virbr0, link-type EN10MB
> (Ethernet), capture size 96 bytes
> >>>    >> > 01:31:25.945151 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF],
> >>> proto ICMP (1),
> >>> length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 1, length 64
> >>>    >> > 01:31:26.945361 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 2, length 64
> >>>    >> > 01:31:27.945420 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 3, length 64
> >>>    >> > 01:31:28.945362 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 4, length 64
> >>>    >> > 01:31:29.945364 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317,
> >>> seq 5, length
> >>> 64
> >>>    >> > 01:31:30.944300 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:30.945359 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 6, length 64
> >>>    >> > 01:31:31.944297 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:31.945444 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >    192.168.120.254 > 192.168.120.1: ICMP echo request,
> id 10317, seq 7, length 64
> >>>    >> > 01:31:32.944294 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:32.945401 IP (tos 0x0, ttl 64, id 0, offset 0,
> flags [DF], proto ICMP (1), length 84)
> >>>    >> >
> >>>
> >>> 192.168.120.254 > 192.168.120.1: ICMP echo request, id
> 10317, seq 8, length 64
> >>>    >> > 01:31:33.947293 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:34.947373 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:35.947353 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:37.948352 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:38.948399 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:39.948376 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:31:40.949356 ARP, Ethernet (len 6), IPv4 (len 4),
> Request
> >>> who-has
> >>> 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> >
> >>>    >> >
> >>>    >> > tcpdump -i vif5.0 -vv -n
> >>>    >> > tcpdump: WARNING: vif5.0: no IPv4 address assigned
> >>>    >> > tcpdump: listening on vif5.0, link-type EN10MB
> (Ethernet), capture size 96 bytes
> >>>    >> > 01:32:19.956358 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:20.956358 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:21.956359 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:23.957311 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:24.957312 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length
> >>> 28
> >>>    >> >
> >>> 01:32:25.957359 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:27.958360 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:28.958310 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> > 01:32:29.958362 ARP, Ethernet (len 6), IPv4 (len 4),
> Request who-has 192.168.120.1 tell 192.168.120.254, length 28
> >>>    >> >
> >>>    >> >
> >>>    >> >
> >>>    >> > Forwarding and iptables don't seem to be the problem,
> because if I
> >>>    >> > initiate a ping from domU (at the same time as the
> failing one from
> >>>    >> > dom0), the ping in dom0 starts to work. As soon as I
> stop the ping in
> >>>    >> > domU, the one in dom0 starts failing again...
> >>>    >> >
> >>>    >> > Is anyone having the same
> >>> problem? Is this a bug
> >>> in the kernel? In
> >>>    >> > dom0 or domU?
> >>>    >> >
> >>>    >> > Thanks in advance,
> >>>    >> > Luís
> >>>    >> >
> >>>    >> >
> >>>    >> > _______________________________________________
> >>>    >> > Xen-devel mailing list
> >>>    >> > [12]Xen-devel@xxxxxxxxxxxxxxxxxxx
> <mailto:[13]Xen-devel@xxxxxxxxxxxxxxxxxxx>
> <mailto:[14]Xen-devel@xxxxxxxxxxxxxxxxxxx>
> >>>    >> > [15]http://lists.xensource.com/xen-devel
> >>>    >> >
> >>>    >>
> >>>    >>
> >>>    >> _______________________________________________
> >>>    >> Xen-devel mailing list
> >>>    >> [16]Xen-devel@xxxxxxxxxxxxxxxxxxx
> <mailto:[17]Xen-devel@xxxxxxxxxxxxxxxxxxx>
> <mailto:[18]Xen-devel@xxxxxxxxxxxxxxxxxxx>
> >>>    >> [19]http://lists.xensource.com/xen-devel
> >>>    >>
> >>>    >>
> >>>    >
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>    -----Inline Attachment Follows-----
> >>
> >>    _______________________________________________
> >>    Xen-users mailing list
> >>    [20]Xen-users@xxxxxxxxxxxxxxxxxxx
> >>    [21]http://lists.xensource.com/xen-users
> >>
> >>
> >
> >
> >    -----Inline Attachment Follows-----
> >
> >    _______________________________________________
> >    Xen-users mailing list
> >    [22]Xen-users@xxxxxxxxxxxxxxxxxxx
> >    </mc/compose?to=[23]Xen-users@xxxxxxxxxxxxxxxxxxx>
> >    [24]http://lists.xensource.com/xen-users
> >
> >
>
> References
>
> Visible links
> 1. file:///mc/compose?to=luis.silva@xxxxxxxxxxxxx
> 2. file:///mc/compose?to=luis.silva@xxxxxxxxxxxxx
> 3. file:///mc/compose?to=bderzhavets@xxxxxxxxx
> 4. file:///mc/compose?to=jeremy@xxxxxxxx
> 5. file:///mc/compose?to=xen-devel@xxxxxxxxxxxxxxxxxxx
> 6. file:///mc/compose?to=xen-users@xxxxxxxxxxxxxxxxxxx
> 7. file:///mc/compose?to=luis.silva@xxxxxxxxxxxxx
> 8. file:///mc/compose?to=luis.silva@xxxxxxxxxxxxx
> 9. file:///mc/compose?to=jeremy@xxxxxxxx
>   10. file:///mc/compose?to=xen-devel@xxxxxxxxxxxxxxxxxxx
>   11. file:///mc/compose?to=xen-users@xxxxxxxxxxxxxxxxxxx
>   12. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   13. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   14. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   15. http://lists.xensource.com/xen-devel
>   16. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   17. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   18. file:///mc/compose?to=Xen-devel@xxxxxxxxxxxxxxxxxxx
>   19. http://lists.xensource.com/xen-devel
>   20. file:///mc/compose?to=Xen-users@xxxxxxxxxxxxxxxxxxx
>   21. http://lists.xensource.com/xen-users
>   22. file:///mc/compose?to=Xen-users@xxxxxxxxxxxxxxxxxxx
>   23. file:///mc/compose?to=Xen-users@xxxxxxxxxxxxxxxxxxx
>   24. http://lists.xensource.com/xen-users

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

Re: [Xen-users] Re: [Xen-devel] ARP problems with xen 4.0 with pvops kernel 2.6.32.15