[Top] [All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-users] Remus dom0 network buffering

Hi Shriram,

Thank you for reply, I am using  Xen version: 4.2.2-pre compiled from sources with kernel 3.2.0-29-generic

~$ dmesg | grep Xen\ version
[    0.000000] Xen version: 4.2.2-pre (preserve-AD)

~$ uname -a
Linux comp1 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

I have try to do simple measurement with ping command but still didn't got satisfying result.

Ping domU no migration :
C:\Users\rdc>ping 192.168.129.55 -t

Pinging 192.168.129.55 with 32 bytes of data:
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=2ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=3ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=4ms TTL=64

Ping domU with migration but without network bufering :
C:\Users\rdc>ping 192.168.129.55 -t

Pinging 192.168.129.55 with 32 bytes of data:
Reply from 192.168.129.55: bytes=32 time=7ms TTL=64
Reply from 192.168.129.55: bytes=32 time=8ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=11ms TTL=64
Reply from 192.168.129.55: bytes=32 time=3ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=3ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=3ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=2ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=2ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=12ms TTL=64


Ping domU with migration and network bufering :
C:\Users\rdc>ping 192.168.129.55 -t

Pinging 192.168.129.55 with 32 bytes of data:
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=6ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=4ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time=1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64
Reply from 192.168.129.55: bytes=32 time<1ms TTL=64

Here some sampe of remus output (with network buffering) :
user1@comp1:~$ sudo remus 1 10.10.10.3
WARNING: suspend event channel unavailable, falling back to slow xenstore signalling
PROF: suspending at 1362068953.644729
PROF: resumed at 1362068953.658836
PROF: flushed memory at 1362068953.669453
PROF: suspending at 1362068953.854217
PROF: resumed at 1362068953.868208
PROF: flushed memory at 1362068953.877483
PROF: suspending at 1362068954.063650
PROF: resumed at 1362068954.08

Could you point out something?

Regards,

Agya


On Thu, Feb 28, 2013 at 3:53 PM, Shriram Rajagopalan <rshriram@xxxxxxxxx> wrote:
start with a simple test. Ping command.
Remember that suspend/resume also takes some time away from the VM.
If you are using the slow version of suspend/resume, then that alone could take
10ms or more (to simply suspend/resume).

I am not sure if you installed remus from the distribution repo
(ubuntu or whatever)
or if you compiled from source. If you used a repo version, then you
might have to
write a script to parse the remus output (PROF: suspend at ..) to see
how long a VM remains
suspended.


On Thu, Feb 28, 2013 at 7:24 AM, agya naila <agya.naila@xxxxxxxxx> wrote:
> Here my lsmod,
>
>  #lsmod
> Module                  Size  Used by
> xt_tcpudp              12603  1
> xt_physdev             12587  4
> iptable_filter         12810  1
> ip_tables              27473  1 iptable_filter
> x_tables               29846  4
> xt_tcpudp,xt_physdev,iptable_filter,ip_tables
> drbd                  303888  4
> blktap                 25553  0
> xen_pciback            48131  0
> xen_netback            27854  0 [permanent]
> xen_blkback            23363  0 [permanent]
> xen_gntalloc           13321  0
> xen_gntdev             17676  1
> xen_evtchn             13172  2
> xenfs                  18311  1
> bridge                 90989  0
> stp                    12931  1 bridge
> dm_multipath           23230  0
> radeon                804372  2
> joydev                 17693  0
> ttm                    76949  1 radeon
> drm_kms_helper         46978  1 radeon
> drm                   242038  4 radeon,ttm,drm_kms_helper
> i2c_algo_bit           13423  1 radeon
> bnep                   18281  2
> parport_pc             32866  0
> shpchp                 37277  0
> bluetooth             180104  7 bnep
> ppdev                  17113  0
> mac_hid                13253  0
> e752x_edac             18564  0
> edac_core              53746  3 e752x_edac
> ifb                    12896  0
> act_mirred             13038  0
> cls_u32                17060  0
> cls_tcindex            13058  0
> cls_basic              12946  0
> sch_ingress            12866  0
> sch_prio               13231  0
> sch_plug               12654  0
> lp                     17799  0
> parport                46562  3 parport_pc,ppdev,lp
> dm_snapshot            39031  3
> usbhid                 47199  0
> hid                    99559  1 usbhid
> usb_storage            49198  0
> mptspi                 22921  21
> mptscsih               44882  1 mptspi
> mptbase               103162  2 mptspi,mptscsih
> tg3                   152032  0
>
>
> Regards,
>
> Agya
>
> On Thu, Feb 28, 2013 at 12:29 PM, agya naila <agya.naila@xxxxxxxxx> wrote:
>>
>> Hi Andrew and Shiram,
>>
>> I am keep trying this setting using my machine that I mention earlier in
>> this email thread.
>> I have using remus with network buffering and no network buffering.
>>
>> with no network buffering
>> #remus --no-net <vm> <backup target>
>>
>> with network buffering
>> #remus <vm> <backup target>
>>
>> However, when I observed the jitter graph result of my data it seem there
>> is no buffering impact on my traffic flow, enclosed both the graph result. I
>> only observed RTP packet and observed the jitter. What I could understand
>> the graph with network protection will doesn't have flow until x time when
>> the buffer released, in this case default value 100ms. Y axis is time and X
>> axis is jitter.
>>
>> Both of command above running without error except  suspend event channel
>> issue that we discussed earlier. Please suggest.
>>
>> Regards,
>>
>> Agya
>>
>> On Wed, Feb 20, 2013 at 1:13 PM, agya naila <agya.naila@xxxxxxxxx> wrote:
>>>
>>> Aha, sorry to mix it :), its clear now.
>>>
>>> Regards,
>>>
>>> Agya
>>>
>>>
>>> On Wed, Feb 20, 2013 at 1:01 PM, Andrew Eross <eross@xxxxxxxxxxxx> wrote:
>>>>
>>>> Hi Agya,
>>>>
>>>> This patch doesn't fix the 'suspend event channel' issue, it just fixes
>>>> the crash from the first e-mail in this thread.
>>>>
>>>> Please see this wiki page:
>>>> http://wiki.xen.org/wiki/Remus_PV_domU_requirements
>>>>
>>>> That discusses the suspend event channel issue.
>>>>
>>>> Cheers,
>>>> Andrew
>>>>
>>>>
>>>> On Wed, Feb 20, 2013 at 8:57 AM, agya naila <agya.naila@xxxxxxxxx>
>>>> wrote:
>>>>>
>>>>> Hello All,
>>>>>
>>>>> I am trying this patch on my machine Ubuntu 12.04 AMD64 both for Dom0
>>>>> and DomU with Xen 4.2.2-pre compiling from sources also DRBD 8.3.11 but
>>>>> unfortunately it seem didn't work since I still got the message WARNING:
>>>>> suspend event channel unavailable, falling back to slow xenstore signalling.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Agya
>>>>>
>>>>> On Tue, Feb 19, 2013 at 12:39 PM, Andrew Eross <eross@xxxxxxxxxxxx>
>>>>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> To close off on this, Shriram's patch worked great, thank you.
>>>>>>
>>>>>> So Ubuntu 12.10's standard generic kernel is now working fine as a
>>>>>> dom0 for Remus with Xen 4.1.4 (with the patch from this thread).
>>>>>>
>>>>>> I wrote up my full install instructions here:
>>>>>>
>>>>>> http://wiki.xen.org/wiki/Install_Xen_4.1.4_with_Remus_and_DRBD_on_Ubuntu_12.10
>>>>>>
>>>>>> I also took a shot at a number of updates to:
>>>>>> http://wiki.xen.org/wiki/Remus
>>>>>>
>>>>>> And added these pages:
>>>>>> http://wiki.xen.org/wiki/Remus_PV_domU_requirements
>>>>>> http://wiki.xen.org/wiki/Remus_dom0_requirements
>>>>>>
>>>>>> Feedback very welcome, a lot of that information was a bit old, so I
>>>>>> did my best.
>>>>>>
>>>>>> Cheers,
>>>>>> Andrew
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 18, 2013 at 8:26 PM, Shriram Rajagopalan
>>>>>> <rshriram@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>> ah darn. Yes, the 3.5 kernel has sch_plug in it. Unfortunately, the
>>>>>>> technique to initialize the sch_plug module is bit
>>>>>>> different with the module thats already mainline. You ll have to
>>>>>>> patch
>>>>>>> the qdisc.py python module. I ll submit the
>>>>>>> patch to xen devel soon.
>>>>>>> Mean while, can you try the patch below and see if it works ?
>>>>>>>
>>>>>>> Secondly, you said you were writing a blog about how you set it up.
>>>>>>> One of the remus users created a wiki
>>>>>>> remusha.wikidot.com, which has a whole tutorial on how to setup remus
>>>>>>> with Debian Squeeze. I have been under fire
>>>>>>> lately for a host of remus issues. It would be great if you could
>>>>>>> post
>>>>>>> your experiences on the wiki too.
>>>>>>> (http://remusha.wikidot.com/system:join)
>>>>>>>
>>>>>>> thanks
>>>>>>> shriram
>>>>>>>
>>>>>>> ---
>>>>>>> diff -r 6435f1edf24e tools/python/xen/remus/qdisc.py
>>>>>>> --- a/tools/python/xen/remus/qdisc.py   Tue Jul 17 19:23:30 2012
>>>>>>> -0400
>>>>>>> +++ b/tools/python/xen/remus/qdisc.py   Mon Feb 18 18:25:22 2013
>>>>>>> -0500
>>>>>>> @@ -1,6 +1,9 @@
>>>>>>>  import socket, struct
>>>>>>>
>>>>>>>  import netlink
>>>>>>> +import platform
>>>>>>> +
>>>>>>> +kernelversion =
>>>>>>> platform.platform(terse=True).split("-")[1].split(".")
>>>>>>>
>>>>>>>  qdisc_kinds = {}
>>>>>>>
>>>>>>> @@ -150,9 +153,14 @@
>>>>>>>  TC_PLUG_RELEASE = 1
>>>>>>>
>>>>>>>  class PlugQdisc(Qdisc):
>>>>>>> -    fmt = 'I'
>>>>>>>
>>>>>>>      def __init__(self, qdict=None):
>>>>>>> +        if int(kernelversion[0]) >= 3 and int(kernelversion[1]) >=
>>>>>>> 4:
>>>>>>> +            self.fmt = 'iI'
>>>>>>> +            self.limit = 10000
>>>>>>> +        else:
>>>>>>> +            self.fmt = 'I'
>>>>>>> +
>>>>>>>          if not qdict:
>>>>>>>              qdict = {'kind': 'plug',
>>>>>>>                       'handle': TC_H_ROOT}
>>>>>>> @@ -161,7 +169,10 @@
>>>>>>>          self.action = ""> >>>>>>>
>>>>>>>      def pack(self):
>>>>>>> -        return struct.pack(self.fmt, self.action)
>>>>>>> +        if int(kernelversion[0]) >= 3 and int(kernelversion[1]) >=
>>>>>>> 4:
>>>>>>> +            return struct.pack(self.fmt, self.action, self.limit)
>>>>>>> +        else:
>>>>>>> +            return struct.pack(self.fmt, self.action)
>>>>>>>
>>>>>>>      def parse(self, args):
>>>>>>>          if not args:
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Xen-users mailing list
>>>>>> Xen-users@xxxxxxxxxxxxx
>>>>>> http://lists.xen.org/xen-users
>>>>>
>>>>>
>>>>
>>>
>>
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.

Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.