Re: [Xen-devel] Migrate/Save of 32-bit domU Broken on Xen 3.1.2 64-bit dom0

[Joshua West <jwest@xxxxxxxxxxxx>]

Hi Dan,

Dan Magenheimer wrote:
> Joshua --
>
> Why do you need so much memory on dom0?  Are you thinking
> that the total memory available to guests is limited to
> what dom0 has?  Or are you running some memory-hungry
> applications on dom0?
>   

Hmm, I thought that a 32-bit dom0 system can only have up to 16GB of 
memory available to it and its domU's -- and a 64-bit dom0 can have a 
max of 32GB.  I only allocate ~512MB of memory to the dom0's 
themselves.  "xm info" shows us the rest of whats available for use by 
domU's.

Thus, even if I have 32GB of RAM in a server, the total memory that will 
be used in dom0 + all domU's can only add up to 16GB.  Am I just plain 
old wrong?

> Note that 64-bit Xen + 32-bit Dom0 will support both 32-bit
> and 64-bit domU's. (Oracle VM uses this configuration by
> default on 64-bit capable machines.)  And save/restore/migration
> has been tested to work for this (with the patch Kurt posted).
>   

Interesting.  For some reason I thought that the Hypervisor had to match 
the dom0's kernel.  So you're saying that if I compile the Xen 
Hypervisor as 64-bit and my dom0 + domU's both as 32-bit, I can reach 
the full capacity of my servers with respect to RAM (32GB) instead of 
being limited to 16GB?  (And this only works with Kurt's patches?).

> Dan
>
>   
> > -----Original Message-----
> > From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of Joshua West
> > Sent: Friday, January 18, 2008 4:06 PM
> > To: Keir Fraser
> > Cc: Kurt Hackel; xen-devel@xxxxxxxxxxxxxxxxxxx
> > Subject: Re: [Xen-devel] Migrate/Save of 32-bit domU Broken 
> > on Xen 3.1.2
> > 64-bit dom0
> >
> >
> > Dang, doesn't work in Xen 3.1.3-rc1 (like you said/expected), with or
> > without Kurt's patches.
> >
> > Hmm... I thought 32-bit paravirtualized domU's on 64-bit dom0's were
> > supported in Xen 3.1.x?  Or am I mistaken?
> >
> > Perhaps standard features such as start/shutdown/pause of 32-bit PV
> > domU's on 64-bit dom0's are supported, but not the more advanced
> > management features like save and migrate - and I was just 
> > unclear from
> > the beginning?
> >
> > And of course... the final question that all developers hate... when
> > s/could one _hope_ (not expect!) to see save/migrate functionality of
> > 32-bit PV domU's on 64-bit dom0's? :-D
> >
> > Our whole reason for moving to 64-bit dom0's was to utilize 32GB of
> > memory, instead of the 16GB limitation on 32-bit PAE dom0's.  We can't
> > convert all of our virtual machines (100+) to 64-bit linux as its too
> > much work and some actually have to be 32-bit due to vendor supplied
> > software (Oracle, Zimbra, etc).  I suppose we'll wait to move 
> > to 64-bit
> > until [live] migration is available, as its used quite 
> > frequently in our
> > machine rooms.
> >
> > Thanks for all of your help Keir and Kurt.
> >
> > Keir Fraser wrote:
> >     
> > > Looking at the issues that need to be handled to support 
> > >       
> > this, it cannot be
> >     
> > > expected to work.
> > >
> > >  -- Keir
> > >
> > > On 18/1/08 22:13, "Joshua West" <jwest@xxxxxxxxxxxx> wrote:
> > >
> > >
> > >       
> > > > Ahh.  Well, I'm building xen-3.1.3-rc1 right now and I'll 
> > > >         
> > see how that goes.
> >     
> > > > Keir, have you seen this issue before?  Where one cannot 
> > > >         
> > xm save/migrate
> >     
> > > > a 32-bit domU on a 64-bit dom0?  Or is this expected to 
> > > >         
> > work and just
> >     
> > > > doesn't go well for me?
> > > >
> > > > Keir Fraser wrote:
> > > >
> > > >         
> > > > > On 18/1/08 22:05, "Joshua West" <jwest@xxxxxxxxxxxx> wrote:
> > > > >
> > > > >
> > > > >
> > > > >           
> > > > > > Just to make sure the patches aren't the cause of this new error
> > > > > > message, I'm going to see what happens with Xen 
> > > > > >             
> > 3.1.3-rc1 w/o patches.
> >     
> > > > > > Kurt, Keir, everybody else... any ideas as to why "xm 
> > > > > >             
> > save" (and thus
> >     
> > > > > > "xm migrate") of 32-bit domU's does not work on 64-bit 
> > > > > >             
> > dom0's?  Has
> >     
> > > > > > anybody found this to be successful in the past?
> > > > > >
> > > > > >
> > > > > >             
> > > > > The extent of the patches that Kurt posted indicate its 
> > > > >           
> > pretty unlikely to
> >     
> > > > > work without considerable code modification. I guess 
> > > > >           
> > those patches aren't
> >     
> > > > > quite fully baked yet, unfortunately, or haven't been 
> > > > >           
> > written with your
> >     
> > > > > precise case in mind.
> > > > >
> > > > >  -- Keir
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >           
> > >
> > >       
> > --
> > Joshua West
> > Systems Engineer
> > Brandeis University
> > http://www.brandeis.edu
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> >
> >     
>
>   


--
Joshua West
Systems Engineer
Brandeis University
http://www.brandeis.edu


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel