[Xen-devel] Re: xenpaing: one way to avoid paging out the page, when the corresponding mfn is in use.

[andres@xxxxxxxxxxxxxxxx]

Page_info structs are carefully laid out to occupy 64 bytes (I think -- or
some other power of two cache-line-friendly thing).

Were you thinking of this new field to be part of a union? which one?
Otherwise you change the size of page_info and hell breaks loose.

The RFC patches I posted a week ago (which are not ready for primetime)
try to ensure serialization of lookups and modification of p2m entries.
That should solve the issue of paging a foreign-mapped page.

Actually the patches attempt to achieve what you state ("When the page
(mfn) is accessed by gfn_to_mfn(), this page should never be paged out
until the mapping action is end"). And hopefully in a much more general
way.

So, stay posted...

Andres

> Date: Tue, 01 Nov 2011 12:23:37 +0000
> From: Hongkaixing <hongkaixing@xxxxxxxxxx>
> Subject: [Xen-devel] xenpaing: one way to avoid paging out the page,
>       when the corresponding mfn is in use.
> To: "olaf@xxxxxxxxx" <olaf@xxxxxxxxx>
> Cc: YangXiaowei <xiaowei.yang@xxxxxxxxxx>,
>       "Xen-devel@xxxxxxxxxxxxxxxxxxx" <Xen-devel@xxxxxxxxxxxxxxxxxxx>,        
> "Eric
>       Li\(Zhentao\)" <lizhentao@xxxxxxxxxx>,  Yanqiangjun
>       <yanqiangjun@xxxxxxxxxx>, hanweidong <hanweidong@xxxxxxxxxx>
> Message-ID:
>       
> <E0AF011477D2AE458DC8801E0E570709014C0556@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
>
> Content-Type: text/plain; charset=Windows-1252
>
> Hello,
>
>    Recently many advanced memory mechanisms are introduced into Xen. One
> problem we found is the conflict between p2m query and setting.
>    For example, backend drivers always map domU?s page to its own space,
> during the mapping procedure, situations as follow may happen,
> when mfn is obtained by gfn_to_mfn(), this mfn is likely to be paged out.
>
> first case:
>    grant mapping                      xenpaing
>   mfn = gfn_to_mfn();
>                        <-----------  p2m_paging_nominate()
>          |                                         |
>      Check type ok                     paged out;
>          |
>      try to map
> What we want is:
> When the page (mfn) is accessed by gfn_to_mfn(), this page should never be
> paged out until the mapping action is end.
>
> second case:
>    grant mapping                            xenpaing
>                                          p2m_paging_nominate()
>
>                                                  gfn_to_mfn();
>   mfn = gfn_to_mfn();  ------------->     |
>                                              check type ok
>          |                                               |
>      Check type ok                     paged out;
>          |
>      try to map
> What we want is:
> When the gfn_to_mfn() action happens during paging nomination, the
> nomination should abort immediately.
>
> Our solution prototype is like this :
> 1. Introduce a new member named last_access in page_info struct to save
> the last access time and access tag.
> 2. when the mfn is obtained through gfn_to_mfn(), we save time stamp and
> access tag in the page_info.
> 3. Paging nominate procedure use access information as a criterion.
>
> How it works?
> 1.Using time stamp to avoid case 1. When the mfn is obtained by mapping
> process,
>    the time stamp can prevent the page from being selected by paging .
> 2.Using access tag to avoid case 2. During the paging nomination, if the
> access tag of page is detected,
>    paging should skip selecting this page.
>
> The pseudo-code of step 3 can be written as follow:
> int p2m_mem_paging_nominate(struct domain *d, unsigned long gfn)
> {
>
>     mfn = gfn_to_mfn_noreference(d, gfn, &p2mt);   -----> avoid saving
> timestamp and access tag
>
>     if ( !mfn_valid(mfn) )
>         goto out;
>
>     clear_access_tag();   ----------> clear the access tag of this page
>     if (test_page_hot())
>        goto out;           ------> if the page is accessed recently, go to
> out
>     ........
>
>     set_p2m_entry(d, gfn, mfn, 0, p2m_ram_paging_out);
>     if ( test_access_tag ( mfn ) )
>         goto out;  --------> if access tag is set, the gfn_to_mfn must
> have happened above, abort anyway.
>     ret = 0;
>  out:
>     p2m_unlock(d->arch.p2m);
>     return ret;
> }
>    Maybe this is an imperfect prototype, do you have any good ideas?
>
>   Hong Kaixing
>
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 01 Nov 2011 13:47:15 +0100
> From: Tobias Heinlein <heinlein@xxxxxxx>
> Subject: Re: [Xen-devel] Re: Xen dom0 linux kernel 3.1 boot failure
>       ptwr_emulate: could not get_page_from_l1e
> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> Cc: 2013pfoley@xxxxxxxxxx
> Message-ID: <4EAFEA53.7070009@xxxxxxx>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I'm not sure if it was obvious, but yesterday I noticed that setting the
> "MPS table mode" to 'Disabled' actually made SMP stop working, i.e. the
> kernel only recognized a single CPU. This is of course not an option, so
> I enabled (set to 'Full Table APIC') the setting again and played around
> with my kernel config a bit. The kernel that crashed had
> CONFIG_X86_MPPARSE=y, and if I disable that, it boots fine (with SMP,
> and with the BIOS setting set to 'Full Table APIC').
>
> So, I for one am quite happy now as I finally found a working
> configuration. But I'd still like to know if this is a hardware-specific
> issue, and/or a bug in Xen.
>
> Konrad Rzeszutek Wilk wrote, on 10/31/2011 03:08 PM:
>> Oh nice. What does you /proc/interrupts look like compared to
>> baremetal?
>
> While I was performing all my kernel tests, I saved the outputs of
> `dmesg` and `cat /proc/interrupts`. Sorry for attaching a tarball, but
> I'd like to give you as much information as possible. You'll probably
> only need the latest tests (#5 to #7), but just in case, I also included
> the others.
>
> Contents of the tarball:
>
> Baremetal tests:
> xen-hp/1/: MPS mode 'Full Table APIC', CONFIG_X86_MPPARSE=y, SMP working
> xen-hp/2/: MPS mode 'Full Table APIC', CONFIG_X86_MPPARSE=n, SMP working
> xen-hp/3/: MPS mode 'Disabled', CONFIG_X86_MPPARSE=y, SMP not working
> xen-hp/4/: MPS mode 'Disabled', CONFIG_X86_MPPARSE=n, SMP not working
>
> Xen tests:
> xen-hp/5/: MPS mode 'Disabled', CONFIG_X86_MPPARSE=n, SMP not working
> xen-hp/6/: MPS mode 'Full Table APIC', CONFIG_X86_MPPARSE=n, SMP working
> xen-hp/7/: MPS mode 'Full Table APIC', CONFIG_X86_MPPARSE=y, CRASHES
>
> (Therefore, #6 is the best working solution; #7 is what originally
> triggered the crash.)
>
> Thanks.
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: xen-hp.tar.bz2
> Type: application/x-bzip
> Size: 97821 bytes
> Desc: not available
> Url :
> http://lists.xensource.com/archives/html/xen-devel/attachments/20111101/4f96b9e7/xen-hp.tar.bin
>
> ------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>
>
> End of Xen-devel Digest, Vol 81, Issue 2
> ****************************************
>



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel