[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] X86 arch_domain ginormous, sizeof(struct domain) already == PAGE_SIZE

  • To: xen-devel@xxxxxxxxxxxxx
  • From: Corneliu ZUZU <czuzu@xxxxxxxxxxxxxxx>
  • Date: Fri, 29 Jan 2016 18:24:18 +0200
  • Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Keir Fraser <keir@xxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>
  • Comment: DomainKeys? See http://domainkeys.sourceforge.net/
  • Delivery-date: Fri, 29 Jan 2016 16:24:36 +0000
  • Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=bitdefender.com; b=mp5QcuH8LnCwqwnCJP/qZFIRXY27CjuHx6roHyNWGyGrIcMTTLMeye5sAY/ptREdOGViFbIAr+BVfc37Dn1DsmL2fkkr7oBuv1KY6V8JVI5bzlCEqY35UC8hhHTDy0AXLkELIkK7j89S1+jm849UEHCfFxKxCjIWZxkU2ByQcM2Q2lQCZ52c9f6JBGjnKSfHoA0LaozsW+0htr4HjZwbPZUR140zuwzWdZ35e5UTI90SR4cVnkdJKSdekXb9Cbf9qtGpQf1bIRvfkdjWH4ajGzuWRYb+3AyLOiBU9ESPqdQ4n2jinkHp/fPxNH7nS0QXi49Nb7B0UEmZwMRGz9TBSQ==; h=Received:Received:Received:Received:Received:To:Cc:From:Subject:Message-ID:Date:User-Agent:MIME-Version:Content-Type:Content-Transfer-Encoding:X-BitDefender-Scanner:X-BitDefender-Spam:X-BitDefender-SpamStamp:X-BitDefender-CF-Stamp;
  • List-id: Xen developer discussion <xen-devel.lists.xen.org>

I'm trying to refactor some arch-specific code into common code and was surprised to find out that the x86 domain structure already occupies PAGE_SIZE bytes, couldn't even add an unsigned short field in it w/o causing a compile-time error. I'm using the master branch of git://xenbits.xenproject.org/xen.git, compiled Xen by simply running "./configure && make dist-xen" (on my x86_64 Ubuntu machine).

Concretely, the change that caused problems was the refactoring of the x86 arch_domain.monitor field [@ xen/include/asm-x86/domain.h], which originally was:
     struct {
        unsigned int write_ctrlreg_enabled       : 4;
        unsigned int write_ctrlreg_sync          : 4;
        unsigned int write_ctrlreg_onchangeonly  : 4;
        unsigned int mov_to_msr_enabled          : 1;
        unsigned int mov_to_msr_extended         : 1;
        unsigned int singlestep_enabled          : 1;
        unsigned int software_breakpoint_enabled : 1;
        unsigned int guest_request_enabled       : 1;
        unsigned int guest_request_sync          : 1;
     } monitor;

by leaving there only the x86-specific part, i.e.:
     struct {
        uint8_t mov_to_msr_enabled          : 1;
        uint8_t mov_to_msr_extended         : 1;
     } monitor;

and moving the rest directly into the domain structure, i.e. add @ the end of struct domain [@ xen/include/xen/sched.h]:
     struct {
        uint16_t write_ctrlreg_enabled       : 4;
        uint16_t write_ctrlreg_sync          : 4;
        uint16_t write_ctrlreg_onchangeonly  : 4;
        uint16_t singlestep_enabled          : 1;
        uint16_t software_breakpoint_enabled : 1;
        uint16_t guest_request_enabled       : 1;
        uint16_t guest_request_sync          : 1;
     } monitor;

After the above change, X86 compilation of [xen/arch/x86/domain.c] fails w/:
    error: static assertion failed: "!(sizeof(*d) > PAGE_SIZE)"
the line that caused the fail being:
    BUILD_BUG_ON(sizeof(*d) > PAGE_SIZE);

To investigate I compiled the unaltered domain.c & used the pahole tool (part of dwarves package) to obtain the complete layout of the domain structure (& its members).
What I obtained:
    * sizeof(struct domain) is already = 4096 bytes (= PAGE_SIZE)
* sizeof(struct arch_domain) [x86] = sizeof(domain.arch) = 3328 bytes (arch_domain is marked __cacheline_aligned, i.e. aligned to 128 bytes)
    * sizeof(domain.arch.hvm_domain) = 2224 bytes
    * sizeof(domain.arch.hvm_domain.pl_time) = 1088 bytes

=> overall, X86 timers-related information occupies the most.

One could shrink the domain structure by transforming some of its fields to pointers, e.g. I could transform the pl_time field into a pointer and dynamically allocate its data when domain_create is called. Since the domain structure was designed to fit in a single page & arch_domain is marked __cacheline_aligned I presume such changes are sensible and should be done wisely.

How I should proceed?

Thank you,

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.