[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [ARM] gvirt_to_maddr fails when DomU is created


  • To: Julien Grall <julien.grall@xxxxxxx>, Volodymyr Babchuk <vlad.babchuk@xxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Juergen Gross <jgross@xxxxxxxx>, Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>, "JBeulich@xxxxxxxx" <JBeulich@xxxxxxxx>
  • From: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • Date: Wed, 28 Nov 2018 00:05:09 +0000
  • Autocrypt: addr=andrew.cooper3@xxxxxxxxxx; prefer-encrypt=mutual; keydata= xsFNBFLhNn8BEADVhE+Hb8i0GV6mihnnr/uiQQdPF8kUoFzCOPXkf7jQ5sLYeJa0cQi6Penp VtiFYznTairnVsN5J+ujSTIb+OlMSJUWV4opS7WVNnxHbFTPYZVQ3erv7NKc2iVizCRZ2Kxn srM1oPXWRic8BIAdYOKOloF2300SL/bIpeD+x7h3w9B/qez7nOin5NzkxgFoaUeIal12pXSR Q354FKFoy6Vh96gc4VRqte3jw8mPuJQpfws+Pb+swvSf/i1q1+1I4jsRQQh2m6OTADHIqg2E ofTYAEh7R5HfPx0EXoEDMdRjOeKn8+vvkAwhviWXTHlG3R1QkbE5M/oywnZ83udJmi+lxjJ5 YhQ5IzomvJ16H0Bq+TLyVLO/VRksp1VR9HxCzItLNCS8PdpYYz5TC204ViycobYU65WMpzWe LFAGn8jSS25XIpqv0Y9k87dLbctKKA14Ifw2kq5OIVu2FuX+3i446JOa2vpCI9GcjCzi3oHV e00bzYiHMIl0FICrNJU0Kjho8pdo0m2uxkn6SYEpogAy9pnatUlO+erL4LqFUO7GXSdBRbw5 gNt25XTLdSFuZtMxkY3tq8MFss5QnjhehCVPEpE6y9ZjI4XB8ad1G4oBHVGK5LMsvg22PfMJ ISWFSHoF/B5+lHkCKWkFxZ0gZn33ju5n6/FOdEx4B8cMJt+cWwARAQABzSlBbmRyZXcgQ29v cGVyIDxhbmRyZXcuY29vcGVyM0BjaXRyaXguY29tPsLBegQTAQgAJAIbAwULCQgHAwUVCgkI CwUWAgMBAAIeAQIXgAUCWKD95wIZAQAKCRBlw/kGpdefoHbdD/9AIoR3k6fKl+RFiFpyAhvO 59ttDFI7nIAnlYngev2XUR3acFElJATHSDO0ju+hqWqAb8kVijXLops0gOfqt3VPZq9cuHlh IMDquatGLzAadfFx2eQYIYT+FYuMoPZy/aTUazmJIDVxP7L383grjIkn+7tAv+qeDfE+txL4 SAm1UHNvmdfgL2/lcmL3xRh7sub3nJilM93RWX1Pe5LBSDXO45uzCGEdst6uSlzYR/MEr+5Z JQQ32JV64zwvf/aKaagSQSQMYNX9JFgfZ3TKWC1KJQbX5ssoX/5hNLqxMcZV3TN7kU8I3kjK mPec9+1nECOjjJSO/h4P0sBZyIUGfguwzhEeGf4sMCuSEM4xjCnwiBwftR17sr0spYcOpqET ZGcAmyYcNjy6CYadNCnfR40vhhWuCfNCBzWnUW0lFoo12wb0YnzoOLjvfD6OL3JjIUJNOmJy RCsJ5IA/Iz33RhSVRmROu+TztwuThClw63g7+hoyewv7BemKyuU6FTVhjjW+XUWmS/FzknSi dAG+insr0746cTPpSkGl3KAXeWDGJzve7/SBBfyznWCMGaf8E2P1oOdIZRxHgWj0zNr1+ooF /PzgLPiCI4OMUttTlEKChgbUTQ+5o0P080JojqfXwbPAyumbaYcQNiH1/xYbJdOFSiBv9rpt TQTBLzDKXok86M7BTQRS4TZ/ARAAkgqudHsp+hd82UVkvgnlqZjzz2vyrYfz7bkPtXaGb9H4 Rfo7mQsEQavEBdWWjbga6eMnDqtu+FC+qeTGYebToxEyp2lKDSoAsvt8w82tIlP/EbmRbDVn 7bhjBlfRcFjVYw8uVDPptT0TV47vpoCVkTwcyb6OltJrvg/QzV9f07DJswuda1JH3/qvYu0p vjPnYvCq4NsqY2XSdAJ02HrdYPFtNyPEntu1n1KK+gJrstjtw7KsZ4ygXYrsm/oCBiVW/OgU g/XIlGErkrxe4vQvJyVwg6YH653YTX5hLLUEL1NS4TCo47RP+wi6y+TnuAL36UtK/uFyEuPy wwrDVcC4cIFhYSfsO0BumEI65yu7a8aHbGfq2lW251UcoU48Z27ZUUZd2Dr6O/n8poQHbaTd 6bJJSjzGGHZVbRP9UQ3lkmkmc0+XCHmj5WhwNNYjgbbmML7y0fsJT5RgvefAIFfHBg7fTY/i kBEimoUsTEQz+N4hbKwo1hULfVxDJStE4sbPhjbsPCrlXf6W9CxSyQ0qmZ2bXsLQYRj2xqd1 bpA+1o1j2N4/au1R/uSiUFjewJdT/LX1EklKDcQwpk06Af/N7VZtSfEJeRV04unbsKVXWZAk uAJyDDKN99ziC0Wz5kcPyVD1HNf8bgaqGDzrv3TfYjwqayRFcMf7xJaL9xXedMcAEQEAAcLB XwQYAQgACQUCUuE2fwIbDAAKCRBlw/kGpdefoG4XEACD1Qf/er8EA7g23HMxYWd3FXHThrVQ HgiGdk5Yh632vjOm9L4sd/GCEACVQKjsu98e8o3ysitFlznEns5EAAXEbITrgKWXDDUWGYxd pnjj2u+GkVdsOAGk0kxczX6s+VRBhpbBI2PWnOsRJgU2n10PZ3mZD4Xu9kU2IXYmuW+e5KCA vTArRUdCrAtIa1k01sPipPPw6dfxx2e5asy21YOytzxuWFfJTGnVxZZSCyLUO83sh6OZhJkk b9rxL9wPmpN/t2IPaEKoAc0FTQZS36wAMOXkBh24PQ9gaLJvfPKpNzGD8XWR5HHF0NLIJhgg 4ZlEXQ2fVp3XrtocHqhu4UZR4koCijgB8sB7Tb0GCpwK+C4UePdFLfhKyRdSXuvY3AHJd4CP 4JzW0Bzq/WXY3XMOzUTYApGQpnUpdOmuQSfpV9MQO+/jo7r6yPbxT7CwRS5dcQPzUiuHLK9i nvjREdh84qycnx0/6dDroYhp0DFv4udxuAvt1h4wGwTPRQZerSm4xaYegEFusyhbZrI0U9tJ B8WrhBLXDiYlyJT6zOV2yZFuW47VrLsjYnHwn27hmxTC/7tvG3euCklmkn9Sl9IAKFu29RSo d5bD8kMSCYsTqtTfT6W4A3qHGvIDta3ptLYpIAOD2sY3GYq2nf3Bbzx81wZK14JdDDHUX2Rs 6+ahAA==
  • Delivery-date: Wed, 28 Nov 2018 00:05:45 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Openpgp: preference=signencrypt

On 27/11/2018 19:40, Julien Grall wrote:
> (+ Stefano)
>
> On 11/27/18 5:12 PM, Volodymyr Babchuk wrote:
>> Hello community,
>
> Hi Volodymyr,
>
>>
>> After creating domU, I'm seeing lots of this messages from hypervisor:
>>
>> (XEN) p2m.c:1442: d1v0: gvirt_to_maddr failed va=0xffff80000efc7f0f
>> flags=0x1 par=0x809
>> (XEN) p2m.c:1442: d1v0: gvirt_to_maddr failed va=0xffff80000efc7f00
>> flags=0x1 par=0x809
>> (XEN) p2m.c:1442: d1v0: gvirt_to_maddr failed va=0xffff80000efc7f0f
>> flags=0x1 par=0x809
>>
>> Interestingly, I'm getting them from both Dom0 and DomU:
>>
>> (XEN) p2m.c:1442: d0v0: gvirt_to_maddr failed va=0xffff80003efd7f0f
>> flags=0x1 par=0x809
>> (XEN) p2m.c:1442: d1v0: gvirt_to_maddr failed va=0xffff80000efc7f0f
>> flags=0x1 par=0x809
>>
>> But only after DomU is created.
>>
>> I attached GDB and found that this is caused by update_runstate_area:
>>
>> (gdb) bt
>> #0  get_page_from_gva (v=0x80005dbe2000, v@entry=0x22f2c8
>> <schedule+1236>,
>>      va=va@entry=18446603337277996815, flags=flags@entry=1) at
>> p2m.c:1440
>> #1  0x000000000024e320 in translate_get_page (write=true, linear=true,
>> addr=18446603337277996815,
>>      info=...) at guestcopy.c:37
>> #2  copy_guest (buf=buf@entry=0x80005dbe20d7,
>> addr=addr@entry=18446603337277996815, len=len@entry=1,
>>      info=..., flags=flags@entry=6) at guestcopy.c:69
>> #3  0x000000000024e45c in raw_copy_to_guest
>> (to=to@entry=0xffff80003efd7f0f,
>>      from=from@entry=0x80005dbe20d7, len=len@entry=1) at guestcopy.c:110
>> #4  0x00000000002497b4 in update_runstate_area
>> (v=v@entry=0x80005dbe2000) at domain.c:287
>> #5  0x0000000000249eb8 in context_switch
>> (prev=prev@entry=0x80005dbe2000,
>>      next=next@entry=0x80005bf3c000) at domain.c:344
>> #6  0x000000000022f2c8 in schedule () at schedule.c:1583
>> #7  0x0000000000232c10 in __do_softirq
>> (ignore_mask=ignore_mask@entry=0) at softirq.c:50
>> #8  0x0000000000232ca4 in do_softirq () at softirq.c:64
>> #9  0x0000000000258254 in leave_hypervisor_tail () at traps.c:2302
>>
>> This issue is encountered on QEMU-ARMv8. Dom0 kernel is Linux 4.19.0
>> My XEN master is at d8ffac1f7 "xen/arm: gic: Remove duplicated comment
>> in do_sgi"
>>
>> The same setup worked perfectly with Xen 4.10.2
>
> The message is only printed in debug build. Do you have CONFIG_DEBUG
> enabled?
>
>>
>> Julien, I saw on mailing list, that you paid attention to issues with
>> gvirt_to_maddr,
>> so maybe you can be interested in this.
>
> Which thread are you speaking about? The problem is not because of
> gvirt_to_maddr but of how update_runstate_area is working at the moment.
>
> update_runstate_area is using a guest virtual address to update the
> vCPU runstate. It blindly assumes the vCPU runstate will always be
> mapped in stage-1 page-tables. However, if KPTI (Kernel Page Table
> Isolation) is enabled the kernel address space (and therefore the vCPU
> runstate) will not be mapped when running at EL0.
>
> So if you are restoring a vCPU that was executing code at EL0 then
> update_runstate_area will fail as the address is not mapped. There are
> a few solution suggested on the ML (see [1]). However I haven't had
> time to look at properly how to implement them.
>
> KPTI is getting used more widely (e.g meltdown and KASLR). So it would
> be good if we try to solve this problem sooner. I would be happy to
> review patches and/or provide advice if you want to tackle the problem.
>
> Cheers,
>
> [1] https://lists.xen.org/archives/html/xen-devel/2018-03/msg00223.html
>

update_runstate_area() using a virtual address is a complete misfeature,
and the sooner we can replace it, the better.  It's history is with x86
PV guests, where the early ABIs were designed in terms of Linux's
copy_{to,from}_user().

It is similarly broken in x86 with meltdown mitigations, as well as SMAP
considerations (PAN in ARM, iirc).

We've got two options.  Invent a new API which takes a gfn/gaddr, or
retrofit the API to be "you pass a virtual address, we translate to
gfn/gaddr, then update that".  Perhaps both.

When this was last discussed, I think the "onetime translate to
gfn/gaddr" was a good enough compatibility to cope with existing guests,
but that we should have a more clean way for modern guests.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.