Re: [Xen-devel] [PATCH-for-4.13] x86/mm: don't needlessly veto migration

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 10/1/19 11:24 AM, Paul Durrant wrote:
>> -----Original Message-----
>> From: Jan Beulich <jbeulich@xxxxxxxx>
>> Sent: 01 October 2019 11:15
>> To: Paul Durrant <Paul.Durrant@xxxxxxxxxx>
>> Cc: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>; George Dunlap 
>> <George.Dunlap@xxxxxxxxxx>; Roger Pau
>> Monne <roger.pau@xxxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx; Juergen Gross 
>> <jgross@xxxxxxxx>; Wei Liu
>> <wl@xxxxxxx>
>> Subject: Re: [Xen-devel] [PATCH-for-4.13] x86/mm: don't needlessly veto 
>> migration
>>
>> On 01.10.2019 11:36, Paul Durrant wrote:
>>>> -----Original Message-----
>>>> From: Jan Beulich <jbeulich@xxxxxxxx>
>>>> Sent: 01 October 2019 10:19
>>>> To: Paul Durrant <Paul.Durrant@xxxxxxxxxx>
>>>> Cc: Andrew Cooper <Andrew.Cooper3@xxxxxxxxxx>; George Dunlap 
>>>> <George.Dunlap@xxxxxxxxxx>; Roger Pau
>>>> Monne <roger.pau@xxxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx; Juergen 
>>>> Gross <jgross@xxxxxxxx>; Wei
>> Liu
>>>> <wl@xxxxxxx>
>>>> Subject: Re: [Xen-devel] [PATCH-for-4.13] x86/mm: don't needlessly veto 
>>>> migration
>>>>
>>>> On 01.10.2019 10:52, Paul Durrant wrote:
>>>>>> -----Original Message-----
>>>>>> From: Jan Beulich <jbeulich@xxxxxxxx>
>>>>>> Sent: 01 October 2019 09:46
>>>>>> To: Paul Durrant <Paul.Durrant@xxxxxxxxxx>
>>>>>> Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx; Andrew Cooper 
>>>>>> <Andrew.Cooper3@xxxxxxxxxx>; Roger Pau Monne
>>>>>> <roger.pau@xxxxxxxxxx>; George Dunlap <George.Dunlap@xxxxxxxxxx>; 
>>>>>> Juergen Gross
>> <jgross@xxxxxxxx>;
>>>> Wei
>>>>>> Liu <wl@xxxxxxx>
>>>>>> Subject: Re: [Xen-devel] [PATCH-for-4.13] x86/mm: don't needlessly veto 
>>>>>> migration
>>>>>>
>>>>>> On 01.10.2019 10:28, Paul Durrant wrote:
>>>>>>> Now that xl.cfg has an option to explicitly enable IOMMU mappings for a
>>>>>>> domain, migration may be needlessly vetoed due to the check of
>>>>>>> is_iommu_enabled() in paging_log_dirty_enable().
>>>>>>> There is actually no need to prevent logdirty from being enabled unless
>>>>>>> devices are assigned to a domain and that domain is sharing HAP mappings
>>>>>>> with the IOMMU (in which case disabling write permissions in the P2M may
>>>>>>> cause DMA faults).
>>>>>>
>>>>>> But that's taking into account only half of the reason of the
>>>>>> exclusion. The other half is that assigned devices may cause pages
>>>>>> to be dirtied behind the back of the log-dirty logic.
>>>>>
>>>>> But that's no reason to veto logdirty. Some devices have drivers (in dom0)
>>>>> which can extract DMA dirtying information and set dirty tracking
>>>>> information appropriately.
>>>>
>>>> It still needs a positive identification then: Such drivers should tell
>>>> Xen for which specific devices such information is going to be provided.
>>>
>>> Why does the hypervisor need have the right of veto though? Surely it is
>>> the toolstack that should decide whether a VM is migratable in the
>>> presence of assigned h/w. Xen need only be concerned with the integrity
>>> of the host, which is why the check for ETP sharing remains.
>>
>> While the tool stack is to decide, the hypervisor is expected to guarantee
>> correct data coming back from XEN_DOMCTL_SHADOW_OP_{PEEK,CLEAN}.
> 
> For some definition of 'correct', yes, and I don't think that this change 
> violates any definition I can find in the domctl header.
> 
> Note: there are already emulators that will be playing with the dirty map on 
> an arbitrary and unsynchronized basis because they are emulating bus 
> mastering h/w.

But the question is, do we want the toolstack to have to become an
expert in what hardware might have external dirty tracking, and whether
such tracking is active?  At the moment that would mean either 1)
putting that information inside of libxc, or 2) duplicating it across
xapi and libxl, for instance.

One thing we could imagine is that when specific devices have an active
emulator (or whatever) propagating the dirty information, for that code
to tell Xen, "I am implementing dirty tracking for this device".  Then
when the toolstack enables logdirty, the check can be, "Are there any
devices *that don't have external dirty tracking enabled* assigned to
the guest?"

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel