Re: xen summit 2023 design session: Using only stable interfaces for (o)xenstored

[Juergen Gross <jgross@xxxxxxxx>]

On 27.06.23 12:21, Edwin Torok wrote:
> Background: 
> https://design-sessions.xenproject.org/uid/discussion/disc_MEXyUIXV6clI8n1kgzQ9/view
>
> Possible solutions:
> Andrew Cooper: use get domain info list instead of querying domains one-by-one 
> to reduce number of hypercalls, even though Alejandro's patch makes the 
> get_domainfo lookups O(1).
> To avoid a hypercall a shared read-only memory page between Xen and Dom0, 
> containing a bitmap of domain existence and shutdown state.
> We have 32k domains max and this fits into a small number of 4K pages.
> Hypervisor can update it using atomic set/clear-bit, Dom0 will memcpy and then 
> check for differences
> Bernhardt/Andrew: discussion about efficiency and find first bit set, XOR. A 
> *lot* more efficient than using hypercalls anyway
> Edwin: do I have to implement that for various architectures?
> Andrew Cooper: already implemented for all arches. Some architectures can only 
> do atomic updates on natural register width (RISC-V), but that is already 
> handled
> Edwin: how would the stable interface look like? Memory layout definition in 
> the header and ..?
> Andrew Cooper: use acquire_resource with a new resource type, already have 
> stable xen libs for resource mapping
>
> Christopher Clarke/Daniel Smith (virtually from chatroom): alternatives to 
> bitmap, how is it indexed?
> Andrew Coooper: indexed by domid, we only have 32k
> Marcus Granado: what if we want to increase that?
> Andrew Coooper: 16-bit domid is baked into so many ABIs that if we want to 
> increase that then this API won't be the blocker. No need to worry about more 
> than 32k domids.
> Christopher Clark: how about using an Argo ring
> Andrew Cooper: Xen must NOT block when delivering the notification and 
> notification delivery cannot be lossy, cannot afford to drop events: 
> toolstacks/xenstored critically rely on this information.
> Christopher Clark: bip buffers: 
> https://www.codeproject.com/Articles/3479/The-Bip-Buffer-The-Circular-Buffer-with-a-Twist
>  as possible solution to avoid running out of space
> Andrew Cooper: bitmap also avoids having to take the domctl lock and reduce 
> background idle CPU usage
> Edwin: on that topic: we should also fix xcp-rrdd so it doesn't shell out to 
> xenpm every 5s to collect statistics...
> Andrew Cooper: should be possible to use libxenctrl directly for that
> Edwin Torok: there are other interfacs that would need to be stabilized (see 
> xenopsd), but lets start small and fix just this one for now, which will result 
> in an immediate improvement for (o)xenstored since this was the final blocker 
> to getting off stable interfaces (there is one other usage, but that already 
> has a patch posted/solution)
>
> Edwin Torok: how about release cadence? Doesn't necessarily have to go into 
> immediately next Xen release, but don't know in which phase the release is in 
> right now
> Andrew Cooper: probably about 6 weeks time to get into next release
See https://lists.xen.org/archives/html/xen-devel/2021-09/msg01098.html


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature