[Xen-devel] Xen 4.5 development update (July update)

[konrad.wilk@xxxxxxxxxx]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is less a week away!

Most of the Xen patches that impacted the hypervisor and had'fair'
(except the HVM introspection one) I moved to the Xen 4.6 list.

They might get Acked by maintainers in the next couple of days which would
be fantastic - and if so I will update the list. But perhaps not. Also
some are in 'good' or in 'ok' condition - but that does not mean they
will automatically go in Xen 4.5.

In terms of QEMU  - I only had three items and since the version of QEMU
from upstream we are using is already established (and stable) I don't see us
backporting any more patches from upstream. But perhaps Stefano has some
other plans...

In terms of the toolstack - I kept the ones that are in the 'fair' category
as I think they are easier to test/rebase/retest than the hypervisor ones.

In terms of Linux I am keeping the 'fair' ones as by the Xen 4.6 release
it could be v3.19, which means there is still an upcoming merge window
for those.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
A bunch of items had moved to the completed phase which is fantastic.

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

In terms of libvirt it has monthly releases. As such not going to track
every release - but closer to when RCs are out.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Coding time: <=== NOW, one week left!

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

If you think your patchset MUST go in Xen 4.5 I will post the procedure
for requesting an exception to get them in past the feature freeze next
week.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM - Device assigment on ARM (good)
   Linux parts at risk.
   v2 for hypervisor out
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
   v12 posted.
  -  Arianna Avanzini

*  ARM Xen UEFI booting on ARM (ok)
   v2
  -  Roy Franz

*  ARM PSCI v0.2 (good)
   v11 posted
  -  Parth Dixit

*  ARM GICv3 support (ok)
   v6a patchset (refactor in, also known as v9a); v7 posted
   v9a posted which does GIC and VGIC code refactoring
   v8 posted
  -  Vijay Kilari

*  ARM - MiniOS (ok)
   v7 posted
  -  Thomas Leonard

*  ARM - VGIC emulation (ok)
   Reposted as gic and vgic fixes and improvements
   v12
  -  Stefano Stabellini

*  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
   Provide kernels an grant->MFN lookup
   v4
  -  Stefano Stabellini

*  ARM - Add Odroid-XU (Exynos5410) support (ok)
   v4
  -  Suriyan Ramasami

*  ARM implement mem_access (ok)
   v3
   https://github.com/tklengyel/xen/tree/arm_memaccess3
  -  Tamas K Lengyel

== x86 ==

*  New Migration (v2). (good)
   v6 posted, going on the 'good' side!
  -  Andrew Cooper & David Vrabel

*  PVH - AMD hardware support. (ok)
   Posted.
  -  Mukesh Rathor

*  VMware backdoor (hypercall) (ok)
   v2 posted.
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
   v9 posted
  -  Boris Ostrovsky

*  Cache QoS Monitoring - hypercalls (ok)
   Just hypercalls - no toolstack changes.
   v14
  -  Chao Peng, Dongxiao Xu, and Shantong Kang

*  XenRT (Preemptive Global Earliest Deadline First) (ok)
   v3
  -  Meng Xu

*  Introspection of HVM guests (ok)
   RFC v11 (dropped the RFC in the list)
  -  Razvan Cojocaru

*  vNUMA in Xen (ok)
   v9 posted
   git://gitorious.org/vnuma/xen_vnuma.git:v9
  -  Elena Ufimtseva

== lib{xc,xl} and toolstack ==

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  Remus in Xen (libxl) (ok)
   v19
   url: https://github.com/laijs/xen remus-v19
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   v6 posted
   git://gitorious.org/vnuma/linux_vnuma.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
   v2 posted
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to QEMU v2.next ==

*  Using qemu-upstream in a stubdomain (fair)
   Will use rump kernels.
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  Xen PV block driver in OVMF (UEFI in guest) (fair)
   RFC posted.
  -  Anthony PERARD

== Deferred to Xen hypervisor 4.6 ==

*  extending mem_access support to PV domain (fair)
   RFC v2
  -  Aravindh Puthiyaparambil (aravindp)

*  Repurpose SEDF Scheduler for Real-time (fair)
   RFC patch posted (v2)
  -  Joshua Whitehead, Robert VanVossen

*  ARM remote processor iommu module (GPUs + IPUs) (fair)
   v3 posted
  -  Andrii Tseglytskyi

*  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
   RFC patch posted
  -  Wen Congyang
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  extend the xenstore ring with a 'closing' signal (fair)
   RFC patch posted
  -  David Scott

*  libxl migrationv2 patches. (none)
  -  Andrew Cooper & David Vrabel

*  tmem migrationv2 patches. (none)
  -  Bob Liu & Andrew Cooper & David Vrabel

*  Remus using migration-v2 (fair)
   RFC posted - depends on v6 of 'New Migration'
  -  Yang Hongyang

*  Xen multiboot2-EFI support (fair)
   Needed for GrUB2
   Depends on Xen Boot info (rework multiboot and other structs)
   See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
   RFC posted
  -  Daniel Kiper

*  dirty vram / IOMMU bug (fair)
   http://bugs.xenproject.org/xen/bug/38
  -  Zhang, Yang Z

*  snapshot API extension (checkpointing disk) (ok)
   v5
   His email bounces.
  -  Bamvor Jian Zhang

*  Rearrange and cleanup installation destination directories (/var -> 
var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  1TB slow destruction (ok)
  -  Bob Liu

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  ARM VM save/restore/live migration (none)
   Need to rebased against migrationv2 - no code posted.
  -  Junghyun Yoo

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  cpuid leveling (none)
   
http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get 
two hung domains
  -  Ian Jackson

*  IO-NUMA - hwloc and xl (none)
   Andrew Cooper had an RFC patch for hwloc
   add restrictions  as to which devices cannot safely/functionally be split 
apart.
  -  Boris Ostrovsky

*  HVM guest NUMA (none)
  -  Matt Wilson

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  ARM GICv2m support (none)
  -  Linaro (unknown)

== Up for grabs ==

*  OSSTest - also test Linux PVH guests

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with 
PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default 
in xl.conf.  For example, if we could have a default "backend" parameter for 
vifs, then it would be easy to switch back and forth between a backend in a 
driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  pvSCSI in Linux (fronted and backend) (done)
   v6
  -  Juergen Gross

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except 
one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)

*  multiple AMD container files appended together in initrd (early initramfs)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Netback multiqueue (good)
  -  Wei Liu

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  ARM DRA7 support (done)
   v3 posted
   v3 with comments applied
  -  Andrii Tseglytskyi

*  rework VM Generation ID (done)
   v7 posted
  -  David Vrabel

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  Linux pvops of Xen EFI hypercall support (done)
  -  Daniel Kiper

*  ARM: Use super pages in p2m (done)
   v5 posted
  -  Ian Campbell

*  QEMU 2.0 branch for qemu-upstream (done)
   It is v2.0 with 2.1 Xen backports.
  -  Stefano Stabellini

*  systemd support (done)
   v11
  -  Luis R. Rodriguez

*  Soft affinity for vcpus libxl/xl changes (done)
   v13 posted
  -  Dario Faggioli

*  HT enabled, virtualization overhead is high (Xen 4.4) (done)
   kernbench demonstrated it
   Looking and tracing
   http://www.gossamer-threads.com/lists/xen/devel/339409
   False alarm.
  -  Dario Faggioli


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel