[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Minios-devel] [UNIKRAFT PATCH RFC 00/23] Impelment virtio_mmio and pci ecam controller for arm64 kvm plat



Hi Felipe

For your mentioned crashing issue, it was caused by missing important memory barrier.

It can be fixed by below patch(with newlib), I will figure out the graceful way to add

the mb.

--- a/plat/drivers/include/virtio/virtqueue.h
+++ b/plat/drivers/include/virtio/virtqueue.h
@@ -285,6 +285,7 @@ static inline void virtqueue_host_notify(struct virtqueue 
*vq)

        if (vq->vq_notify_host && virtqueue_notify_enabled(vq)) {
                uk_pr_debug("notify queue %d\n", vq->queue_id);
+               dmb(oshst);
                vq->vq_notify_host(vq->vdev, vq->queue_id);
        }
 }

I will continue to setup the nginx env.

---
Cheers,
Justin (Jia He)


On 2020/3/13 18:00, Felipe Huici wrote:
Hi Justin,

Thanks so much for the series, it's great to have networking going on arm64! 
I've tested it against http reply and it works fine. However, I have a number 
of issues with the series:

1. It doesn't build against newlib, which precludes running a large set of 
applications (e.g., Redis, nginx, sqlite, etc.). The fix here is simple:

diff --git a/lib/fdt/fdt_addresses.c b/lib/fdt/fdt_addresses.c
index b362b41..30c7ff5 100644
--- a/lib/fdt/fdt_addresses.c
+++ b/lib/fdt/fdt_addresses.c
@@ -60,7 +60,7 @@

  int fdt_address_cells(const void *fdt, int nodeoffset)
  {
-       __u32 cells;
+       uint32_t cells;
         int parent;
         int off = nodeoffset;
         int *prop;
@@ -81,7 +81,7 @@ int fdt_address_cells(const void *fdt, int nodeoffset)

  int fdt_size_cells(const void *fdt, int nodeoffset)
  {
-       __u32 cells;
+       uint32_t cells;
         int parent;
         int off = nodeoffset;
         int *prop;

2. I get a large number of (spurious?) error messages such as:

[    0.036700] ERR:  [libkvmpf] platform_bus.c @ 180  : Platform Failed to 
initialize device driver, ret(-19)

Things work despite these, should these really be errors?

3. There's a large number of compilation warnings (too many to list here)

4. When setting ukdebug's Kernel message level to "Show all types of messages" 
I get a register dump during boot:

[    0.073966] Info: [libkvmpci] pci_ecam.c @ 370  :  -> imaplen=90
[    0.074064] Info: [libkvmpci] pci_ecam.c @ 325  :  -> match=0 (imaplen=86)
[    0.075033] Info: [libkvmpci] pci_ecam.c @ 370  :  -> imaplen=80
[    0.075131] Info: [libkvmpci] pci_ecam.c @ 325  :  -> match=1 (imaplen=76)
[    0.076096] Info: [libkvmpci] pci_ecam.c @ 370  :  -> imaplen=70
[    0.076193] Info: [libkvmpci] pci_ecam.c @ 388  :  -> new parent: 0x175cOF
[    0.076313] Info: [libkvmpci] pci_ecam.c @ 289  :  -> got it !
[    0.076417] Info: [libkvmpci] pci_bus_arm64.c @ 100  : pci dev 
base(0x3eff0040) irq(37)
[    0.076547] Info: [libkvmvirtio] virtio_pci.c @ 379  : Added virtio-pci 
device 1009
[    0.076668] Info: [libkvmvirtio] virtio_pci.c @ 381  : Added virtio-pci 
subsystem_device_id 0009
[    0.076885] Info: [libkvmvirtio] virtio_bus.c @ 129  : Virtio device 
0x7ffc3010 initialized
[    0.077322] CRIT: [libkvmplat] traps.c @ 67   : Unikraft: EL1 sync trap 
caught
[    0.077468] CRIT: [libkvmplat] traps.c @ 39   : Unikraft: Dump registers:
[    0.077598] CRIT: [libkvmplat] traps.c @ 40   : SP       : 0x00000000402cfce0
ESR_EL1  : 0x0000000096000010
ELR_EL1  : 0x0000000040106a88
LR (x30) : 0x000000004010a13c
PSTATE   : 0x0000000020000345
FAR_EL1  : 0x000000003eff0060
x00 ~ x03: 0x000000003eff0060 0x000000003eff0056 0x000000007ffd0010 
0x000000000000ff03
[    0.078162] CRIT: [libkvmplat] traps.c @ 50   : x04 ~ x07: 
0x00000000000000ff 0xffffffffc100ffaa 0x000000000000ff03 0x00000000402cfe00
[    0.078363] CRIT: [libkvmplat] traps.c @ 50   : x08 ~ x11: 
0x00000000012dfdaa 0x000000003effff59 0x0000000000000000 0x0000000000000016
[    0.078556] CRIT: [libkvmplat] traps.c @ 50   : x12 ~ x15: 
0x000000003eff0040 0x00000000402dfd10 0x0000000000000000 0x0000000000000000
[    0.078752] CRIT: [libkvmplat] traps.c @ 50   : x16 ~ x19: 
0x000000000000000a 0x0000000000000000 0x0000000000000000 0x000000007ffc4010
[    0.078945] CRIT: [libkvmplat] traps.c @ 50   : x20 ~ x23: 
0x0000000079000001 0x000000004026d000 0x0000000000000001 0x0000000000000001
[    0.079143] CRIT: [libkvmplat] traps.c @ 50   : x24 ~ x27: 
0x000000004023e000 0x000000004023e000 0x0000000040257168 0x0000020000000000
[    0.079356] CRIT: [libkvmplat] traps.c @ 53   : x28 ~ x29: 
0x000000004023e000 0x00000000402dfd10
[    0.079513] Info: [libkvmplat] shutdown.c @ 35   : Unikraft halted

5. I've tested the series against nginx: it inits fine, and I can ping the 
network stack, but nginx itself doesn't reply to HTTP/wget requests, not sure 
why...please let me know if you'd like to replicate this.

Thanks,

-- Felipe

On 06.03.20, 05:50, "Jia He" <Justin.He@xxxxxxx> wrote:

     To support virtio-* family device, we have 2 choices (virtio mmio, virtio 
pci)
     on arm64 kvm plat. This patch series enable the support of virtio mmio. 
virtio
     pci needs pci controller ECAM mode in arm64 Unikraft.
Previously, I proposed a separated series at [1], Julien had given some
     comments including using fdt_get_address and refining the platform device
     probing sequence. This series address them, but prevent changing too much
     in original virtio mmio series.
Patch 1-5 implement the platform bus and virtio mmio skeleton.
     Patch 8-9 implement the pci ecam skeleton
     Patch 10-12 introduce the new probing interface for virtio mmio and pci
           on arm.
     Patch 7,13-16 lib/fdt or ofw/fdt changes
     Patch 17-18 split the arch specific pci bus codes into different directory
     Others are bug fixing or building system changes.
I set it as "RFC" since I haven't resolved all the compilation warnings or
     checkpatch issues. Even single patch might not compilable. But I tested
     the whole series, it works fine in both virtio mmio and virtio pci for
     net device. The test case is a simple httpreply app.
test qemu cmdline:
     - for virtio mmio
     -device virtio-net-device,id=net1,netdev=hostnet1,mac=52:54:00:09:a4:38
     - for virtio pci
     -device 
virtio-net-pci,id=net0,netdev=hostnet0,mac=52:54:00:09:a4:31,disable-modern=on
Jia He (23):
       plat/virtio: Introduce several helpers for virtio ring
       plat/platform: Introduce new platform bus
       plat/virtio: Support virtio_cread/cwrite on arm64
       plat/virtio: Add new virtio_mmio device/driver on arm64
       build: Enable the platform bus and virtio_mmio device on arm64
       plat/kvm: arm64: Fix arm64 memory layout for pcie ecam
       plat/ofw: Export fdt_find_irq_parent_offset
       plat/pci_ecam: Introduce pci/pci_ecam.h
       plat/pci_ecam: Introduce pci ecam skeleton
       lib/ukbus: Adjust default bus register priority
       plat/platform_bus: Add probe/match nterface for platform devices
       plat/virtio_mmio: Implement platform probe for virtio_mmio
       plat/ofw: Support completed ranges mapping
       lib/fdt: Fix fdt_address_cell and fdt_size_cell
       plat/ofw: add fdt_node_offset_idx_by_compatible_list
       plat/ofw: Add fdt_prop_read_bool
       plat/pci_bus: Split specific code into arch directory
       plat/pci_bus: arm64: Implement arch specific add/probe interfaces on
         arm64
       build: make KVM_PCI default enable on arm64
       plat/pci_bus: change the type of pci_device.base from u16 to unsigned
         long
       build: Enable pci library on arm64
       plat/virtio: Fix virtio net device mac negotiation
       plat/virtio: Change the type of irq,base from u16 to u64 for arm64
lib/fdt/exportsyms.uk | 1 +
      lib/fdt/fdt_addresses.c                     |  57 +-
      lib/ukbus/include/uk/bus.h                  |  16 +-
      plat/common/arm/pci_bus_arm64.c             | 204 +++++++
      plat/common/include/arm/arm64/cpu_defs.h    |   3 +
      plat/common/include/pci/pci_bus.h           | 105 +++-
      plat/common/include/pci/pci_ecam.h          | 101 ++++
      plat/common/include/platform_bus.h          | 116 ++++
      plat/common/pci_bus.c                       | 167 +-----
      plat/common/pci_ecam.c                      | 524 ++++++++++++++++++
      plat/common/platform_bus.c                  | 238 ++++++++
      plat/common/x86/pci_bus_x86.c               | 174 ++++++
      plat/drivers/include/ofw/fdt.h              |  63 +++
      plat/drivers/include/virtio/virtio_config.h |  63 ++-
      plat/drivers/include/virtio/virtio_mmio.h   | 137 +++++
      plat/drivers/include/virtio/virtqueue.h     |  33 ++
      plat/drivers/ofw/fdt.c                      |  81 ++-
      plat/drivers/virtio/virtio_mmio.c           | 567 ++++++++++++++++++++
      plat/drivers/virtio/virtio_net.c            |  12 +-
      plat/drivers/virtio/virtio_pci.c            |   6 +-
      plat/drivers/virtio/virtio_ring.c           |  32 ++
      plat/kvm/Config.uk                          |  14 +-
      plat/kvm/Makefile.uk                        |  29 +-
      plat/kvm/arm/pagetable64.S                  |  22 +-
      24 files changed, 2537 insertions(+), 228 deletions(-)
      create mode 100644 plat/common/arm/pci_bus_arm64.c
      create mode 100644 plat/common/include/pci/pci_ecam.h
      create mode 100644 plat/common/include/platform_bus.h
      create mode 100644 plat/common/pci_ecam.c
      create mode 100644 plat/common/platform_bus.c
      create mode 100644 plat/common/x86/pci_bus_x86.c
      create mode 100644 plat/drivers/include/virtio/virtio_mmio.h
      create mode 100644 plat/drivers/virtio/virtio_mmio.c
--
     2.17.1
_______________________________________________
Minios-devel mailing list
Minios-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/minios-devel




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.