Re: Xen Ovmf/UEFI HVM error after upgrade to Debian Bookworm (was: xen ovmf/uefi firmware does not save screen resolution)

[Chuck Zmudzinski <brchuckz@xxxxxxxxxxxx>]

On 8/18/2023 9:18 AM, Paul Leiber wrote:
> Am 10.07.2023 um 18:47 schrieb Chuck Zmudzinski:
>> On 7/4/2023 5:06 PM, Paul Leiber wrote:
>>> On 14/02/2023 8:31 AM, Chuck Zmudzinski wrote:
>>>   > I am adding this information to complete the discussion of this
>>>   > problem I reported several months ago:
>>>   >
>>>   > I discovered the cause of the problems of ovmf/uefi booting with more
>>>   > recent Xen
>>>   > versions - it is not a bug in Xen, but the problem occurs because
>>>   > newer versions
>>>   > of ovmf do not have Xen support in the OvmfPkgX64 target from edk2
>>> and > the
>>>   > Xen support is only available from the Xen-specific OvmfXen target
>>>   > from edk2.
>>>   >
>>>   > The problem is discussed here on the Arch Linux forums:
>>>   >
>>>   > https://bbs.archlinux.org/viewtopic.php?pid=2012659#p2012659
>>>   >
>>>   > The last version that had the Xen support in the OvmfPkgX64 target was
>>>   > edk2-stable202105, so for newer versions it is necessary to build and
>>>   > use the
>>>   > Xen-specific OVMF.fd firmware target to boot properly on Xen.
>>>   >
>>>   > After building using 'OvmfPkg/build.sh -p OvmfPkg/OvmfXen.dsc' as
>>>   > explained
>>>   > here:
>>>   >
>>>   > https://lore.kernel.org/all/20190813113119.14804-1-
>>>   > anthony.perard@xxxxxxxxxx/
>>>   >
>>>   > from the edk2 source at https://github.com/tianocore/edk2 and using
>>>   > the
>>>   > resulting
>>>   > OVMF.fd firmware target with Xen support to boot the Xen HVM guest,
>>>   > the guest
>>>   > works properly with versions of ovmf edk2-stable202108 and newer.
>>>   >
>>>   > Unfortunately distros such as Debian and Fedora don't provide the Xen
>>>   > specific
>>>   > target in their ovmf packages, so it is necessary to build it from
>>>   > source for
>>>   > ovmf
>>>   > versions of edk2-stable202108 and newer.
>>>
>>> Thanks a lot for posting this, Chuck! It seems that I have run into this
>>> situation after upgrading from Debian Bullseye to Debian Bookworm. With
>>> standard packages, existing HVM DomUs using "firmware = 'ovmf'" (Windows
>>> Server 2022 and Windows 10) can't boot anymore. Booting these systems
>>> leads to the Windows error 0xc0000225. I wasn't able to fix this error.
>>> Booting an installation .iso leads to the same error. Booting the
>>> installation media with "firmware = 'bios'" leads to a normal boot.
>>>
>>> I tried building Ovmf following
>>> https://lore.kernel.org/all/20190813113119.14804-1-anthony.perard@xxxxxxxxxx/#t
>>> , but I wasn't fully able to create a working system:
>>>
>>> (1) Using the resulting OVMF.fd from the build process with "firmware =
>>> '/path/to/new/OVMF.fd' led consistently to a black screen in VNC or
>>> Spice with the text "Guest has not initialized the display (yet)".
>>>
>>> (2) Replacing the OVMF.fd in /var/lib/ovmf with the freshly built
>>> OVMF.fd led to a slight improvement. I could then boot the Windows
>>> Server installation .iso, but booting the Windows 10 installation .iso
>>> lead to a crash where the TianoCore logo was visible, but nothing
>>> happened. The two existing DomUs were still not bootable. When trying to
>>> boot any of them, in Ovmf log appears an error "FATAL ERROR - RaiseTpl
>>> with OldTpl(0x1F) > NewTpl(0x10)".
>>>
>>> However, I am not sure that I followed the procedure correctly, I might
>>> very well have done something very wrong. Any pointers are welcome.
>>>
>>> My HVM config file:
>>>
>>> type = "hvm"
>>> memory = 6144
>>> vcpus = 2
>>> name = "kalliope"
>>> firmware = 'ovmf'
>>> firmware = '/usr/local/lib/ovmf/OVMF.fd'
>>> vif = ['bridge=xenbr0,mac=XX:XX:XX:XX:XX:XX,ip=10.0.0.4']
>>> disk = ['phy:/dev/vg0/matrix,hda,w']
>>> device_model_version = 'qemu-xen'
>>> hdtype = 'ahci'
>>> pae = 1
>>> acpi = 1
>>> apic = 1
>>> vga = 'stdvga'
>>> videoram = 16
>>> xen_platform_pci = 1
>>> vendor_device = 'xenserver'
>>> viridian = 1
>>> on_crash = 'restart'
>>> device_model_args_hvm = [
>>>     # Debug OVMF
>>>     '-chardev', 'file,id=debugcon,path=/var/log/xen/ovmf.log,',
>>>     '-device', 'isa-debugcon,iobase=0x402,chardev=debugcon',
>>> ]
>>> sdl = 0
>>> serial = 'pty'
>>> vnc = 1
>>> vnclisten = "0.0.0.0"
>>> vncpasswd = ""
>>>
>>>
>>> There is a Debian bug report which seems to be related, I also described
>>> my situation there: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978595
>>>
>>> Paul
>>>
>> 
>> Hi Paul,
>> 
>> Your config looks OK to me, except that I have no pointers for a Windows 
>> guest.
>> I have a Windows HVM guest, but it uses BIOS / MBR instead of EFI / GPT so I
>> cannot easily reproduce your setup. I did not see any problems when using the
>> OVMF.fd file built for Xen in a bookworm dom0 according to Anthony's guide 
>> and
>> booting Linux HVM guests with EFI / GPT, so the problem might be that the 
>> graphics
>> for the Windows efi bootloader is not compatible with the efi implementation
>> that Xen uses (edk2 and Qemu with bochs vga) but the grub bootloader and
>> Linux graphics is compatible with the efi implementation for Xen HVM, but 
>> that is
>> just a guess.
>> 
>> Someone with more expertise about booting Windows in the Qemu / edk2 / bochs
>> environment might be able to shed more light on the error you are seeing with
>> some of your Windows guests. IIRC, I think there are some discussions of this
>> problem on Qemu or Xen or edk2 mailing lists in past years and you might try
>> searching those mailing lists for clues to the problem and patches that might
>> fix it.
>> 
>> Now, I have migrated my dom0 from Debian to Fedora since Fedora provides
>> an official ovmf-xen package now, but Debian still does not and I would have
>> to maintain it myself if I stayed with Debian.
>> 
>> Kind regards,
>> 
>> Chuck
> 
> Hi Chuck,
> 
> thanks for the reply, and sorry for the delay. I didn't have time yet to 
> dig any deeper, but with some help I found a workaround I'd like to 
> share. I could successfully start the Windows DomU with ovmf firmware 
> after removing the current ovmf package and installing a previous ovmf 
> package from Debian repositories, specifically version 2020.11-2+deb11u1.
> 
> I'll continue to try to help getting this fixed in current Debian 
> Bookworm packages.

It should just be a matter of filing a bug report on the Debian BTS
against the edk2 package and asking them to add an ovmf-xen package
that contains the OVMF.fd firmware with support for Xen and install
it where the Debian xl tools can find it. You might explain in the
bug report that, according to the upstream edk2 project, a separate
build of the firmware is needed to support Xen in versions
edk2-stable202108 and newer as explained in the upstream bug report:

https://bugzilla.tianocore.org/show_bug.cgi?id=2122

> 
> Best regards,
> 
> Paul
>