Re: [PATCH v3 2/4] Build system: Replace git:// and http:// with https://

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Fri, Feb 17, 2023 at 04:35:25PM -0500, Demi Marie Obenour wrote:
> Obtaining code over an insecure transport is a terrible idea for
> blatently obvious reasons.  Even for non-executable data, insecure
> transports are considered deprecated.
> 
> This patch enforces the use of secure transports in the build system.
> Some URLs returned 301 or 302 redirects, so I replaced them with the
> URLs that were redirected to. 

https://gitlab.com/xen-project/patchew/xen/-/pipelines/781679811

I'm a bit confused about debian build errors:

    ERROR: The certificate of 'xenbits.xen.org' is not trusted.
    ERROR: The certificate of 'xenbits.xen.org' has expired.

Is clock on gitlab runners (way) off?

>  I also found that the old zlib used in
> the I/O emulator stubdomain can no longer be obtained from
> https://www.zlib.net and that the TPM emulator and PolarSSL (used by the
> vTPM and vTPM manager stubdomains) can no longer be obtained from their
> respective original URLs.  Therefore, configure will now error out
> instead of trying to download them.

First of all, such change definitely wants a separate patch,
de-supporting some configurations do not belong to "Replace git:// and
http:// with https://"; patch. But then, I don't think that's correct
approach. It is a bug to be fixes, instead of breaking it even more.
configure script already supports Xen's mirror, and I think it's even
enabled by default (see --enable-extfiles), and also supports providing
alternative download location (via env variables). So it seems your
change here in fact breaks something that was working before...

> Signed-off-by: Demi Marie Obenour <demi@xxxxxxxxxxxxxxxxxxxxxx>
> ---
>  Config.mk                         |  2 +-
>  stubdom/configure                 | 24 +++++++++++++++---------
>  stubdom/configure.ac              | 24 +++++++++++++++---------
>  tools/firmware/etherboot/Makefile |  6 +-----
>  4 files changed, 32 insertions(+), 24 deletions(-)
> 
> diff --git a/Config.mk b/Config.mk
> index 
> 75f1975e5e78af44d36c2372cba6e89b425267a5..b2bef45b059976d5a6320eabada6073004eb22ee
>  100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -191,7 +191,7 @@ APPEND_CFLAGS += $(foreach i, $(APPEND_INCLUDES), -I$(i))
>  EMBEDDED_EXTRA_CFLAGS := -fno-pie -fno-stack-protector 
> -fno-stack-protector-all
>  EMBEDDED_EXTRA_CFLAGS += -fno-exceptions -fno-asynchronous-unwind-tables
>  
> -XEN_EXTFILES_URL ?= http://xenbits.xen.org/xen-extfiles
> +XEN_EXTFILES_URL ?= https://xenbits.xen.org/xen-extfiles
>  # All the files at that location were downloaded from elsewhere on
>  # the internet.  The original download URL is preserved as a comment
>  # near the place in the Xen Makefiles where the file is used.
> diff --git a/stubdom/configure b/stubdom/configure
> index 
> b8bffceafdd46181e26a79b85405aefb8bc3ff7d..e40aca9afd0de2c5074978d654d4e78f4f63e3d2
>  100755
> --- a/stubdom/configure
> +++ b/stubdom/configure
> @@ -3535,7 +3535,7 @@ if test "x$ZLIB_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    ZLIB_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  ZLIB_URL="http://www.zlib.net";
> +  ZLIB_URL="https://www.zlib.net";
>  fi
>  
>  fi
> @@ -3550,7 +3550,7 @@ if test "x$LIBPCI_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    LIBPCI_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  LIBPCI_URL="http://www.kernel.org/pub/software/utils/pciutils";
> +  LIBPCI_URL="https://mirrors.edge.kernel.org/pub/software/utils/pciutils";
>  fi
>  
>  fi
> @@ -3565,7 +3565,7 @@ if test "x$NEWLIB_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    NEWLIB_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  NEWLIB_URL="ftp://sources.redhat.com/pub/newlib";
> +  NEWLIB_URL="https://sourceware.org/ftp/newlib";
>  fi
>  
>  fi
> @@ -3580,7 +3580,7 @@ if test "x$LWIP_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    LWIP_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  LWIP_URL="http://download.savannah.gnu.org/releases/lwip";
> +  LWIP_URL="https://download.savannah.gnu.org/releases/lwip";
>  fi
>  
>  fi
> @@ -3595,7 +3595,7 @@ if test "x$GRUB_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    GRUB_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  GRUB_URL="http://alpha.gnu.org/gnu/grub";
> +  GRUB_URL="https://alpha.gnu.org/gnu/grub";
>  fi
>  
>  fi
> @@ -3607,7 +3607,7 @@ GRUB_VERSION="0.97"
>  
>  if test "x$OCAML_URL" = "x"; then :
>  
> -     OCAML_URL="http://caml.inria.fr/pub/distrib/ocaml-4.02";
> +     OCAML_URL="https://caml.inria.fr/pub/distrib/ocaml-4.02";
>  
>  fi
>  OCAML_VERSION="4.02.0"
> @@ -3621,7 +3621,7 @@ if test "x$GMP_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    GMP_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  GMP_URL="ftp://ftp.gmplib.org/pub/gmp-4.3.2";
> +  GMP_URL="https://gmplib.org/download/gmp/archive";
>  fi
>  
>  fi
> @@ -3636,7 +3636,7 @@ if test "x$POLARSSL_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    POLARSSL_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  POLARSSL_URL="http://polarssl.org/code/releases";
> +  POLARSSL_URL="https://polarssl.org/code/releases";
>  fi
>  
>  fi
> @@ -3651,7 +3651,7 @@ if test "x$TPMEMU_URL" = "x"; then :
>       if test "x$extfiles" = "xy"; then :
>    TPMEMU_URL=\$\(XEN_EXTFILES_URL\)
>  else
> -  TPMEMU_URL="http://download.berlios.de/tpm-emulator";
> +  TPMEMU_URL="https://download.berlios.de/tpm-emulator";
>  fi
>  
>  fi
> @@ -3669,6 +3669,12 @@ vtpmmgr="n"
>  fi
>  
>  
> +if test "x$vtpm" != xn || test "x$vtpmmgr" != xn || test "x$ioemu" != xn; 
> then
> +    if test "x$extfiles" != xy; then
> +        as_fn_error $? "Sources needed for the vTPM, vTPM manager, and IO 
> emulator stubdomains are no longer at their original URLs" "$LINENO" 5
> +    fi
> +fi
> +
>  #Conditionally enable these stubdoms based on the presense of dependencies
>  
>  if test "x$vtpm" = "xy" || test "x$vtpm" = "x"; then :
> diff --git a/stubdom/configure.ac b/stubdom/configure.ac
> index 
> e20d99edac0da88098f4806333edde9f31dbc1a7..d27f2bc1f17140ab41a687e1e8faaa66e2b4483b
>  100644
> --- a/stubdom/configure.ac
> +++ b/stubdom/configure.ac
> @@ -55,19 +55,25 @@ AC_PROG_INSTALL
>  AX_DEPENDS_PATH_PROG([vtpm], [CMAKE], [cmake])
>  
>  # Stubdom libraries version and url setup
> -AX_STUBDOM_LIB([ZLIB], [zlib], [1.2.3], [http://www.zlib.net])
> -AX_STUBDOM_LIB([LIBPCI], [libpci], [2.2.9], 
> [http://www.kernel.org/pub/software/utils/pciutils])
> -AX_STUBDOM_LIB([NEWLIB], [newlib], [1.16.0], 
> [ftp://sources.redhat.com/pub/newlib])
> -AX_STUBDOM_LIB([LWIP], [lwip], [1.3.0], 
> [http://download.savannah.gnu.org/releases/lwip])
> -AX_STUBDOM_LIB([GRUB], [grub], [0.97], [http://alpha.gnu.org/gnu/grub])
> -AX_STUBDOM_LIB_NOEXT([OCAML], [ocaml], [4.02.0], 
> [http://caml.inria.fr/pub/distrib/ocaml-4.02])
> -AX_STUBDOM_LIB([GMP], [libgmp], [4.3.2], 
> [ftp://ftp.gmplib.org/pub/gmp-4.3.2])
> -AX_STUBDOM_LIB([POLARSSL], [polarssl], [1.1.4], 
> [http://polarssl.org/code/releases])
> -AX_STUBDOM_LIB([TPMEMU], [berlios tpm emulator], [0.7.4], 
> [http://download.berlios.de/tpm-emulator])
> +AX_STUBDOM_LIB([ZLIB], [zlib], [1.2.3], [https://www.zlib.net])
> +AX_STUBDOM_LIB([LIBPCI], [libpci], [2.2.9], 
> [https://mirrors.edge.kernel.org/pub/software/utils/pciutils])
> +AX_STUBDOM_LIB([NEWLIB], [newlib], [1.16.0], 
> [https://sourceware.org/ftp/newlib])
> +AX_STUBDOM_LIB([LWIP], [lwip], [1.3.0], 
> [https://download.savannah.gnu.org/releases/lwip])
> +AX_STUBDOM_LIB([GRUB], [grub], [0.97], [https://alpha.gnu.org/gnu/grub])
> +AX_STUBDOM_LIB_NOEXT([OCAML], [ocaml], [4.02.0], 
> [https://caml.inria.fr/pub/distrib/ocaml-4.02])
> +AX_STUBDOM_LIB([GMP], [libgmp], [4.3.2], 
> [https://gmplib.org/download/gmp/archive])
> +AX_STUBDOM_LIB([POLARSSL], [polarssl], [1.1.4], 
> [https://polarssl.org/code/releases])
> +AX_STUBDOM_LIB([TPMEMU], [berlios tpm emulator], [0.7.4], 
> [https://download.berlios.de/tpm-emulator])
>  
>  #These stubdoms should be enabled if the dependent one is
>  AX_STUBDOM_AUTO_DEPENDS([vtpmmgr], [vtpm])
>  
> +if test "x$vtpm" != xn || test "x$vtpmmgr" != xn || test "x$ioemu" != xn; 
> then
> +    if test "x$extfiles" != xy; then
> +        AC_MSG_ERROR([Sources needed for the vTPM, vTPM manager, and IO 
> emulator stubdomains are no longer at their original URLs])
> +    fi
> +fi
> +
>  #Conditionally enable these stubdoms based on the presense of dependencies
>  AX_STUBDOM_CONDITIONAL_FINISH([vtpm-stubdom], [vtpm])
>  AX_STUBDOM_CONDITIONAL_FINISH([vtpmmgr-stubdom], [vtpmmgr])
> diff --git a/tools/firmware/etherboot/Makefile 
> b/tools/firmware/etherboot/Makefile
> index 
> 4bc3633ba3d67ff9f52a9cb7923afea73c861da9..6ab9e5bc6b4cc750f2e802128fbc71e9150397b1
>  100644
> --- a/tools/firmware/etherboot/Makefile
> +++ b/tools/firmware/etherboot/Makefile
> @@ -4,11 +4,7 @@ XEN_ROOT = $(CURDIR)/../../..
>  include $(XEN_ROOT)/tools/Rules.mk
>  include Config
>  
> -ifeq ($(GIT_HTTP),y)
> -IPXE_GIT_URL ?= http://git.ipxe.org/ipxe.git
> -else
> -IPXE_GIT_URL ?= git://git.ipxe.org/ipxe.git
> -endif
> +IPXE_GIT_URL ?= https://github.com/ipxe/ipxe.git
>  
>  # put an updated tar.gz on xenbits after changes to this variable
>  IPXE_GIT_TAG := 3c040ad387099483102708bb1839110bc788cefb
> -- 
> Sincerely,
> Demi Marie Obenour (she/her/hers)
> Invisible Things Lab
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature