gitlab-docker-machine-oyster failure, Was: [PATCH v6 00/25] xl / libxl: named PCI pass-through devices

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

Hi Doug,

After chatting with Wei on IRC, it became obvious that the issue is that
gitlab-docker-machine-oyster failed, see its grey status here under
"Runners":

https://gitlab.com/xen-project/patchew/xen/-/settings/ci_cd

Maybe it is just a matter of rebooting the VM? Doug, could you give it a
try?

Thank you!

Cheers,

Stefano



On Thu, 10 Dec 2020, Wei Liu wrote:
> On Wed, Dec 09, 2020 at 06:41:03PM -0800, Stefano Stabellini wrote:
> > On Wed, 9 Dec 2020, Stefano Stabellini wrote:
> > > On Wed, 9 Dec 2020, Wei Liu wrote:
> > > > On Tue, Dec 08, 2020 at 05:02:50PM -0800, Stefano Stabellini wrote:
> > > > > The pipeline failed because the "fedora-gcc-debug" build failed with a
> > > > > timeout: 
> > > > > 
> > > > > ERROR: Job failed: execution took longer than 1h0m0s seconds
> > > > > 
> > > > > given that all the other jobs passed (including the other Fedora 
> > > > > job), I
> > > > > take this failed because the gitlab-ci x86 runners were overloaded?
> > > > > 
> > > > 
> > > > The CI system is configured to auto-scale as the number of jobs grows.
> > > > The limit is set to 10 (VMs) at the moment.
> > > > 
> > > > https://gitlab.com/xen-project/xen-gitlab-ci/-/commit/832bfd72ea3a227283bf3df88b418a9aae95a5a4
> > > > 
> > > > I haven't looked at the log, but the number of build jobs looks rather
> > > > larger than when we get started. Maybe the limit of 10 is not good
> > > > enough?
> > > 
> > > Interesting! That's only for the x86 runners, not the ARM runners (we
> > > only have 1 ARM64 runner), is that right?
> > > 
> > > If we could increase the number of VMs for x86 I think that would be
> > > helpful because we have very many x86 jobs.
> > 
> > I don't know what is going on but at the moment there seems to be only
> > one x86 build active
> > (https://gitlab.com/xen-project/patchew/xen/-/pipelines/227280736).
> > Should there be at least 3 of them?
> 
> Not sure what you meant here. That pipeline is green.
> 
> It may take some time for the CI to scale up if it is "cold". By default
> there is only 1 standby runner to reduce cost.