Re: [PATCH v1 3/3] automation: allow selecting individual jobs via CI variables

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 14 Feb 2025, Marek Marczykowski-Górecki wrote:
> On Thu, Feb 13, 2025 at 05:36:47PM -0800, Stefano Stabellini wrote:
> > On Thu, 13 Feb 2025, Marek Marczykowski-Górecki wrote:
> > > Debugging sometimes involves running specific jobs on different
> > > versions. It's useful to easily avoid running all of the not interesting
> > > ones (for given case) to save both time and CI resources. Doing so used
> > > to require changing the yaml files, usually in several places.
> > > Ease this step by adding SELECTED_JOBS_ONLY variable that takes a regex.
> > > Note that one needs to satisfy job dependencies on their own (for
> > > example if a test job needs a build job, that specific build job
> > > needs to be included too).
> > > 
> > > The variable can be specified via Gitlab web UI when scheduling a
> > > pipeline, but it can be also set when doing git push directly:
> > > 
> > >     git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/"
> > > 
> > > More details at 
> > > https://docs.gitlab.co.jp/ee/user/project/push_options.html
> > > 
> > > The variable needs to include regex for selecting jobs, including
> > > enclosing slashes.
> > > A coma/space separated list of jobs to select would be friendlier UX,
> > > but unfortunately that is not supported:
> > > https://gitlab.com/gitlab-org/gitlab/-/issues/209904 (note the proposed
> > > workaround doesn't work for job-level CI_JOB_NAME).
> > > On the other hand, the regex is more flexible (one can select for
> > > example all arm32 jobs).
> > 
> > I was trying to find workarounds so that we could also support the
> > simple list of comma-separated jobs you mentioned, but I couldn't find
> > an easy way to do that.
> > 
> > However, one thing we can do is to support writing SELECTED_JOBS_ONLY in
> > .gitlab-ci.yml as a commit in xen.git, for people that don't know or
> > don't remember how to set ci.variable using the git command line.
> 
> You can always do it, in `variables` setting AFAIR.
> 
> > Given that this is for testing, I think it would be no problem to adding
> > a special commit on top of your tree. We are just trying to make it
> > easier compared to having to manually delete the list of jobs we don't
> > need.
> 
> Yes, manually delete was awful. In practice I usually added always-false
> rules, but still.
> 
> > But even with the special commit, I couldn't think of an easy way to
> > achieve the nicer comma-separated list of jobs...
> > 
> > 
> > > Signed-off-by: Marek Marczykowski-Górecki 
> > > <marmarek@xxxxxxxxxxxxxxxxxxxxxx>
> > > ---
> > > This probably wants documenting beyond this commit message. I don't
> > > think we have any CI-related docs anywhere, do we? Some new file in
> > > docs/misc?
> > 
> > Yes please :-)
> > 
> > It would be also worth documenting how to create a simple pipeline by
> > removing everything that you don't need for a single test
> 
> You mean how to find what jobs you need?
> 
> > > And also, it's possible to extend web ui for starting pipelines to
> > > include pre-defined variables. I use it in qubes here if you want to
> > > see:
> > > https://gitlab.com/QubesOS/qubes-continuous-integration/-/pipelines/new
> > 
> > I don't have access to this
> 
> Oh, sorry. Screenshot attached.
> 
> And its definition looks like this:
> https://gitlab.com/QubesOS/qubes-continuous-integration/-/blob/main/.gitlab-ci.yml?ref_type=heads#L15-26

Actually this gave me an idea. What if we flip the default? Normally we
want to run all the jobs.

When doing development, we typically want to run one specific job
attached to the work we are currently doing. This patch introduces a
blacklist, but it looks like we want a whitelist instead?

Wouldn't it be easier to say: "delete everything except for
adl-smoke-x86-64-dom0pvh-hvm-gcc-debug"

Then we can arrange it so adl-smoke-x86-64-dom0pvh-hvm-gcc-debug and its
dependencies are left enabled. Everything else is disabled?