Re: [XEN PATCH] tools: Have flex and bison mandatory

[Elliott Mitchell <ehem+xen@xxxxxxx>]

On Wed, Feb 08, 2023 at 09:02:35AM +0100, Jan Beulich wrote:
> On 07.02.2023 19:02, Anthony PERARD wrote:
> > 
> > I think the issue that Andrew saw in the CI with "libxlu_cfg_y.o"
> > failing to build while "libxlu_cfg_l.[ch]" are been regenerated is
> > probably because make doesn't consider "libxlu_cfg_l.[ch]" as a group of
> > targets.
> > 
> > If for some reason we have:
> >     libxlu_cfg_l.h newer than libxlu_cfg_l.l newer than libxlu_cfg_l.c
> > 
> > Then make seems to take two parallel decision:
> >     When building libxlu_cfg_y.o:
> >         libxlu_cfg_l.h is newer than libxlu_cfg_l.l
> >         -> libxlu_cfg_l.h is up-to-date, start building libxlu_cfg_y.o
> >     When building libxlu_cfg_l.o:
> >         libxlu_cfg_l.c is older than libxlu_cfg_l.l
> >         -> we need to generate libxlu_cfg_l.c first
> > Then, libxlu_cfg_y.o fails to build because libxlu_cfg_l.h is been
> > updated do to the parallel build of libxlu_cfg_l.o.
> > 
> > I can easily reproduce the issue with:
> >     touch libxlu_cfg_l.c; sleep .1; touch libxlu_cfg_l.l; sleep .1;
> >     touch libxlu_cfg_l.h; sleep .1; make -j
> > And having "sleep 1" in "%.c %.h: %.l" recipe, between `rm` and `flex`.
> 
> Interesting. In my somewhat similar experiments I found that an issue
> exists only when one of the .c files is missing. Things looked to work
> properly when both were there, but one was outdated. Clearly that's
> yet more surprising behavior than, as you appear to observe, the issue
> simply being tied too "needs updating".

The actual problem is flex and bison output both files at the same time.
One process sees all of its prerequisites are up to do and therefore
proceeds to the next step.  A second process sees an out of date
prerequisite, and starts to run flex which wipes out the prerequisite of
the first process.


Crucially libxlu_cfg_y.o does NOT depend on libxlu_cfg_l.c, therefore GNU
Make is allowed to proceed with building libxlu_cfg_y.o even if
libxlu_cfg_l.c is out of date.

The problem is the rule to update libxlu_cfg_l.c ALSO updates
libxlu_cfg_l.h as a side-effect.


As such libxlu_cfg_y.o needs an order-only dependency (assuming GNU Make
syntax is allowed) on libxlu_cfg_l.c, so the two rules DON'T run in
parallel and cause havoc.


> > I don't know how to properly fix this yet.
> > Event writing "%.c %.h &: %.l" doesn't work, with make 4.3, which is
> > supposed to be grouped targets. But that's is maybe fixed in 4.4.
> 
> Well, in my tests while 4.4 has fixed one issue with intermediate files
> affecting the behavior here, the issue we're possibly running into is
> still there. I've sent an inquiry to bug-make@xxxxxxx.

I do not believe this is a bug in GNU Make.  Merely double-output rules
cause precisely this sort of trouble in Make when run in parallel.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@xxxxxxx  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445