Xen project Mailing List

On Fri, Jul 27, 2018 at 1:34 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:

>>> On 26.07.18 at 23:16, <christopher.w.clark@xxxxxxxxx> wrote:
> Exclude named output files from the Xen tree setup.
>
> The linkfarm.stamp content will differ between top level "make"
> and "make install" invocations, due to the introduction of these
> output files that are produced during the "make" build.
>
> Filter these out to prevent an unnecessary rebuild of the shim
> during "make install", after "make" within a fresh source tree.
>
> Excluded from consideration with this change: differences in stamp
> content when performing incremental builds in an existing tree.

I don't understand this (as well as you most recent remark on the
v2 thread): The "make install" invocation _is_ an incremental
rebuild. Hence I don't understand how excluding some but not all
generated files helps. But I'm not going to exclude that this is
simply because I don't understand well enough the logic in
xen-dir/Makefile when to trigger a rebuild.

OK, so "make install" is considered to be an incremental rebuild here - which makes sense - because in the implementation, it causes almost the same actions, within the same tree, as the preceding "make":

"make" has the default target of all, which only depends on dist, dist depends on the dist-<COMPONENT>s, and those each depend on install-<COMPONENT>s. "make install" depends on install-<COMPONENT>s. A main difference with "make install" is the destination directories that are populated with the output.

Here's what's actually going on: it's parallel builds that are not working correctly ("make -j <N>", where N>1). When multiple threads are available to build with, the tools build starts before the xen subsystem build finishes, so the first contents of linkfarm.stamp is a snapshot of a moving tree, and it ends up missing the final xen subsystem build products.

The reason the v3 patch (mostly) works is that it lists the later build products of the xen subsystem. The earlier ones, such as the headers, are added to the xen tree long enough before the tools build gets started, so they are witnessed and captured in the linkfarm.stamp file. I say the v3 patch mostly works, because I've found that its short exclusion list is missing "xen.gz.new", which is briefly present as a temporary file and can end up caught in the linkfarm.stamp file. It would need to be added to the exclusion list if continuing with that filtering approach.

An alternative approach is to serialize the xen and tools subsystem builds in the top level Makefile, not allowing the tools build to proceed until the xen build is complete. I don't currently have a patch to propose to do this. In the absence of that in place, the workaround is simple: build just the xen subsystem alone first.

To demonstrate the behaviour described above:

===

With a single-threaded build, which will behave correctly:

make

cp ./tools/firmware/xen-dir/linkfarm.stamp ../linkfarm.stamp.compile

make install

diff -u ../linkfarm.stamp.compile ./tools/firmware/xen-dir/linkfarm.stamp

linkfarm.stamp.compile contents will be populated after the xen subsystem build is complete, so contents of linkfarm.stamp will not change when invoking "make install" and the diff will be empty.

===

With a multi-threaded build on an SMP machine (eg. an 8-way build host), which may show the defect:

JOBS="12"

make -j $JOBS

cp ./tools/firmware/xen-dir/linkfarm.stamp ../linkfarm.stamp.compile

make -j $JOBS install

diff -u ../linkfarm.stamp.compile ./tools/firmware/xen-dir/linkfarm.stamp

linkfarm.stamp.compile contents will likely be populated _before_ the xen subsystem build is complete, and if so the diff will _not_ be empty. The shim will then rebuild during the "install" build.

Since this is racey, different results are visible with different numbers of threads and different load on the machine. In testing, in one test 40 threads actually built just as well as the single threaded case, but each of 2, 4, 8, 12, 16 threads showed different contents of the incomplete linkfarm.stamp files after the first compile.

===

Workaround to build with multiple threads and no patch applied: build the xen subsystem alone first:

make -j $JOBS xen

make -j $JOBS

make -j $JOBS install

===

Christopher

Re: [Xen-devel] [PATCH v3] firmware/shim : filter output files during Xen tree setup