Re: [Xen-devel] [PATCH v3] firmware/shim : filter output files during Xen tree setup

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 30.07.18 at 03:22, <christopher.w.clark@xxxxxxxxx> wrote:
> On Fri, Jul 27, 2018 at 1:34 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
> 
>> >>> On 26.07.18 at 23:16, <christopher.w.clark@xxxxxxxxx> wrote:
>> > Exclude named output files from the Xen tree setup.
>> >
>> > The linkfarm.stamp content will differ between top level "make"
>> > and "make install" invocations, due to the introduction of these
>> > output files that are produced during the "make" build.
>> >
>> > Filter these out to prevent an unnecessary rebuild of the shim
>> > during "make install", after "make" within a fresh source tree.
>> >
>> > Excluded from consideration with this change: differences in stamp
>> > content when performing incremental builds in an existing tree.
>>
>> I don't understand this (as well as you most recent remark on the
>> v2 thread): The "make install" invocation _is_ an incremental
>> rebuild. Hence I don't understand how excluding some but not all
>> generated files helps. But I'm not going to exclude that this is
>> simply because I don't understand well enough the logic in
>> xen-dir/Makefile when to trigger a rebuild.
>>
>>
>>
> OK, so "make install" is considered to be an incremental rebuild here -
> which makes sense - because in the implementation, it causes almost the
> same actions, within the same tree, as the preceding "make":
> 
> "make" has the default target of all, which only depends on dist, dist
> depends on the dist-<COMPONENT>s, and those each depend on
> install-<COMPONENT>s. "make install" depends on install-<COMPONENT>s. A
> main difference with "make install" is the destination directories that are
> populated with the output.
> 
> Here's what's actually going on: it's parallel builds that are not working
> correctly ("make -j <N>", where N>1). When multiple threads are available
> to build with, the tools build starts before the xen subsystem build
> finishes, so the first contents of linkfarm.stamp is a snapshot of a moving
> tree, and it ends up missing the final xen subsystem build products.
> 
> The reason the v3 patch (mostly) works is that it lists the later build
> products of the xen subsystem. The earlier ones, such as the headers, are
> added to the xen tree long enough before the tools build gets started, so
> they are witnessed and captured in the linkfarm.stamp file. I say the v3
> patch mostly works, because I've found that its short exclusion list is
> missing "xen.gz.new", which is briefly present as a temporary file and can
> end up caught in the linkfarm.stamp file. It would need to be added to the
> exclusion list if continuing with that filtering approach.
> 
> An alternative approach is to serialize the xen and tools subsystem builds
> in the top level Makefile, not allowing the tools build to proceed until
> the xen build is complete. I don't currently have a patch to propose to do
> this. In the absence of that in place, the workaround is simple: build just
> the xen subsystem alone first.

Thanks for the thorough analysis. I don't think the goal should be to
restrict the two builds to happen one after the other. Instead, as said
before, the goal ought to be to disregard _all_ generated files when
producing the stamp file.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel