Re: [Xen-devel] automation: Creating a patchwork instance to improve pre-commit build testing

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 07/24/2018 04:26 PM, George Dunlap wrote:
> On 07/24/2018 12:23 PM, Lars Kurth wrote:
>>
>> On 24/07/2018, 11:50, "Julien Grall" <julien.grall@xxxxxxx> wrote:
>>
>>     Hi Lars,
>>     
>>     On 24/07/18 11:33, Lars Kurth wrote:
>>     > 
>>     > On 24/07/2018, 11:19, "Wei Liu" <wei.liu2@xxxxxxxxxx> wrote:
>>     >      On Tue, Jul 24, 2018 at 04:04:05AM -0600, Jan Beulich wrote:
>>     >      > I'm afraid my personal bar for any such automation is pretty
>>     >      > high: There must not ever be any negative effect from such an
>>     >      > addition. Positive effects would of course be very welcome. I
>>     >      > realize this is an unrealistic goal, but it should at least come
>>     >      > close (perhaps after some initial learning phase). But this 
>> implies
>>     >      > that at least in theory it is possible to come close in the 
>> first
>>     >      > place, which I can't take for given with the information I've 
>> been
>>     >      > provided so far.
>>     >      
>>     >      Then I'm afraid the only suggestion I get for you at the moment 
>> is to
>>     >      add a filter to dump those emails to /dev/null -- you already 
>> realised
>>     >      that's an unrealistic goal (at least at the beginning).
>>     >      
>>     >      Wei.
>>     >      
>>     > First of all, there should only be mail (aka spam) if there was a 
>> failure.
>>     
>>     This seems a little strange to only send e-mail on failure. How do you 
>>     differentiate between the bot has successfully tested that series and 
>>     the series is still in queue then?
>>     
>> Yes, that would be a trade-off to minimize "spam"
>>
>> It seems to me there are a number of options we have and thus some decisions
>> that need to be made.
>>  
>> 1: Do we trigger a CI cycle for *every* patch?
> 
> In a world with infinite resources, yes, because we want to detect
> broken bisections.  My guess is that this would be too
> resource-intensive for the real world.
> 
> No matter what, I'd prefer only one email per series; Definitely *don't*
> want a success email for every patch.

What about having "check-bisectability" as a separate test?  Rather than
doing a full build test from a clean tree for every possible distro, we
could do something like

for patch in $patches; do
  patch -p1 < $patch
  make
done

That should catch most bisection-breaking issues without being overly
resource-intensive.

From the CI's perspective, you'd be running on the whole series, and
check-bisectability would be a single sub-test.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel