Re: [Xen-devel] [OSSTEST PATCH 00/13] Speed up and restore host history

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

On 08/11/2019 19:49, Ian Jackson wrote:
> Earlier this week we discovered that sg-report-host-history was running
> extremely slowly.  We applied an emergency fix 0fa72b13f5af
>   sg-report-host-history: Reduce limit from 2000 to 200
> 
> The main problem is that sg-report-host-history runs once for each
> flight, and must generate a relevant history view of the recent
> history for each host - including much history that is already in the
> old version of the html file.
> 
> The slow part is asking the database about information about each job,
> including its final step, allocation step, etc.  (The main query which
> digs out relevant jobs is also rather time consuming it runs all in
> one go and takes only a minute or two.)
> 
> In this series we introduce a mechanism which caches much of the
> historical analysis.
> 
> It is not straightforward to reuse old html data as-is because we
> would have to do a merge sort with the new data and that would involve
> rewriting the alternating background colour (!)
> 
> So instead, we stuff the information we got from the database into
> comments in the HTML, which we can then scan on future runs.

Not mend to bike shed, so just for consideration:
- Have you considered (inline) css for the background colouring, or does
  it have to be html only  ?
- And for caching perhaps a materialized view with aggregated data only
  refreshed at a more convient time could perhaps help at the database
  level ?

--
Sander

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel