Re: [Xen-devel] [RFC] Documentation formats, licenses and file system structure

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 11 Oct 2019, Lars Kurth wrote:
> On 11/10/2019, 09:32, "Jan Beulich" <jbeulich@xxxxxxxx> wrote:
> 
>     On 10.10.2019 20:30, Lars Kurth wrote:
>     > On 10/10/2019, 18:05, "Andrew Cooper" <Andrew.Cooper3@xxxxxxxxxx> wrote:
>     >     On 10/10/2019 13:34, Lars Kurth wrote:
>     >     > Existing formats and licenses
>     >     > * Hypercall ABI Documentation generated from Xen public headers
>     >     > Format: kerndoc
>     >     > License: typically BSD-3-Clause (documentation is generated from 
> public headers)
>     >     
>     >     Its homegrown markup&perl, superimposed on what used to be doxygen 
> in
>     >     the past.
>     > 
>     > Oh, I forgot
>     >     
>     >     I wasn't planning on reusing any of the markup, and wasn't 
> expecting to
>     >     use much of the text either.  I'm still considering the option of
>     >     defining that xen/public/* isn't the canonical description of the 
> ABI,
>     >     because C is the wrong tool for the job.
>     >     
>     >     Its fine to provide a C set of headers implementing an ABI, but 
> there is
>     >     a very deliberate reason why the canonical migration v2 spec is in a
>     >     text document.
>     > 
>     > @Stefano: as you and I believe Brian will be spending time on improving 
> the
>     > ABI docs, I think we need to build some agreement here on what/how
>     > to do it. I was assuming that generally the consensus was to have
>     > docs close to the code in source, but this does not seem to be the case.
>     
>     Well, for migration v2 having the spec in a text file seems sensible
>     to me. For the public ABI, however, I think it's more helpful to have
>     the doc next to the actual definitions. Considering the possible use
>     of languages other than C I can certainly see why separating both
>     would be even more clean, but I think here we want to weigh practical
>     purposes against cleanliness.
> 
> I think that is an area where we need to build some consensus. The problem
> falls under what is considered "traceability" in safety speak: in other 
> words, 
> for the ABI documentation use-case it must be easy to be able to
> keep documentation and code in sync. And ideally, we would be able to
> check this automatically somehow, or have a bot provide hints such as 
> "You changed XYZ and should have a look and check whether ABC needs
> changing also".
> 
> I have thought about the problem of "traceability" for some time, which
> goes far beyond what we need for this use-case. Typical things that need
> to be maintained for a "traceable (safety) documentation set" are
> 
> ## Keeping key docs and code in sync 
> The use-cases here are things such as
> - keep man pages and xl sources in sync
> - keep ABI docs and headers in sync
> - keep documents such as the migration b2 spec in sync with
>   actual source
>  
> This is a problem we already have today and where we do this often
> fairly poorly manually (as can be seen on how out-of-date
> man pages often are)
> 
> Possible solutions for this are
> - store docs alongside headers (maybe using the same base
> file name) => that would work for ABI docs
> 
> - have some tagging or meta-information scheme which links
> specific source files to docs files => that would work for most
> other docs (albeit not always perfectly - e.g. when functionality
> is spread over many files and just portions of them)
> 
> For example: tools/xl/xl_cmdtable.c  
> is linked to files in docs/man/xl*
> 
> This means creating a bot/tool which warns that when you change
> foo.c to also look at foo.rst and/or ../../docs/.../bar.rst should be
> relatively straightforward. It would require some initial effort
> to create initial mappings, but these would never really change,
> unless we refactor code significantly.
>      
> ## Keeping dependent documents (or portions of documents) in sync
> This is something we have not really faced, because we do not
> have a lot of docs.  
> 
> In a large documentation set having the right chapter/tree
> structure enables this. In waterfall software engineering
> models, where you start off with high-level documents/
> requirements/specs/etc. documents which become
> increasingly detailed this is done through a chapter/tree
> structure, with the capability to make separate documents
> (or sections thereof) on other documents (or sections
> thereof). When you change something, a tool such as DOORS 
> forces you review and sign off child documents.
> 
> This is conceptually similar to what we would need for
> "linking" sources to docs as outlined above, only that
> the "linking" is between docs. It would also be easy enough
> to check and highlight, what else may have to be looked at.  
> 
> ## Proving that your tests verify design claims and that *all claims* are 
> tested
> This is typically the hardest problem to solve. It requires
> for test cases (be it a document or actual code) to
> link to claims (in a design, architecture, spec, ...)
> and to prove that all of them are tested.
> 
> If there is linkage capability, then it is straightforward
> to verify automatically that all your branches have
> test-case leaves in your documentation tree. But at least
> in a safety context you would also have to augment this
> with code coverage information. 
> 
> But for the purpose of this discussion, this use-case
> does not add any extra linkage capability other than
> being able to tell whether a piece of doc/code
> is a test case.
> 
> ## In summary
> I feel that having some basic linkage mechanism    
> between key documents and key documents of
> code is generally valuable for the community as a
> whole, if we can tie to some checking into it.
> 
> For rst docs we already have linkage via the
> ".. toctree::" directives, which may be entirely
> sufficient. But we don't have linkage from source
> to docs. 
> 
> Building some checking around this should not
> be too hard, in particular in light of the CIv2
> checking we agreed (but is somewhat stalled).
> 
> Enabling the same checks to be run locally (before)
> sending a patch, should also not be too hard.
> 
> I am just raising ideas at this stage. Any opinions
> are highly welcome.

As long as the linking between code and docs is done with a simple
unobtrusive tag, I think it is a good idea. Once we have a proper
checkpatch script in the tree, we could imagine adding a check to
it to see whether any docs need updating.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel