Re: ocamldoc generation and packed files

[Thomas Gazagnaire <thomas.gazagnaire@xxxxxxxxx>]

Well, actually it's seems quite easy to do, as the packed .annot file contains 
already the right location information. I can try to work out on a 
quick-and-dirty tool to fix your issue :-)

 
On Aug 19, 2011, at 1:21 PM, Thomas Gazagnaire wrote:

> I don't have concrete available solution for the .annot files. The two I can 
> see are :
> 
> * releasing the .ml -> .annot part of the compiler as a stand-alone tool and 
> use it to generate individual .annot files (I guess that's not very different 
> from running ocamlc -annot on every file)
> * having a tool to split out a packed .annot file into individual .annot 
> files. I guess it's not so difficult to do (just need to parse the directive 
> lines in the packed source file + its annot file and do some comparison), but 
> we don't have it yet.
> 
> Thomas
> 
> On Aug 19, 2011, at 11:58 AM, Anil Madhavapeddy wrote:
> 
>> As a followup to this, I've integrated ocaml-ocp into my tree and it's 
>> working really well. Thanks for releasing this Thomas!  There was an 
>> off-by-one in the cpp directives for line numbering (fixed in our tree).
>> 
>> It would be very useful to have some way for .annot files to work without 
>> compiling the individual sub-files; any thoughts on how this might work?
>> 
>> Anil
>> 
>> On 13 Aug 2011, at 20:39, Anil Madhavapeddy wrote:
>> 
>>> Sure, go for it if you get a chance.
>>> 
>>> The current packing "works", but the doc target cannot be built at the same 
>>> time as the normal code (because it causes the source pack to leave .ml 
>>> files in lib/std/foo.ml, which then causes the lib/std/foo.cm{xo} to be 
>>> rebuilt instead of the -packed copied .cmx to be used).
>>> 
>>> Basically, its a bit of a mess :-)
>>> 
>>> As a sneak preview, check out http://github.com/avsm/mirage-browser.git (on 
>>> index.html). This is a *very* rough jQuery-based module browser of the 
>>> Mirage OCamldoc that I'd like to have live on the website before the CUFP 
>>> tutorial.  The live search is pretty nice, and I'll flesh out the rendering 
>>> over the next few days.
>>> 
>>> This tool is also quite standalone as it uses the output of 
>>> odoc_json/ocamldoc, and so could be applied to the standard distribution 
>>> and also to the Citrix xapi-libs (if you're interested, David). 
>>> 
>>> Killer feature: the js_of_ocaml interactive top-level should work in 
>>> here... :-)
>>> 
>>> Anil
>>> 
>>> On 13 Aug 2011, at 17:50, Thomas Gazagnaire wrote:
>>> 
>>>> No ocamlbuild integration yet but would be definitely useful to add. I
>>>> can have a look at integrating it to mirage build next week if it's
>>>> useful.
>>>> 
>>>> --
>>>> Thomas
>>>> 
>>>> 2011/8/13 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>>> Thanks Thomas, that looks very useful and definitely better than the 
>>>>> script in tree at the moment.
>>>>> 
>>>>> One question: ocp-pack also requires that the list of files be passed in 
>>>>> dependency order, or else the resulting pack file will not compile.  Have 
>>>>> you tried to integrate it as an ocamlbuild rule, so that it can 
>>>>> automatically sort the input modules before generates the packed file / 
>>>>> functor?
>>>>> 
>>>>> Anil
>>>>> 
>>>>> On 12 Aug 2011, at 20:01, Thomas Gazagnaire wrote:
>>>>> 
>>>>>> It will be anounced quite shortly on the ocaml mailing list, so I can
>>>>>> give you the link to ocp-pack :
>>>>>> 
>>>>>> http://www.ocamlpro.com/code/2011-08-10-ocaml-pack-functors.html
>>>>>> http://www.ocamlpro.com/files/ocp-pack-1.0.0.tar.gz
>>>>>> 
>>>>>> It should help you to pack modules easily and have a working doc.
>>>>>> 
>>>>>> --
>>>>>> Thomas
>>>>>> 
>>>>>> 2011/7/28 Anil Madhavapeddy <anil@xxxxxxxxxx>:
>>>>>>> That tool will be very useful! I've committed the 'make doc' target. 
>>>>>>> The output is still pretty dirty as we don't use the documentation tags 
>>>>>>> properly, but I'll go through adding .mli files and adding proper 
>>>>>>> documentation on the more stable interfaces.
>>>>>>> 
>>>>>>> It would be quite nice to eventually have a single documentation output 
>>>>>>> for all the backends, with an addition section saying 'only present in 
>>>>>>> Xen' or 'only present in Node'. I think that should be possible by 
>>>>>>> parsing the ocamldoc dump outputs, but something for the future!
>>>>>>> 
>>>>>>> Anil
>>>>>>> 
>>>>>>> On 27 Jul 2011, at 16:56, Thomas Gazagnaire wrote:
>>>>>>> 
>>>>>>>> We have as well a program which pack together ML files (and is able to 
>>>>>>>> functorize packs as well...) it is not released yet, but I guess we 
>>>>>>>> can open-source it shortly.
>>>>>>>> 
>>>>>>>> But I think overriding the default rule for ocamldoc+pack in 
>>>>>>>> ocamlbuild is sufficient for now on so you should push your patch :-)
>>>>>>>> 
>>>>>>>> Thomas
>>>>>>>> 
>>>>>>>> On Jul 27, 2011, at 5:36 PM, Anil Madhavapeddy wrote:
>>>>>>>> 
>>>>>>>>> I'd really like to generate ocamldoc (HTML/PDF) of all the various 
>>>>>>>>> libraries so that it's easier to learn Mirage (and support editor 
>>>>>>>>> auto-completion, etc).
>>>>>>>>> 
>>>>>>>>> The big problem is that ocamldoc doesn't support packed modules, and 
>>>>>>>>> we use packing quite extensively (in Net, Http, Block, etc).
>>>>>>>>> 
>>>>>>>>> So I've hacked up an ocamlbuild target that concats together the 
>>>>>>>>> *source* ML files from an .mlpack and uses that to generate the 
>>>>>>>>> ocamldoc for the standard library, with one set of HTML files 
>>>>>>>>> generated per backend (Xen, Net-Direct, Net-Socket, and so on).
>>>>>>>>> 
>>>>>>>>> The issue is that these rules are a little grim: everywhere where we 
>>>>>>>>> have a .mlpack file at the moment, we need to override that rule to 
>>>>>>>>> generate a concatenated ML file that is used for ocamldoc (but not 
>>>>>>>>> for actual compilation, since line numbers get lost since those 
>>>>>>>>> aren't preserved when converting from ML files into a single big one).
>>>>>>>>> 
>>>>>>>>> Does anyone have a better solution for ocamldoc and packed files? All 
>>>>>>>>> of the grimness here is hidden away in the ocamlbuild rules, so I'm 
>>>>>>>>> inclined to just commit this patch, and perhaps see about adding 
>>>>>>>>> -pack support into ocamldoc at a later stage (there's an open bug in 
>>>>>>>>> Mantis somewhere).
>>>>>>>>> 
>>>>>>>>> Dave, do you use pack in XAPI, and/or ocamldoc?  I wonder if everyone 
>>>>>>>>> else (like Core) also have their own swanky 'cat ML files into one' 
>>>>>>>>> script too...
>>>>>>>>> 
>>>>>>>>> -anil
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
>