Re: [Xen-API] Xapi Project 2.0 release

[Jon Ludlam <jjl25@xxxxxxxxx>]

On 02/06/14 17:11, George Shuklin wrote:
> On 06/02/2014 06:26 PM, Jon Ludlam wrote:
>> On 02/06/14 14:13, George Shuklin wrote:
>>> I don't want to sound to skeptical, but after about 5 year of deep
>>> work with xapi, I was really disappointed by Citrix VS opensource
>>> relationship. XenServer was and is not libre. There is source code but
>>> there is no way to rebuild original ISO from those sources. Decisions
>>> were made purely in-house sole by Citrix, and published opensource
>>> version (xapi packages) were broken beyond repair (and were removed
>>> from repository).
>>>
>> Yes, building the ISO was and still is awkward for non-Citrix people.
>> This isn't a release of the ISO though, this is just a release of some
>> of the tools on the ISO, and the biggest focus is on making it work
>> _properly_ outside the context of XenServer. As Dave mentioned,
>> buildroot is proof that the source can work in both a Debian-like and
>> CentOS-like environment.
> All problems with xapi and Xenserver is that XenServer is 'tightly
> build' bunch of scripts and patched programs, outside of xapi. If you
> looks to execution path for real operations with VMs, you can see,
> that right after nice and cool ocaml code in xapi, there is an
> extremely strange code on python ( os.popen('ls') for the file list is
> strong example of 'strange'). But right after python code
> (/opt/xensource/sm) there is much much more problematic set of shell
> scripts for many host-wide operations (/opt/xensource/scripts).
>
> It is not core part and it was very poorly maintained outside
> XenServer. I filed more than 10 bugs of completely broken functions
> (like host rename) - they are clearly no 'xapi', but xapi without them
> is cripple.
>
> if this has been changed, it really nice. I really hope so.
>

This is interesting. We've been working under the assumption that bits
of the API just won't work when you're not in a XenServer context - for
example, the patch management APIs - they just wouldn't make sense.
However, the VM, storage and most of the networking APIs are intended to
be first class and should work correctly. Then there's a grey area where
it's not clear whether people will expect things to work or not - host
rename is a good example. Should it work?


>>> Does something really changed here? I see lot of problems in xapi VS
>>> community relationship and Citrix is kinda not opensource company
>>> (Presentation Server, World of Windows and so on). Xensource was bit
>>> unexpected addition to this tight and cozy wold of Citrix and
>>> Microsoft, and source publication is not made xapi libre software.
>>> Only 'open source', but not libre.
>>>
>> Yes, something has changed. For example, the problem back when we got
>> xapi into Debian was that we had to get _much_ too involved in the
>> packaging process - it wasn't simply a case of grabbing the source
>> tarballs, building and packaging them - it effectively meant that a few
>> of us had to spend several months making the thing work at all, and
>> those patches ended up in the debian source package, rotting gently
>> while the master branch moved on. There was no way back then that xapi
>> could ever have been packaged up from source by anyone other than us.
>>
>> This is _very_ different from now. We've spent a long time splitting
>> repositories up, adding standard build system files, removing hard-coded
>> paths, splitting things up into more sensible smaller chunks and
>> generalizing the code. We've split two massive monolithic repositories
>> (xen-api and xen-api-libs) into about 30 smaller more sensible ones. By
>> the time of the release, the installation workflow should for these
>> individual repositories should simply be 'clone from github', then
>> 'configure', 'make', 'make install', and you should be there, and I
>> think it's entirely reasonable that someone with a working Xen
>> installation could package it up successfully.
>>
>> Of course, simply building is not enough. The buildroot repository
>> demonstrates that, with a minimum of patching (we've still got patches
>> for xcp-sm, vncterm and opasswd, but these will be fixed before the
>> release) viable packages can be created that work well enough to run VMs
>> to OpenStack's satisfaction.
>>
>> I'm aware that this doesn't address all of your concerns. But I believe
>> it's a really good first step, and hopefully our next steps will all go
>> in the same direction too :-)
>>
> Is buildroot cover tests for shell scripts? If you compare neutron ovs
> plugin code to xensource scripts - ovs got very neat tests and
> constant configuration checking. Xensource scripts do not check
> configuration, they blindly change configuration with expectation of
> perfectly sane environment (inside OVS config). This is on of 'tighly
> build' part of xenserver, which cause huge pain in case of slightest
> changes in host environment.
>
Buildroot is just example packaging configuration. The tests are from
OpenStack, and are tests for the functionality OpenStack wants.
> I don't want to be rude, but xapi is too  'api-centric' and just
> ignore all 'dirty' (in CS meaning of 'dirty') operations like disk
> initialization, volume manipulation and so on. And it passes all those
> operations to 'dirty' languages like python and bash to handle dirty
> work. And they do it dirty (pun intended).
>
Which isn't entirely unreasonable - it at least gives an opportunity to
make quick 'dirty' fixes when the environment has changed.

You're dead right that XenServer is, and has always been, a 'tightly
coupled' system. It's an embedded system, and that's the mindset; it has
always had complete control of the system. However, it would be a
mistake to try and make that work in a general linux environment. The
bits I would like to see work are the sorts of things that Xen
Orchestra, CloudStack, OpenStack, Vagrant,  & co all want to do -
install, start & stop VMs, configure their networking, snapshot their
disks, migrate and so on. Part of that is indeed fixing things like the
network and storage scripts so that they are more tolerant and careful,
and there is definitely work to do on that front. We've got a start, for
example 'ffs', a storage backend that is much easier to run on an
already-installed system, but there's still plenty of work to do. The
intention here is first to make the master branches at least work, so
that any work to make them work _well_ can be easily upstreamed.

Jon


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api