Re: [Xen-devel] [GSoc] GSoc Introduction : Xen on ARM: create multiple guests from device tree

[Methuku Karthik <kmeth@xxxxxxxxxxxxxx>]

Hi Stefano,

I have used attached bin2c.c convert the contents of .config file to hex.

(echo "static const char xen_config_data[] __attribute__((used)) =
./bin2c.o <xen/.config ; echo ";") > xen_config.h

the above command will generate the header file which will contain the
contents of config file in hex form.

i wrote a tester code to check if i .config is output correctly.

any inputs on makefile modification would be of great help.

Will addition of below rules to makefile suffice
1.Add rule to compile bin2c.c always.
2.add the above mentioned shell command to make file to generate the
header file.
3. which object file should include the header file ?

I am thinking to add bin2c.c file in scripts folder, access .config
file from xen folder to give it as input to bin2.o. and generate
xen_config.h

As you mentioned that it would be nice to add hyper call, where should
the generated xen_config.h file be added and which file should have
the include?

I am attaching bin2c.c file and sample xen_config.h file, with tester
for your reference.

If the procedure seems fine , i will go ahead and change the makefile
accordingly and commit the changes.

On Tue, Apr 4, 2017 at 1:29 PM, Stefano Stabellini
<sstabellini@xxxxxxxxxx> wrote:
> One more thing: the deadline for microtasks (the patch submission below
> to fix XEN-38) is technically Monday the 10th of April, but in practice
> the patch needs to be sent this week to be able to follow up review
> comments appropriately. Ideally the patch should be already committed by
> Monday the 10th.
>
> On Mon, 3 Apr 2017, Methuku Karthik wrote:
>> Hi Stefano,
>>
>> I have asked questions in inline. Clarification below questions would really 
>> help me in contribution. Please look into the questions. I am highlighting 
>> them in this mail.
>>
>>  For example, Dom1 should be able to share a page with Dom2 and a
>>   different page with Dom3. It needs to be clear which page is shared with
>> which VM from the VM config files.
>>
>>
>> when we create vms using xl create , for example if i am planning create 
>> three VMs,
>>
>> Dom1, Dom2 and Dom3, because of the page sharing are we imposing any order of
>>   creating VMs.
>>
>>   I am asking this question to clarify this point, while creation of Dom1 if 
>> its
>>   sharing pages with Dom 2 and Dom 3 , should Xen already be aware of Dom2 
>> and Dom3?
>>
>>   I am referring to following links to understand about mem sharing.
>>
>>   
>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/tests/mem-sharing/memshrtoo
>>   l.c;h=8e5e22b9e95d91f1441d8eb226b64852eca075d5;hb=HEAD
>>   http://xenbits.xen.org/docs/unstable/misc/grant-tables.txt
>>
>>   I also want to figure out how domains are created and how xl tool parses 
>> the file
>>   and passes on the information to domain creation . Let me know if i am 
>> thinking in
>>   right direction.
>>
>>   suggest any resource or work which would help with designing config file 
>> options.
>>
>>
>>  I will start with Xen-38 that would help me in exploring init code. Correct 
>> me if i
>>   am wrong.
>>
>>   I have a few questions and clarifications before proceeding further. I 
>> have checked
>>   how config.gz file is generated in linux kernel source.
>>   In linux kernel sources, if CONFIG_IKCONFIG_PROC option is set, .conifg 
>> file which
>>   is generated after choosing options with lets say from make menuconfig  is 
>> read into
>>   a variable, this way its part of build.
>>
>>   during init time proc_create service is used to create this file config.gz.
>>   http://lxr.free-electrons.com/source/kernel/configs.c
>>
>>
>>   I guess i have to do something similar.
>>
>>   Questions :
>>
>>   1. When Xen is build using the make command, we effectively set 
>> XEN_COMPILE_ARCH,
>>   XEN_OS, XEN_TARGET which allow using corresponding .mk file from config 
>> folder.
>>   These variable in turn decide what are the config options. I wasnt able to 
>> find any
>>   .config. Please direct me to find the file or if i am missing something.
>>
>>   2. Where and how this config file should be accessible to  User once in 
>> Dom0. Is the
>>   xen folder created to keep the information about guest domains like proc 
>> for process
>>   in linux kernel ? Will that be suitable location to have config file.
>>
>>   3. if i assume that i will approach similarly, i have to add services to 
>> be called
>>   during init stage. As am not acquainted with code base, i could just grep 
>> with
>>   _start or _init or similar strings to find out initialization code. Any
>>   input(function name or filename) to look for will be of great help.
>>
>> On Mon, Apr 3, 2017 at 3:35 PM, Stefano Stabellini <sstabellini@xxxxxxxxxx> 
>> wrote:
>>       Thank you! I am looking forward to your contribution on the list! If 
>> you
>>       encounter any issues, please let us know.
>>
>>       The code contribution is more important, but if you find the time in 
>> the
>>       next few days, it would be nice to add more details to the
>>       implementation plan, such as where the memory gets allocated, whether 
>> it
>>       is taken from a VM, and if so, which one. Also what kind of "token"
>>       could be used in the config option and how the toolstack could keep
>>       track of the token - memory page references.
>>
>>       Thanks,
>>
>>       Stefano
>>
>>       On Mon, 3 Apr 2017, Methuku Karthik wrote:
>>       > Hi Stefano,
>>       >
>>       > Thanks for Input. I was not able to spend enough time last couple of 
>> weeks due to
>>       > projects. I have received mail from Lars Kurt explaining submission 
>> of draft
>>       > proposal and possibility to work on micro tasks.
>>       >
>>       > I have created a draft proposal from with your inputs and what i 
>> learnt about
>>       > sharing pages and memory management in Xen, please access it from 
>> here
>>       >
>>       > 
>> https://docs.google.com/document/d/1xLmR7x4yfCbRgpuefZQNhZ4lAu-6slW0oXPmjnxcnz0/edi
>>       > t#heading=h.1yvc35w6t3fu
>>       >
>>       > I haven't written anything about maintenance. I have included some 
>> links i thought
>>       > will be helpful under references and referenced wherever applicable.
>>       >
>>       > Please suggest comments and inputs.
>>       >
>>       > On Tue, Mar 28, 2017 at 8:12 PM, Stefano Stabellini 
>> <sstabellini@xxxxxxxxxx> wrote:
>>       > > CC'ing a couple of maintainers that might have more insights on 
>> this
>>       > > project.
>>       > >
>>       > > On Tue, 28 Mar 2017, Stefano Stabellini wrote:
>>       > >> On Tue, 28 Mar 2017, Methuku Karthik wrote:
>>       > >> > Hi Stefano,
>>       > >> >
>>       > >> > Kindly suggest me reading material that could help me 
>> understand and
>>       > >> > come up with the proposal.
>>       > >>
>>       > >> Sure! For the "Xen on ARM: create multiple guests from device 
>> tree"
>>       > >> project, the idea is that on many embedded systems the user knows 
>> how
>>       > >> many guests to create beforehand, and usually it is a small 
>> number, like
>>       > >> 2 or 3. Often these guests don't even have any PV frontends, but 
>> just a
>>       > >> set of devices assigned to them. An example could be an extremely 
>> simple
>>       > >> guest that only accesses one physical device (which is assigned 
>> to it at
>>       > >> boot) and prints messages using the debug hypercalls (see
>>       > >> xen/arch/arm/traps.c:do_debug_trap).
>>       > >>
>>       > >> In this scenario, there is no need to wait for Dom0 to boot to 
>> create
>>       > >> this second guest (I'll call it Dom1, to distinguish it from 
>> Dom0). Xen
>>       > >> could actually create it directly by itself, the same way it 
>> starts Dom0
>>       > >> (see xen/arch/arm/domain_build.c:construct_dom0 and
>>       > >> docs/misc/arm/device-tree/booting.txt).
>>       > >>
>>       > >> The project is about extending the existing device interface to 
>> pass an
>>       > >> additional kernel, initrd, command line arguments for the second 
>> virtual
>>       > >> machine. It would also need to include which devices should be 
>> assigned
>>       > >> to it. In response, Xen should build the second VM the same way 
>> it would
>>       > >> do normally when done via the toolstack (xl/libxl), but it would 
>> happen
>>       > >> at boot time, before Dom0 is fully up and running.
>>       > >>
>>       > >>
>>       > >>
>>       > >> For the "Share a page in memory from the VM config file" project,
>>       > >
>>       > > Ops, I forgot to add this description :-)
>>       > >
>>       > > I was saying, for the "Share a page in memory from the VM config 
>> file"
>>       > > project, the idea is that we want to share a page in memory 
>> between two
>>       > > VMs just by adding one line to their VM config files. They should 
>> be
>>       > > able to communicate with each others straight away by writing at 
>> the
>>       > > right address in memory. That way, even small embedded systems 
>> with no
>>       > > xenstore support can still setup a communication channel with each
>>       > > others.
>>       > >
>>       > > Fundamentally, it just requires the xl/libxl toolstack (see 
>> tools/xl and
>>       > > tools/libxl) to parse a new VM config file option, and in response 
>> share
>>       > > a page at the specified address, or map a page at a specified 
>> address.
>>       > > If the memory address is wrong, the toolstack needs to be able to 
>> handle
>>       > > the failure. Sharing pages and mapping pages in xl/libxl is easy; 
>> the
>>       > > difficulty of the project is coming up with the right parameter in 
>> the
>>       > > VM config file so that multiple VMs can share different pages with 
>> each
>>       > > others. It probably requires the introduction of a "token" to 
>> identify
>>       > > the page you want to share across multiple VM config files.
>>       > >
>>       > > For example, Dom1 should be able to share a page with Dom2 and a
>>       > > different page with Dom3. It needs to be clear which page is 
>> shared with
>>       > > which VM from the VM config files.
>>       > >
>>       > when we create vms using xl create , for example if i am planning 
>> create three VMs,
>>       >
>>       > Dom1, Dom2 and Dom3, because of the page sharing are we imposing any 
>> order of
>>       > creating VMs.
>>       >
>>       > I am asking this question to clarify this point, while creation of 
>> Dom1 if its
>>       > sharing pages with Dom 2 and Dom 3 , should Xen already be aware of 
>> Dom2 and Dom3?
>>       >
>>       > I am referring to following links to understand about mem sharing.
>>       >
>>       > 
>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/tests/mem-sharing/memshrtoo
>>       > l.c;h=8e5e22b9e95d91f1441d8eb226b64852eca075d5;hb=HEAD
>>       > http://xenbits.xen.org/docs/unstable/misc/grant-tables.txt
>>       >
>>       > I also want to figure out how domains are created and how xl tool 
>> parses the file
>>       > and passes on the information to domain creation . Let me know if i 
>> am thinking in
>>       > right direction.
>>       >
>>       > suggest any resource or work which would help with designing config 
>> file options.
>>       >
>>       > >
>>       > >
>>       > >> > On Tue, Mar 28, 2017 at 1:38 PM, Stefano Stabellini
>>       > >> > <sstabellini@xxxxxxxxxx> wrote:
>>       > >> > > I would also like to take the opportunity to remind you of 
>> the upcoming
>>       > >> > > deadline for applications submissions, which is the 3rd of 
>> April for
>>       > >> > > GSoC, see:
>>       > >> > >
>>       > >> > > http://marc.info/?l=xen-devel&m=149071502330534
>>       > >> > >
>>       > >> > > The GSoC application system is
>>       > >> > > https://summerofcode.withgoogle.com/get-started/.
>>       > >> > >
>>       > >> > > Please give a look at the Xen Project application template 
>> here:
>>       > >> > >
>>       > >> > > 
>> https://wiki.xenproject.org/wiki/GSoC_Student_Application_Template
>>       > >> > >
>>       > >> > > It also includes an "Implementation Plan", where you have the 
>> chance to
>>       > >> > > explain the implementation plan for the projects you would 
>> like to apply
>>       > >> > > for. You can apply for more than one project if you want to.
>>       > >> >
>>       > >> > Thanks for the reminder Stefano. I will post some questions 
>> about my
>>       > >> > implementation plan.
>>       > >> >
>>       > >> > I would like to spend my time fixing the bug but as deadline is
>>       > >> > approaching , I would like to know the bare essential expected 
>> in the
>>       > >> > proposal?
>>       > >>
>>       > >> It needs to include the basic milestones and how long you 
>> estimate it is
>>       > >> going to take to complete each of them. More details you add, more
>>       > >> you'll show your understanding of the problem.
>>       > >>
>>       > >>
>>       > >> > >
>>       > >> > > Thanks,
>>       > >> > >
>>       > >> > > Stefano
>>       > >> > >
>>       > >> > >
>>       > >> > > On Mon, 27 Mar 2017, Stefano Stabellini wrote:
>>       > >> > >> Hello Methuku,
>>       > >> > >>
>>       > >> > >> It is great to hear that you are interested in working on 
>> Xen on ARM.
>>       > >> > >> It's good that you already know how to build and install Xen 
>> on an ARM
>>       > >> > >> board.
>>       > >> > >>
>>       > >> > >> Do you have serial access to your Nvidia Jetson TK1? If not, 
>> I would
>>       > >> > >> suggest to setup another test environment based on a 
>> different board, or
>>       > >> > >> simply use the ARM Foundation Platform which is a free (as 
>> in beer)
>>       > >> > >> software emulator.
>>       > >> > >>
>>       > >> > >> In terms of bug fixes, you could give a look at
>>       > >> > >>
>>       > >> > >> https://xenproject.atlassian.net/projects/XEN/issues
>>       > >> > >>
>>       > >> > >> Most of the items listed are large projects, but some of 
>> them are pretty
>>       > >> > >> small, small enough to be a starting point. For example 
>> XEN-29 (Add
>>       > >> > >> -fstack-protector support to the hypervisor), XEN-30 (Make 
>> out-of-tree
>>       > >> > >> builds work) or XEN-38 (Xen - Embed .config in Xen binary).
>>       > >> >
>>       > >> > I  started working on XEN-38 for now. But i am also interested 
>> to look
>>       > >> > into XEN-30.
>>       > >>
>>       > >> Be careful that XEN-30 is longer than it looks because there are 
>> lots of
>>       > >> Makefiles to fix. However, even if you don't convert all 
>> Makefiles to
>>       > >> support out-of-tree builds but only some, it would still be OK.
>>       >
>>       > I will start with Xen-38 that would help me in exploring init code. 
>> Correct me if i
>>       > am wrong.
>>       >
>>       > I have a few questions and clarifications before proceeding further. 
>> I have checked
>>       > how config.gz file is generated in linux kernel source.
>>       > In linux kernel sources, if CONFIG_IKCONFIG_PROC option is set, 
>> .conifg file which
>>       > is generated after choosing options with lets say from make 
>> menuconfig  is read into
>>       > a variable, this way its part of build.
>>       >
>>       > during init time proc_create service is used to create this file 
>> config.gz.
>>       > http://lxr.free-electrons.com/source/kernel/configs.c
>>       >
>>       >
>>       > I guess i have to do something similar.
>>       >
>>       > Questions :
>>       >
>>       > 1. When Xen is build using the make command, we effectively set 
>> XEN_COMPILE_ARCH,
>>       > XEN_OS, XEN_TARGET which allow using corresponding .mk file from 
>> config folder.
>>       > These variable in turn decide what are the config options. I wasnt 
>> able to find any
>>       > .config. Please direct me to find the file or if i am missing 
>> something.
>>       >
>>       > 2. Where and how this config file should be accessible to  User once 
>> in Dom0. Is the
>>       > xen folder created to keep the information about guest domains like 
>> proc for process
>>       > in linux kernel ? Will that be suitable location to have config file.
>>       >
>>       > 3. if i assume that i will approach similarly, i have to add 
>> services to be called
>>       > during init stage. As am not acquainted with code base, i could just 
>> grep with
>>       > _start or _init or similar strings to find out initialization code. 
>> Any
>>       > input(function name or filename) to look for will be of great help.
>>       >
>>       > Best,
>>       > Karthik.
>>       >
>>       > >>
>>       > >>
>>       > >> > >> Cheers,
>>       > >> > >>
>>       > >> > >> Stefano
>>       > >> > >>
>>       > >> > >>
>>       > >> > >> On Mon, 27 Mar 2017, Methuku Karthik wrote:
>>       > >> > >> > Hi Everyone,
>>       > >> > >> >
>>       > >> > >> > My name is Karthik. I am a first year graduate student in 
>> Embedded Systems
>>       > at University of Pennsylvania. I am avid c, c++ and python 
>> programmer.I have 4 years
>>       > of
>>       > >> > >> > work experience as Embedded Software developer at Airbus.
>>       > >> > >> >
>>       > >> > >> > For the last four months I have been working as a research 
>> assistant in
>>       > PRECISE lab at the University of Pennsylvania, helping Meng Xu 
>> (cc.ed) to evaluate
>>       > the
>>       > >> > >> > performance of the Xen's RTDS scheduler.
>>       > >> > >> >
>>       > >> > >> > I'm interested in the following two projects. I'm happy to 
>> work on one of
>>       > them.
>>       > >> > >> > (1) Share a page in memory from the VM config file.
>>       > >> > >> > (2)  Xen on ARM: create multiple guests from device tree.
>>       > >> > >> >
>>       > >> > >> > I have experience in compiling Xen and installing it on 
>> ARM board(Nvidia
>>       > Jetson TK1). I'm also well equiped in using the xl tool to manage 
>> the VMs.
>>       > >> > >> >
>>       > >> > >> > I noticed that I should contribute some bug fixes to be 
>> able to get the
>>       > ticket to the Xen GSoC.
>>       > >> > >> > Do you have some suggestions on how I should start?
>>       > >> > >> > Do you have a list of simple bugs that I can fix?
>>       > >> > >> >
>>       > >> > >> > I'm very interested in the Xen Project. I want to become a 
>> Xen contributor
>>       > in the near future.
>>       > >> > >> >
>>       > >> > >> > Thank you very much for your help!
>>       > >> > >> >
>>       > >> > >> > Best,
>>       > >> > >> > Karthik
>>       > >> > >> >
>>       > >> > >> >
>>       > >> >
>>       > >> > Best,
>>       > >> > Karthik.
>>       > >> >
>>       > >>
>>       >
>>       >
>>       >
>>
>>
>>
>>

Attachment: bin2c.c
Description: Text Data

Attachment: tester.c
Description: Text Data

Attachment: xen_config.h
Description: Text Data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel