Re: [Xen-users] Networking how does it really work?

["Rafael Weingartner" <rafaelweingartner@xxxxxxxxx>]

Sorry Todd, I forgot the CC in the last message.

Hey guys.
I found out the problem with my VMs and the network.

Take a look at the annexed picture(is it possible to look it?).

I have that structure over virtual box instances.
So, each instance has a interface that connects it to the others VMs in the virtual box network.
My NFS server has 2 network interfaces, one that is plugged with the virtual internal network created by virtual box and another which has access to the internet using NAT provided by virtual box.

This network created by virtual box, does not have a gateway or router. It is like a simple hub where I can plug the PCs. The PCs in that network can see each other because they are in the same network level(10.1.1.0/24) without a router/gateway.

So When my VMs over the Xen server(10.1.1.101/24) get the IP address on the virtual box DHCP, they get the IP address at the same network level(10.1.1.0/24).
That is why the VMs over the Xen server can see each other. And they are directly “plugged” to the Xen server, that is why they also can access the Xen server.

But my VMs over the virtual box instances do not have a route to the VMs that are over the Xen server instances. As an example, when my VM-2 tries to ping the gateway(10.1.1.100/24), the ping goes to the server 10.1.1.100/24, passing by the bridge in the Xen server. But when the server(10.1.1.100/24) answers, it does not know how to access the VM-2. That server cannot find a route to VM-2. They are at the same network level but they need a gateway to “keep in touch”.

That is it guys... I will work around this problem ... And finally I can do my tests.

Thank for the help guys.

 

From: Todd Deshane

Sent: Friday, June 24, 2011 10:18 PM

To: Rafael Weingartner

Cc: xen-users mailing list

Subject: Re: [Xen-users] Networking how does it really work?

 

On Fri, Jun 24, 2011 at 2:15 PM, Rafael Weingartner <rafaelweingartner@xxxxxxxxx> wrote:

Hey guys,

 

You forgot to CC the list.

 

I tried to use that template “Other install media”, and it did not work in my case. As you said that template requires HVM support. And I do not have. Actually my hardware has, But I do not know why the Xen server on a virtual box instance seems not have that support. Even I ableing the HVM on the Virtual box instance. And yes the HVM is abled in the BIOS

 

looks like virtualbox might not support nested virtualization yet

http://www.virtualbox.org/ticket/4032

 

 

 

So I am working in the network case. I found out using that shell provided during the repository OS installation. That my VM is getting an IP in the network 10.1.1.0/24 which coincidentally is the same network that the NFS, gateway and Xen server are. The IP should be in a different range, no?

 

 

I think the problem is the you need to use bridged networking on your virtualbox VM. The IP will be on the same network and get a DHCP address on your local network. Then it should work.

 

And I think that is the problem. The VM got an IP 10.1.1.100/24, I have no idea why. But that is one of the problems, this IP address is coincidentally the same as my gateway address, so when I ran the command “wget -c http://10.1.1.101/www/index.html” to see if the VM could get a file in the Xen server. I lost the SSH connection that I was using with the Xen server and off course the VNC connection with the VM. When I re-connected with the Xen server using SSH and with the VM using VNC I saw that the VM got the file in the Xen server.

So, I changed the VM IP address to 10.1.1.103/24, just to avoid the conflict with my gateway. And when I tried to get the file again. I did not lose the connection. But when I tried to reach some file on the gateway it did not work. It says that it does not have a route to the host.

VM ip route

1.1.1.0/24 dev eth0 src 10.1.1.103

default via 10.1.1.101 dev eth0

I think that bridge on Xen server should have any problem.

But anyway, the VM can reach the Xen server. So I put the files that I need to install ubuntu in the Xen server, the files can be reached by the address “http://10.1.1.101/ubuntu”.

I created and started the VM with these commands as you can see.

[root@xenserver1 ~]# xe vm-install template=Ubuntu\ Lucid\ Lynx\ 10.04\ \(32-bit\)\ \(experimental\) new-name-label=ubuntu-local-repository

919c5304-3aba-1e34-42cf-87a85c3bdf1a

 

[root@xenserver1 ~]# xe network-list bridge=xenbr0 --minimal

a83ffa7b-593c-edde-9d56-ac007d56a1a9

[root@xenserver1 ~]# xe vif-create vm-uuid=919c5304-3aba-1e34-42cf-87a85c3bdf1a network-uuid=a83ffa7b-593c-edde-9d56-ac007d56a1a9 mac=random device=0

36f4a139-5944-553e-8406-557f1c11460d

[root@xenserver1 ~]# xe vm-param-set uuid=919c5304-3aba-1e34-42cf-87a85c3bdf1a other-config:install-repository=http://10.1.1.101/ubuntu

[root@xenserver1 ~]# xe vm-start uuid=919c5304-3aba-1e34-42cf-87a85c3bdf1a

The installation began, I changed the IP to avoid the conflict with the gateway, and then I entered manually the repository address. But I am still getting the message saying that the installation could not reach the files that it need.

Here are my mirror configurations that I am using.

############# config ##################
#
set base_path /var/spool/apt-mirror
#
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch <running host architecture>
# set postmirror_script $var_path/postmirror.sh
set run_postmirror 0
set nthreads 20
set _tilde 0
#
############# end config ##############
#Debian mirror
#deb http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-src http://ftp.us.debian.org/debian unstable main contrib non-free
# mirror additional architectures
#deb-alpha http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-amd64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-armel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-hppa http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-i386 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-ia64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-m68k http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mips http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mipsel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-powerpc http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-s390 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-sparc http://ftp.us.debian.org/debian unstable main contrib non-free
#clean http://ftp.us.debian.org/debian
#
#Ubuntu mirror
deb http://archive.ubuntu.com/ubuntu lucid main restricted
#deb http://archive.ubuntu.com/ubuntu lucid-updates main restricted
#deb http://archive.ubuntu.com/ubuntu lucid-backports main restricted
#deb-src http://archive.ubuntu.com/ubuntu lucid-backports main restricted
deb http://security.ubuntu.com/ubuntu lucid-security main restricted
clean http://archive.ubuntu.com/ubuntu

Those configurations should be enough to create a mirror to install Ubuntu from a local repository. As you can see in the logs below.

The problem is that I have no idea about what files it is trying to reach but it could not reach.

Since in the log what appears is just a message like this “wget: server returned error: HTTP/1.1 404 Not Found”

Xen server ifconfig

[root@xenserver1 ubuntu]# ifconfig

eth0 Link encap:Ethernet HWaddr 08:00:27:7F:EA:B2

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:21142 errors:0 dropped:0 overruns:0 frame:0

TX packets:15423 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:14973421 (14.2 MiB) TX bytes:4058505 (3.8 MiB)

 

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

UP LOOPBACK RUNNING MTU:16436 Metric:1

RX packets:14225 errors:0 dropped:0 overruns:0 frame:0

TX packets:14225 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:16262432 (15.5 MiB) TX bytes:16262432 (15.5 MiB)

 

vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF

UP BROADCAST RUNNING NOARP MTU:1500 Metric:1

RX packets:557 errors:0 dropped:0 overruns:0 frame:0

TX packets:2456 errors:0 dropped:100 overruns:0 carrier:0

collisions:0 txqueuelen:32

RX bytes:61311 (59.8 KiB) TX bytes:974090 (951.2 KiB)

 

xenbr0 Link encap:Ethernet HWaddr 08:00:27:7F:EA:B2

inet addr:10.1.1.101 Bcast:10.1.1.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:21681 errors:0 dropped:0 overruns:0 frame:0

TX packets:14771 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:15040677 (14.3 MiB) TX bytes:4603635 (4.3 MiB)

 

Xen server brctl show

[root@xenserver1 ubuntu]# brctl show

bridge name bridge id STP enabled interfaces

xenbr0 0000.0800277feab2 no eth0

vif2.0

 

/var/log/syslog

Jun 24 17:44:41 main-menu[229]: DEBUG: resolver (libc6-udeb): package doesn't exist (ignored)
Jun 24 17:44:41 main-menu[229]: DEBUG: resolver (libnewt0.52): package doesn't exist (ignored)
Jun 24 17:44:41 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:42 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:42 main-menu[229]: INFO: Menu item 'choose-mirror' selected
Jun 24 17:44:42 anna-install: Installing apt-mirror-setup
Jun 24 17:44:42 anna[2370]: ERROR **: can't find packages file
Jun 24 17:44:46 choose-mirror[2374]: DEBUG: command: wget -q http://10.1.1.101/ubuntu/dists/lucid/Release -O - | grep ^Suite: | cut -d' ' -f 2
Jun 24 17:44:46 choose-mirror[2374]: DEBUG: command: wget -q http://10.1.1.101/ubuntu/dists/lucid/Release -O - | grep ^Codename: | cut -d' ' -f 2
Jun 24 17:44:46 choose-mirror[2374]: INFO: codename set to: lucid
Jun 24 17:44:46 choose-mirror[2374]: DEBUG: command: wget -q http://10.1.1.101/ubuntu/dists/lucid/main/binary-i386/Release -O - | grep Architecture
Jun 24 17:44:46 anna-install: Installing lucid-support
Jun 24 17:44:46 anna[2386]: ERROR **: can't find packages file
Jun 24 17:44:46 main-menu[229]: DEBUG: resolver (libc6-udeb): package doesn't exist (ignored)
Jun 24 17:44:46 main-menu[229]: DEBUG: resolver (libnewt0.52): package doesn't exist (ignored)
Jun 24 17:44:46 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:46 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:46 main-menu[229]: INFO: Menu item 'download-installer' selected
Jun 24 17:44:46 net-retriever: gpgv:
Jun 24 17:44:46 net-retriever: Signature made Thu Apr 29 17:25:07 2010 UTC using DSA key ID 437D05B5
Jun 24 17:44:46 net-retriever: gpgv:
Jun 24 17:44:46 net-retriever: Good signature from "Ubuntu Archive Automatic Signing Key <ftpmaster@xxxxxxxxxx>"
Jun 24 17:44:46 net-retriever:
Jun 24 17:44:46 anna[2403]: wget: server returned error: HTTP/1.1 404 Not Found
Jun 24 17:44:46 anna[2403]: wget: server returned error: HTTP/1.1 404 Not Found
Jun 24 17:44:46 anna[2403]: wget: server returned error: HTTP/1.1 404 Not Found
Jun 24 17:44:46 anna[2403]: wget: server returned error: HTTP/1.1 404 Not Found
Jun 24 17:44:46 anna[2403]: cat: can't open '/tmp/net-retriever-2407-deduplicate/*': No such file or directory
Jun 24 17:44:50 anna[2403]: WARNING **: bad d-i Packages file
Jun 24 17:44:50 main-menu[229]: INFO: Menu item 'download-installer' succeeded but requested to be left unconfigured.
Jun 24 17:44:50 main-menu[229]: DEBUG: resolver (libc6-udeb): package doesn't exist (ignored)
Jun 24 17:44:50 main-menu[229]: DEBUG: resolver (libnewt0.52): package doesn't exist (ignored)
Jun 24 17:44:50 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:51 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb
Jun 24 17:44:51 main-menu[229]: INFO: Menu item 'save-logs' selected
Jun 24 17:45:06 main-menu[229]: DEBUG: resolver (libc6-udeb): package doesn't exist (ignored)
Jun 24 17:45:06 main-menu[229]: DEBUG: resolver (libnewt0.52): package doesn't exist (ignored)
Jun 24 17:45:06 main-menu[229]: INFO: Falling back to the package description for console-setup-udeb

/var/log/hardwareSummary

uname -a: Linux ubuntu 2.6.32-21-generic-pae #32-Ubuntu SMP Fri Apr 16 09:39:35 UTC 2010 i686 unknown
lsmod: Module Size Used by
lsmod: vfat 8901 0
lsmod: fat 47767 1 vfat
lsmod: rsrc_nonstatic 10559 0
lsmod: pcmcia_core 32996 1 rsrc_nonstatic
lsmod: xen_netfront 14887 0
df: Filesystem 1K-blocks Used Available Use% Mounted on
df: tmpfs 124820 108 124712 0% /dev
free: total used free shared buffers
free: Mem: 249644 31688 217956 0 0
free: Swap: 0 0 0
free: Total: 249644 31688 217956
/proc/cmdline: -- quiet console=hvc0
/proc/cpuinfo: processor : 0
/proc/cpuinfo: vendor_id : GenuineIntel
/proc/cpuinfo: cpu family : 6
/proc/cpuinfo: model : 30
/proc/cpuinfo: model name : Intel(R) Core(TM) i7 CPU Q 740 @ 1.73GHz
/proc/cpuinfo: stepping : 5
/proc/cpuinfo: cpu MHz : 1706.599
/proc/cpuinfo: cache size : 6144 KB
/proc/cpuinfo: fdiv_bug : no
/proc/cpuinfo: hlt_bug : no
/proc/cpuinfo: f00f_bug : no
/proc/cpuinfo: coma_bug : no
/proc/cpuinfo: fpu : yes
/proc/cpuinfo: fpu_exception : yes
/proc/cpuinfo: cpuid level : 5
/proc/cpuinfo: wp : yes
/proc/cpuinfo: flags : fpu de tsc msr pae cx8 sep cmov pat clflush mmx fxsr sse sse2 nx constant_tsc up pni ssse3 hypervisor
/proc/cpuinfo: bogomips : 3413.19
/proc/cpuinfo: clflush size : 64
/proc/cpuinfo: cache_alignment : 64
/proc/cpuinfo: address sizes : 36 bits physical, 48 bits virtual
/proc/cpuinfo: power management:
/proc/cpuinfo:
/proc/ioports: 0000-ffff : paravirt-ioport
/proc/ioports: 0070-0071 : rtc_cmos
/proc/iomem: 00000000-00001fff : System RAM
/proc/iomem: 00002000-00005fff : reserved
/proc/iomem: 00006000-0009ffff : System RAM
/proc/iomem: 000a0000-000fffff : reserved
/proc/iomem: 000f0000-000fffff : System ROM
/proc/iomem: 00100000-0fffffff : System RAM
/proc/iomem: 00100000-005b6a1a : Kernel code
/proc/iomem: 005b6a1b-007df7a7 : Kernel data
/proc/iomem: 0088d000-0092f7d7 : Kernel bss
/proc/interrupts: CPU0
/proc/interrupts: 0: 9511 xen-dyn-virq timer0
/proc/interrupts: 1: 0 xen-dyn-ipi spinlock0
/proc/interrupts: 2: 0 xen-dyn-ipi resched0
/proc/interrupts: 3: 0 xen-dyn-ipi callfunc0
/proc/interrupts: 4: 0 xen-dyn-virq debug0
/proc/interrupts: 5: 0 xen-dyn-ipi callfuncsingle0
/proc/interrupts: 6: 109 xen-dyn-event xenbus
/proc/interrupts: 7: 1751 xen-dyn-event hvc_console
/proc/interrupts: 8: 1287 xen-dyn-event eth0
/proc/interrupts: NMI: 0 Non-maskable interrupts
/proc/interrupts: LOC: 0 Local timer interrupts
/proc/interrupts: SPU: 0 Spurious interrupts
/proc/interrupts: PMI: 0 Performance monitoring interrupts
/proc/interrupts: PND: 0 Performance pending work
/proc/interrupts: RES: 0 Rescheduling interrupts
/proc/interrupts: CAL: 0 Function call interrupts
/proc/interrupts: TLB: 0 TLB shootdowns
/proc/interrupts: TRM: 0 Thermal event interrupts
/proc/interrupts: THR: 0 Threshold APIC interrupts
/proc/interrupts: MCE: 0 Machine check exceptions
/proc/interrupts: MCP: 0 Machine check polls
/proc/interrupts: ERR: 0
/proc/interrupts: MIS: 0
/proc/meminfo: MemTotal: 249644 kB
/proc/meminfo: MemFree: 217956 kB
/proc/meminfo: Buffers: 0 kB
/proc/meminfo: Cached: 23284 kB
/proc/meminfo: SwapCached: 0 kB
/proc/meminfo: Active: 14976 kB
/proc/meminfo: Inactive: 10740 kB
/proc/meminfo: Active(anon): 2540 kB
/proc/meminfo: Inactive(anon): 0 kB
/proc/meminfo: Active(file): 12436 kB
/proc/meminfo: Inactive(file): 10740 kB
/proc/meminfo: Unevictable: 0 kB
/proc/meminfo: Mlocked: 0 kB
/proc/meminfo: HighTotal: 0 kB
/proc/meminfo: HighFree: 0 kB
/proc/meminfo: LowTotal: 249644 kB
/proc/meminfo: LowFree: 217956 kB
/proc/meminfo: SwapTotal: 0 kB
/proc/meminfo: SwapFree: 0 kB
/proc/meminfo: Dirty: 0 kB
/proc/meminfo: Writeback: 0 kB
/proc/meminfo: AnonPages: 2460 kB
/proc/meminfo: Mapped: 1316 kB
/proc/meminfo: Shmem: 108 kB
/proc/meminfo: Slab: 3884 kB
/proc/meminfo: SReclaimable: 1764 kB
/proc/meminfo: SUnreclaim: 2120 kB
/proc/meminfo: KernelStack: 440 kB
/proc/meminfo: PageTables: 348 kB
/proc/meminfo: NFS_Unstable: 0 kB
/proc/meminfo: Bounce: 0 kB
/proc/meminfo: WritebackTmp: 0 kB
/proc/meminfo: CommitLimit: 124820 kB
/proc/meminfo: Committed_AS: 6516 kB
/proc/meminfo: VmallocTotal: 600056 kB
/proc/meminfo: VmallocUsed: 340 kB
/proc/meminfo: VmallocChunk: 599560 kB
/proc/meminfo: HardwareCorrupted: 0 kB
/proc/meminfo: HugePages_Total: 0
/proc/meminfo: HugePages_Free: 0
/proc/meminfo: HugePages_Rsvd: 0
/proc/meminfo: HugePages_Surp: 0
/proc/meminfo: Hugepagesize: 2048 kB
/proc/meminfo: DirectMap4k: 262144 kB
/proc/meminfo: DirectMap2M: 0 kB
/proc/bus/input/devices: I: Bus=0017 Vendor=0001 Product=0001 Version=0100
/proc/bus/input/devices: N: Name="Macintosh mouse button emulation"
/proc/bus/input/devices: P: Phys=
/proc/bus/input/devices: S: Sysfs=/devices/virtual/input/input0
/proc/bus/input/devices: U: Uniq=
/proc/bus/input/devices: H: Handlers=mouse0 event0
/proc/bus/input/devices: B: EV=7
/proc/bus/input/devices: B: KEY=70000 0 0 0 0 0 0 0 0
/proc/bus/input/devices: B: REL=3
/proc/bus/input/devices:

Do you guys have any idea?

 

I am downloading a prebuilt Xen Virtual Appliances that you suggest me to see if I can work aroud those problems.

 

 

-----Mensagem Original-----

From: Todd Deshane

Sent: Thursday, June 23, 2011 12:26 AM

To: Rafael Weingartner

Cc: Xen-user-list

Subject: Re: [Xen-users] Networking how does it really work?

 

On Wed, Jun 22, 2011 at 10:32 PM, Rafael Weingartner

<rafaelweingartner@xxxxxxxxx> wrote:

>

> Hey guys...

> I hope that you can help me with these information...

<snip>

 

Very clear and detailed information for sure. You should write

howtos/documentation.

 

One thing that should help you is that you should use:

 

"xe vm-install template=Other\ install\ media" for installing VM based on ISOs.

 

I still don't understand why your network doesn't work. It should

"just work" like we said and from what you have shown everything looks

normal. The best way to debug that part is to do tcmpdump or tshark

network traces at all sides of this. So on the bridge, on the vif, and

within the guest (once you have one). See:

 

http://support.citrix.com/article/CTX120869

 

 

To get a guest working if the other install media option doesn't work

for some reason, you can download prebuilt Xen Virtual Appliances

(.xva) files and copy them from a USB drive to your XCP server to be

able to import them. For example:

 

http://www.xvpsource.org/?topic=build&page=appliancevm

 

(as an aside XVA: downloads are also available from Stacklet.com at a

small subscription fee)

 

I hope that the "Other install media" option gets you farther. Please

feel free to post your results from the network debugging as well as

any logs that might contain useful information /var/log/messages,

dmesg, /var/log/xensource.log, etc.

 

Thanks,

Todd

 

--

Todd Deshane

http://www.linkedin.com/in/deshantm

http://www.xen.org/products/cloudxen.html

http://runningxen.com/

--
Todd Deshane
http://www.linkedin.com/in/deshantm
http://www.xen.org/products/cloudxen.html
http://runningxen.com/

Attachment: Virtual Structure.png
Description: PNG image

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users