|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-API] [XCP 1.6 BETA] BUG: problem with creation of VIF on VLAN created on bonded interface
Hi Rob thanks for your email. I wasn't aware of the Xenserver 6 logic you described for replacing a NIC in situ. In my case it didn't apply as the NIC change had happened prior to the upgrade, but it's useful to keep in mind. I did actually use the xe-reset-networking, from xsconsole, which enabled me to get out of my situation eventually. The problem this introduced, however, was exactly what you described as resetting the NIC device names, and being unable to set them to the desired values. There are use cases where setting specific device names are important. For example, one could have different technologies of network interfaces (100M, 1G, 10G) for different groups of networks depending upon the throughput requirements, and in a pool environment one might want to simplify administration by standardising on a device naming convention for all pool members (both my use cases). I couldn't find a documented way of doing this, and I eventually found a way as per my previous email (through use of the static-rules.conf file). I think a xapi facility to reset the device names, or at least officially documenting the process I described, would be useful to some users. thanks again for your feedback. On Fri, Oct 19, 2012 at 12:25 AM, Rob Hoes <Rob.Hoes@xxxxxxxxxx> wrote: > Hi, > > Since version 6.0, XenServer has some logic to automatically deal with > replacing NICs, adding new NICs etc. For example, if a NIC breaks, and you > place a new one in the same slot, XenServer will recognise this and use the > same device name for this interface, and it will also update the MAC address > in the PIF record. A consequence is that it is no longer possible to manually > change device names using pif-forget and pif-introduce. The Citrix knowledge > base article that you refer to only applies to older releases. > > I am not exactly sure what has happened in your case, but in XS 6.0.2/6.1 and > XCP 1.6 you should be able to use the xe-reset-networking command to reset > all networking state in your host, which will also reset the NIC device names > as if you have newly installed the host. There is some documentation about > this command here: > http://docs.vmd.citrix.com/XenServer/6.1.0/1.0/en_gb/reference.html#id889904. > > Cheers, > Rob > >> -----Original Message----- >> From: xen-api-bounces@xxxxxxxxxxxxx [mailto:xen-api- >> bounces@xxxxxxxxxxxxx] On Behalf Of Black Bird >> Sent: 18 October 2012 06:16 >> To: xen-api@xxxxxxxxxxxxx >> Subject: Re: [Xen-API] [XCP 1.6 BETA] BUG: problem with creation of VIF >> on VLAN created on bonded interface >> >> Hi all >> >> an update to the below. >> >> After further investigation, I concluded that the below was the result >> of some changes to the NIC configuration just prior to the upgrade. >> >> I thought I'd document something I found during my investigations. It >> was a long story, caused by xenserver allocating my desired devices >> with 2 new devices (eth7/8). In trying to fix this, I ended up with >> the host involved loosing connectivity to the pool, and eventually >> resulted in having to reset the network configuration from the >> console. >> >> What was interesting was that after doing so, xenserver refused to >> allocate 2 of my eth device names in the preferred order, and >> basically continued replacing eth1/2 with eth7/8. I tried the usual >> sequence of pif-forget, pif-scan and pif-introduce to change the >> device did not work - the last command resulted in an error message >> saying that the device and mac address did not match. >> >> The instructions at http://support.citrix.com/article/CTX129946 also >> could not be followed as no ifcfg-ethx configuration files existed. >> >> Further searches on the internet found that this was a common problem, >> encountered when trying to change NICs due to a faulty card for >> example, but no solution. There was one report of a solution, by >> manually editing the file /etc/udev/rules.d/60-net.rules and modifying >> the device name corresponding to the mac address, but in my case this >> also failed, as I understand this file is automatically generated by >> xenserver during restart based on its last known boot configuration. >> >> I did find a way around this however. There is another file which is >> apparently used by the xenserver installer to configure network cards >> on a clean installation. It transpires that this file is also used on >> restart to manually override the udev file. It can be found at >> /etc/sysconfig/network-scripts/interface-rename-date/static-rules.conf. >> Basically you add a line consisting of the desired device name, a >> colon, and an identifier for the device. In my case, i modified the >> file to contain: >> >> eth1: mac=<addr1> >> eth2: mac=<addr2> >> >> There are other identifiers you can use - the file contains a short >> description. >> >> After this change, I simply rebooted, and the dom0 identified the >> desired device names to the interfaces. These could be seen with >> ifconfig -a. I then had to remove all bonds that were automatically >> replicated by xenserver from the pool (they were generated in the >> wrong order and thus given wrong bondx labels), did a pif-scan, >> recreated bonds etc. At this point, xenserver also recreated the pool >> vlans and I was back in business. >> >> I did take me 2-3 days to come to this point, so I hope this >> information is useful to anyone who has passed through a similar >> experience. >> >> One final point - the static configuration in the static-rules.conf >> file can be deleted as soon as xenserver has recognised the new >> configuration. This is because xenserver now starts reflecting the >> updated device configuration into the 60-net.rules file, so the >> updates persist. >> >> >> >> >> On Mon, Oct 15, 2012 at 6:59 PM, Black Bird <blackbird1758@xxxxxxxxx> >> wrote: >> > I have just carried out a rolling pool upgrade of a 3-host pool from >> > XCP 1.1.0 (50674c) to XCP 1.6 Beta (61002c). I was able to bring up >> > the VMs as normal and network connectivity is ok. >> > >> > However when I tried to implement the first change I encountered a >> > problem. I tried to change a vif from one network (a physical >> > interface eth0, network uuid ef66fdda-e249-a01d-545c-e002f945bdc6) to >> > another (a pool vlan 255 on a bonded interface bond0 created with >> > slaves eth1 and eth2, network uuid >> > 7568abf3-470f-21fc-9ec2-24e854d4522c). The vif-destroy and vif- >> create >> > process were executed as normal, but on attempting to start the vm, I >> > got an error saying "stderr = ovs-vsctl: cannot create a port named >> > eth1 because an interface named eth1 already exists on bridge xapi3". >> > >> > My network configuration, on each of the 3 hosts, is: >> > >> > interface bond >> > eth0 n.a. >> > eth1 bond0 >> > eth2 bond0 >> > eth3 bond1 >> > eth4 bond1 >> > eth5 bond2 >> > eth6 bond2 >> > >> > I have a number of pool vlans created on bond0 and bond2. The >> > particular vlan involved in this report is vlan255, corresponding to >> > an 802.1q tag of 255, created on bond0 >> > >> > The command log follows. The vm and previous vif had their >> parameters >> > listed prior to the change. The attempt to startup the vm results in >> > the error. I thought it was strange that ovs-vsctl is attempting to >> > add eth1 to the bridge xapi3 (which corresponds to bond0) when the vm >> > starts - I would have expected this to take place on xenserver >> > startup. >> > >> > I have carried out this operation prior to the upgrade dozens of >> > times, so I know it should work. I'd be grateful for any advice. >> > >> > [root@xen3v3 ~]# xe vm-param-list uuid=449fb2b2-5df2-2c6b-33ca- >> 7627d1eed6cb >> > uuid ( RO) : 449fb2b2-5df2-2c6b-33ca- >> 7627d1eed6cb >> > name-label ( RW): efw2 >> > name-description ( RW): Installed via xe CLI >> > user-version ( RW): 1 >> > is-a-template ( RW): false >> > is-a-snapshot ( RO): false >> > snapshot-of ( RO): <not in database> >> > snapshots ( RO): >> > snapshot-time ( RO): 19700101T00:00:00Z >> > snapshot-info ( RO): >> > parent ( RO): <not in database> >> > children ( RO): >> > is-control-domain ( RO): false >> > power-state ( RO): running >> > memory-actual ( RO): 33505280 >> > memory-target ( RO): 33554432 >> > memory-overhead ( RO): 4194304 >> > memory-static-max ( RW): 33554432 >> > memory-dynamic-max ( RW): 33554432 >> > memory-dynamic-min ( RW): 33554432 >> > memory-static-min ( RW): 33554432 >> > suspend-VDI-uuid ( RW): <not in database> >> > suspend-SR-uuid ( RW): <not in database> >> > VCPUs-params (MRW): >> > VCPUs-max ( RW): 1 >> > VCPUs-at-startup ( RW): 1 >> > actions-after-shutdown ( RW): Destroy >> > actions-after-reboot ( RW): Restart >> > actions-after-crash ( RW): Restart >> > console-uuids (SRO): 146df0b4-a963-b399-8b97- >> f10646fd4f4b >> > platform (MRW): timeoffset: 0; nx: false; acpi: >> > true; apic: true; pae: true; viridian: true >> > allowed-operations (SRO): hard_reboot; hard_shutdown; >> pause >> > current-operations (SRO): >> > blocked-operations (MRW): >> > allowed-VBD-devices (SRO): 1; 2; 3 >> > allowed-VIF-devices (SRO): 2; 3 >> > possible-hosts ( RO): >> > 3e760f02-a3a5-470f-94f5-50023805f9c3; >> > 403452ad-6d8e-4757-a9c2-305e867134fd; >> > a53d1a5a-a9a3-4314-8295-1af5584cf02b >> > HVM-boot-policy ( RW): BIOS order >> > HVM-boot-params (MRW): order: dc >> > HVM-shadow-multiplier ( RW): 1.000 >> > PV-kernel ( RW): >> > PV-ramdisk ( RW): >> > PV-args ( RW): >> > PV-legacy-args ( RW): >> > PV-bootloader ( RW): >> > PV-bootloader-args ( RW): >> > last-boot-CPU-flags ( RO): vendor: AuthenticAMD; features: >> > 00002001-178bfbff-0000001f-ebd3fbff >> > last-boot-record ( RO): >> > '<value><struct><member><name>uuid</name><value>449fb2b2-5df2-2c6b- >> 33ca- >> 7627d1eed6cb</value></member><member><name>allowed_operations</name><va >> lue><array><data/></array></value></member><member><name>current_operat >> ions</name><value><struct><member><name>OpaqueRef:3bbc686b-cff2-c9b2- >> d165- >> 1971021e3ba6</name><value>start_on</value></member></struct></value></m >> ember><member><name>power_state</name><value>Halted</value></member><me >> mber><name>name_label</name><value>efw2</value></member><member><name>n >> ame_description</name><value>Installed >> > via xe >> CLI</value></member><member><name>user_version</name><value>1</value></ >> member><member><name>is_a_template</name><value><boolean>0</boolean></v >> alue></member><member><name>suspend_VDI</name><value>OpaqueRef:NULL</va >> lue></member><member><name>resident_on</name><value>OpaqueRef:NULL</val >> ue></member><member><name>affinity</name><value>OpaqueRef:NULL</value>< >> /member><member><name>memory_overhead</name><value>4194304</value></mem >> ber><member><name>memory_target</name><value>33554432</value></member>< >> member><name>memory_static_max</name><value>33554432</value></member><m >> ember><name>memory_dynamic_max</name><value>33554432</value></member><m >> ember><name>memory_dynamic_min</name><value>33554432</value></member><m >> ember><name>memory_static_min</name><value>33554432</value></member><me >> mber><name>VCPUs_params</name><value><struct/></value></member><member> >> <name>VCPUs_max</name><value>1</value></member><member><name>VCPUs_at_s >> tartup</name><value>1</value></member><member><name>actions_after_shutd >> own</name><value>destroy</value></member><member><name>actions_after_re >> boot</name><value>restart</value></member><member><name>actions_after_c >> rash</name><value>restart</value></member><member><name>consoles</name> >> <value><array><data/></array></value></member><member><name>VIFs</name> >> <value><array><data><value>OpaqueRef:d606e1f2-ec63-64b3-3603- >> c2232244eeaa</value><value>OpaqueRef:4deed917-1d9f-62fa-173f- >> 46ad5d88d2d6</value></data></array></value></member><member><name>VBDs< >> /name><value><array><data><value>OpaqueRef:ae670fba-2c5d-d736-d19e- >> 7247e842dd4a</value></data></array></value></member><member><name>crash >> _dumps</name><value><array><data/></array></value></member><member><nam >> e>VTPMs</name><value><array><data/></array></value></member><member><na >> me>PV_bootloader</name><value/></member><member><name>PV_kernel</name>< >> value/></member><member><name>PV_ramdisk</name><value/></member><member >> ><name>PV_args</name><value/></member><member><name>PV_bootloader_args< >> /name><value/></member><member><name>PV_legacy_args</name><value/></mem >> ber><member><name>HVM_boot_policy</name><value>BIOS >> > >> order</value></member><member><name>HVM_boot_params</name><value><struc >> t><member><name>order</name><value>dc</value></member></struct></value> >> </member><member><name>HVM_shadow_multiplier</name><value><double>1</do >> uble></value></member><member><name>platform</name><value><struct><memb >> er><name>timeoffset</name><value>- >> 1</value></member><member><name>nx</name><value>false</value></member>< >> member><name>acpi</name><value>true</value></member><member><name>apic< >> /name><value>true</value></member><member><name>pae</name><value>true</ >> value></member><member><name>viridian</name><value>true</value></member >> ></struct></value></member><member><name>PCI_bus</name><value/></member >> ><member><name>other_config</name><value><struct><member><name>last_shu >> tdown_time</name><value>20121015T04:25:36Z</value></member><member><nam >> e>last_shutdown_action</name><value>Destroy</value></member><member><na >> me>last_shutdown_initiator</name><value>external</value></member><membe >> r><name>last_shutdown_reason</name><value>halted</value></member><membe >> r><name>auto_poweron</name><value>true</value></member><member><name>im >> port_task</name><value>OpaqueRef:8ca87370-044d-3fcf-2718- >> 0e31a989f87a</value></member><member><name>mac_seed</name><value>b95def >> a9-572a-1f7f-7ce7-dd834400f8fb</value></member><member><name>install- >> methods</name><value>cdrom</value></member></struct></value></member><m >> ember><name>domid</name><value>- >> 1</value></member><member><name>domarch</name><value>hvm</value></membe >> r><member><name>last_boot_CPU_flags</name><value><struct><member><name> >> vendor</name><value>AuthenticAMD</value></member><member><name>features >> </name><value>00002001-178bfbff-0000001f- >> ebd3fbff</value></member></struct></value></member><member><name>is_con >> trol_domain</name><value><boolean>0</boolean></value></member><member>< >> name>metrics</name><value>OpaqueRef:e43062d7-2b48-e192-27cd- >> c16a872b2ce1</value></member><member><name>guest_metrics</name><value>O >> paqueRef:NULL</value></member><member><name>last_booted_record</name><v >> alue/></member><member><name>recommendations</name><value><restricti >> ons><restriction >> > field="memory-static-max" max="34359738368" >> > /><restriction field="vcpus-max" max="8" >> > /><restriction property="number-of-vbds" >> > max="7" /><restriction >> > property="number-of-vifs" max="7" >> > >> /></restrictions></value></member><member><name>xenstore_data< >> /name><value><struct/></value></member><member><name>ha_always_run</nam >> e><value><boolean>0</boolean></value></member><member><name>ha_restart_ >> priority</name><value>best- >> effort</value></member><member><name>is_a_snapshot</name><value><boolea >> n>0</boolean></value></member><member><name>snapshot_of</name><value>Op >> aqueRef:NULL</value></member><member><name>snapshots</name><value><arra >> y><data/></array></value></member><member><name>snapshot_time</name><va >> lue><dateTime.iso8601>19700101T00:00:00Z</dateTime.iso8601></value></me >> mber><member><name>transportable_snapshot_id</name><value/></member><me >> mber><name>blobs</name><value><struct/></value></member><member><name>t >> ags</name><value><array><data/></array></value></member><member><name>b >> locked_operations</name><value><struct/></value></member><member><name> >> snapshot_info</name><value><struct/></value></member><member><name>snap >> shot_metadata</name><value/></member><member><name>parent</name><value> >> OpaqueRef:NULL</value></member><member><name>children</name><value><arr >> ay><data/></array></value></member><member><name>bios_strings</name><va >> lue><struct/></value></member><member><name>protection_policy</name><va >> lue>OpaqueRef:NULL</value></member><member><name>is_snapshot_from_vmpp< >> /name><value><boolean>0</boolean></value></member><member><name>applian >> ce</name><value>OpaqueRef:NULL</value></member><member><name>start_dela >> y</name><value>0</value></member><member><name>shutdown_delay</name><va >> lue>0</value></member><member><name>order</name><value>0</value></membe >> r><member><name>VGPUs</name><value><array><data/></array></value></memb >> er><member><name>attached_PCIs</name><value><array><data/></array></val >> ue></member><member><name>suspend_SR</name><value>OpaqueRef:NULL</value >> ></member><member><name>version</name><value>0</value></member></struct >> ></value>' >> > resident-on ( RO): 403452ad-6d8e-4757-a9c2- >> 305e867134fd >> > affinity ( RW): <not in database> >> > other-config (MRW): vgpu_pci: ; last_shutdown_time: >> > 20121015T04:25:36Z; last_shutdown_action: Destroy; >> > last_shutdown_initiator: external; last_shutdown_reason: halted; >> > auto_poweron: true; import_task: >> > OpaqueRef:8ca87370-044d-3fcf-2718-0e31a989f87a; mac_seed: >> > b95defa9-572a-1f7f-7ce7-dd834400f8fb; install-methods: cdrom >> > dom-id ( RO): 2 >> > recommendations ( RO): <restrictions><restriction >> > field="memory-static-max" max="34359738368" /><restriction >> > field="vcpus-max" max="8" /><restriction property="number-of-vbds" >> > max="7" /><restriction property="number-of-vifs" max="7" >> > /></restrictions> >> > xenstore-data (MRW): vm-data: >> > ha-always-run ( RW) [DEPRECATED]: false >> > ha-restart-priority ( RW): best-effort >> > blobs ( RO): >> > start-time ( RO): 20121015T04:48:27Z >> > install-time ( RO): 19700101T00:00:00Z >> > VCPUs-number ( RO): 1 >> > VCPUs-utilisation (MRO): 0: 0.007 >> > os-version (MRO): <not in database> >> > PV-drivers-version (MRO): <not in database> >> > PV-drivers-up-to-date ( RO): <not in database> >> > memory (MRO): <not in database> >> > disks (MRO): <not in database> >> > networks (MRO): <not in database> >> > other (MRO): <not in database> >> > live ( RO): <not in database> >> > guest-metrics-last-updated ( RO): <not in database> >> > cooperative ( RO) [DEPRECATED]: true >> > protection-policy ( RW): <not in database> >> > is-snapshot-from-vmpp ( RO): false >> > tags (SRW): >> > appliance ( RW): <not in database> >> > start-delay ( RW): 0 >> > shutdown-delay ( RW): 0 >> > order ( RW): 0 >> > version ( RO): 0 >> > [root@xen3v3 ~]# >> > [root@xen3v3 ~]# xe vif-param-list uuid=d4403431-0363-d9a5-a708- >> 8eddf85e531a >> > uuid ( RO) : d4403431-0363-d9a5-a708- >> 8eddf85e531a >> > vm-uuid ( RO): 449fb2b2-5df2-2c6b-33ca- >> 7627d1eed6cb >> > vm-name-label ( RO): efw2 >> > allowed-operations (SRO): attach >> > current-operations (SRO): >> > device ( RO): 1 >> > MAC ( RO): be:5c:1e:a2:77:3c >> > MAC-autogenerated ( RO): false >> > MTU ( RO): 1500 >> > currently-attached ( RO): false >> > qos_algorithm_type ( RW): >> > qos_algorithm_params (MRW): >> > qos_supported_algorithms (SRO): >> > other-config (MRW): >> > network-uuid ( RO): ef66fdda-e249-a01d-545c- >> e002f945bdc6 >> > network-name-label ( RO): Pool-wide network associated with >> eth0 >> > io_read_kbs ( RO): <unknown> >> > io_write_kbs ( RO): <unknown> >> > locking-mode ( RW): network_default >> > ipv4-allowed (SRW): >> > ipv6-allowed (SRW): >> > >> > [root@xen3v3 ~]# xe vif-destroy uuid=d4403431-0363-d9a5-a708- >> 8eddf85e531a >> > [root@xen3v3 ~]# xe vif-create >> > vm-uuid=449fb2b2-5df2-2c6b-33ca-7627d1eed6cb >> > network-uuid=7568abf3-470f-21fc-9ec2-24e854d4522c device=1 >> > 56e4b508-7254-3315-f3af-eea22d2f826d >> > [root@xen3v3 ~]# xe vm-start uuid=449fb2b2-5df2-2c6b-33ca- >> 7627d1eed6cb >> > An unknown error occurred while attempting to configure an interface. >> > PIF: 1ce7d205-df3a-f6c6-be8b-6d8f8585cd6f >> > msg: script error [script = /usr/bin/ovs-vsctl, args = --timeout=20 - >> - >> > --may-exist add-br xapi3 -- set bridge xapi3 >> > other-config:hwaddr="00:0e:0c:b5:66:8e" -- set bridge xapi3 >> > fail_mode=standalone -- remove bridge xapi3 other_config >> > disable-in-band -- br-set-external-id xapi3 xs-network-uuids >> > daaabcf2-8998-15c5-524f-5c19a5dd14b4 -- --may-exist add-port xapi3 >> > eth1 -- --may-exist add-port xapi3 eth2, code = 1, stdout = , stderr >> = >> > ovs-vsctl: cannot create a port named eth1 because an interface named >> > eth1 already exists on bridge xapi3] >> >> _______________________________________________ >> Xen-api mailing list >> Xen-api@xxxxxxxxxxxxx >> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api _______________________________________________ Xen-api mailing list Xen-api@xxxxxxxxxxxxx http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |