cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valery Fongang <fongang.val...@pertechtual.com>
Subject RE: NIC / VLAN ISSUE ON KVM HOST
Date Tue, 19 Mar 2013 10:23:48 GMT
Hi Bjoern,

Thanks for the feedback. Let me try to get my head around what you explained and I will get
back to you later...

Thanks,

-----Original Message-----
From: Bjoern Teipel [mailto:bjoern.teipel@internetbrands.com] 
Sent: 16 March 2013 06:08
To: Valery Fongang
Cc: 'Ahmad Emneina'; 'Marcus Sorensen'; 'Edison Su'; 'Cloudstack users'
Subject: Re: NIC / VLAN ISSUE ON KVM HOST

Hi Valery,

here is the network layout I did (Advanced, shared network) :

KVM :

bond0  (also eth0 or any other Ethernet device works)
+
+----- bond0.x (x = Network VLANID ) -----> cloud-mgmt (Linux bridge)
+
+----- bond0.y (another VLAN) ------- > cloud-storage (Linux bridge)
+
+----- cloudbr0 (Linux Bridge, just a helper)


CSTK :

Management using traffic/KVM label : cloud-mgmt (the Linux interface name, untagged (no tag
support anyway in CSTK)) Storage using traffic label : cloud-storage (untagged) Guest Network
using traffic label : cloudbr0 with VLANID A Public Network using traffic label : cloudbr0
with VLANID B

The magic begins with cloudbr0, because CSTK finds out the physical interface (bond0 here)
and creates new VLANs A and B to new bridges cloudVirBrA and cloudVirBrB.
Everything happens automatically so you only have to create the management ,storage and cloud0
bridges.


Here a real "brctl show" sample from my network:

cloud-mgmt        8000.00219b937e8c     no bond0.2610
cloud-storage     8000.00219b937e8c     no bond0.2578
cloudVirBr2562    8000.00219b937e8c     no bond0.2562
cloudVirBr2615    8000.00219b937e8c     no bond0.2615
cloudbr0          8000.00219b937e8c     no bond0

If you cloud0 is a bridge which connects the local link interface of the VM with the hypervisor.
Also automatically created by CSTK You can also check out https://cwiki.apache.org/confluence/display/CLOUDSTACK/Cloud+Operators
there are some nice examples shown for different network designs.
If you are not familiar with all that routing and VLAN and bridging stuff I can send you some
examples.

Have fun,
Bjoern

On 3/14/2013 5:59 AM, Valery Fongang wrote:
> Hi Bjoern,
>
> Thanks for your reply (and sorry for my late response) but I would like more indication
on how to do this: "Bridge on VLAN and don't use VLAN ID. That's the combination I choose
because I had trouble with the first solution.".
>
> My main goal is to implement Cloudstack with advanced networking with VLANs and I have
a single NIC on my host with no intention of separating the Storage LAN. As you may have mentioned
yes I have a physical network declarartion in CloudStack Management interface a range of VLAN
specified. I'm actually not too sure of what I'm doing apart from trying to get all this to
work with VLANs. I will be very happy if you can give me a detailed config example for my
Host NIC and Network in general.
>
>
>
>
> -----Original Message-----
> From: Bjoern Teipel [mailto:bjoern.teipel@internetbrands.com]
> Sent: 11 March 2013 19:36
> To: Ahmad Emneina
> Cc: Valery Fongang; Marcus Sorensen; Edison Su; Cloudstack users
> Subject: Re: NIC / VLAN ISSUE ON KVM HOST
>
> Looks like you specified a network in CS, probably the storage, using 
> a VLAN ID (VID) but you have the vlan ID already bound to a cloudbr 
> bridge. You can only do either one,
>
> Bridge on native ETH/bond interface and use Vlan ID in CS. CS will do 
> the rest. I had problems using a VID on the storage network
>
> Or
>
> Bridge on VLAN and don't use VLAN ID. That's the combination I choose because I had trouble
with the first solution.
>
> Guest and public networks worked fine for me using VID in CS.
>
> Bjoern
>
> Ahmad Emneina <aemneina@gmail.com> wrote:
>
>> adding the KVM gurus. Edison and Marcus. They are heavily active on 
>> the dev list relating to KVM.
>>
>>
>> On Mon, Mar 11, 2013 at 3:16 AM, Valery Fongang < 
>> fongang.valery@pertechtual.net> wrote:
>>
>>> Hi,
>>>
>>> I have some issues sorting out how I need to set my NIC on my KVM 
>>> Host for CloudStack to work correctly. Here is my setup:
>>>
>>> - 1 KVM (Centos 6.3) host with a single NIC and I am using 
>>> SharedMointPoint option for my Primary Storage to leverage Local 
>>> Storage on this server.
>>> - 1 VM acting as CMS on a different Physical Server. NFS Secondary 
>>> Storage is shared from this VM.
>>>
>>> My Network settings on the KVM host are as follow and at the bottom 
>>> my CMS log. How am I supposed to set my network to prevent having 
>>> the error in this log?
>>>
>>> #######################Network Config ###########################
>>>
>>> cloud0    Link encap:Ethernet  HWaddr 76:D4:8A:DB:9C:B3
>>>            inet addr:169.254.0.1  Bcast:169.254.255.255  Mask:255.255.0.0
>>>            inet6 addr: fe80::74d4:8aff:fedb:9cb3/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:0 (0.0 b)  TX bytes:468 (468.0 b)
>>>
>>> cloudbr0  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet addr:10.26.0.143  Bcast:10.26.255.255  Mask:255.255.0.0
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:3215 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:2148 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:318720 (311.2 KiB)  TX bytes:1490094 (1.4 MiB)
>>>
>>> cloudbr1  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:0 (0.0 b)  TX bytes:578 (578.0 b)
>>>
>>> eth0      Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:5783 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:7097 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:1000
>>>            RX bytes:640522 (625.5 KiB)  TX bytes:1838861 (1.7 MiB)
>>>            Interrupt:34
>>>
>>> eth0.1    Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:4798 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:3310 errors:0 dropped:6 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:514130 (502.0 KiB)  TX bytes:1551030 (1.4 MiB)
>>>
>>> eth0.200  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:2601 errors:0 dropped:7 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:0 (0.0 b)  TX bytes:225281 (220.0 KiB)
>>>
>>> eth0.300  Link encap:Ethernet  HWaddr C8:60:00:58:9F:13
>>>            inet6 addr: fe80::ca60:ff:fe58:9f13/64 Scope:Link
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:1179 errors:0 dropped:3 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:0 (0.0 b)  TX bytes:61972 (60.5 KiB)
>>>
>>> lo        Link encap:Local Loopback
>>>            inet addr:127.0.0.1  Mask:255.0.0.0
>>>            inet6 addr: ::1/128 Scope:Host
>>>            UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>>            RX packets:2 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:190 (190.0 b)  TX bytes:190 (190.0 b)
>>>
>>> virbr0    Link encap:Ethernet  HWaddr 52:54:00:93:DC:18
>>>            inet addr:192.168.122.1  Bcast:192.168.122.255
>>>   Mask:255.255.255.0
>>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>>            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>>            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>>            collisions:0 txqueuelen:0
>>>            RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b) 
>>> ###################### brctl show ###################### [root@kvm 
>>> home]# brctl show
>>> bridge name     bridge id               STP enabled     interfaces
>>> cloud0          8000.000000000000       no
>>> cloudVirBr1             8000.000000000000       no
>>> cloudbr0                8000.c86000589f13       yes             eth0.1
>>>                                                          eth0.200
>>> cloudbr1                8000.c86000589f13       yes             eth0.300
>>> virbr0          8000.52540093dc18       yes             virbr0-nic
>>>
>>>
>>> ##################### Recent Error Log################
>>>
>>> 2013-03-11 09:56:17,421 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Done executing storage capacity update
>>> 2013-03-11 09:56:17,421 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Executing capacity updates for public ip and 
>>> Vlans
>>> 2013-03-11 09:56:17,429 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Done capacity updates for public ip and 
>>> Vlans
>>> 2013-03-11 09:56:17,429 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Executing capacity updates for private ip
>>> 2013-03-11 09:56:17,432 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Done executing capacity updates for private 
>>> ip
>>> 2013-03-11 09:56:17,432 DEBUG [cloud.alert.AlertManagerImpl]
>>> (catalina-exec-11:null) Done recalculating system capacity
>>> 2013-03-11 09:56:21,888 DEBUG [cloud.server.StatsCollector]
>>> (StatsCollector-2:null) VmStatsCollector is running...
>>> 2013-03-11 09:56:23,743 DEBUG [agent.transport.Request]
>>> (AgentManager-Handler-5:null) Seq 5-1100559375: Processing:  { Ans: 
>>> ,
>>> MgmtId: 16279293006507, via: 5, Ver: v1, Flags: 110, 
>>> [{"StartAnswer":{"vm":{"id":16209,"name":"s-16209-VM","type":"Second
>>> a
>>> ryStorageVm","cpus":1,"speed":500,"minRam":268435456,"maxRam":268435
>>> 4 56,"arch":"x86_64","os":"Debian GNU/Linux 5.0 
>>> (32-bit)","bootArgs":"
>>> template=domP type=secstorage
>>> host=10.26.0.90 port=8250 name=s-16209-VM zone=4 pod=4 
>>> guid=s-16209-VM 
>>> resource=com.cloud.storage.resource.PremiumSecondaryStorageResource
>>> instance=SecStorage sslcopy=true role=templateProcessor mtu=1500
>>> eth2ip=172.18.1.15 eth2mask=255.255.0.0 eth2mask=255.255.0.0
>>> gateway=172.18.1.1 public.network.device=eth2 eth0ip=169.254.1.144
>>> eth0mask=255.255.0.0 eth0mask=255.255.0.0 eth1ip=10.26.0.136
>>> eth1mask=255.255.0.0 eth1mask=255.255.0.0
>>> mgmtcidr=10.26.0.0/16localgw=10.26.0.1 private.network.device=eth1
>>> eth3ip=10.26.0.134
>>> eth3mask=255.255.0.0 eth3mask=255.255.0.0 storageip=10.26.0.134
>>> storagenetmask=255.255.0.0 storagegateway=10.26.0.1
>>> internaldns1=10.26.0.160 internaldns2=8.8.8.8 dns1=10.26.0.160 
>>> dns2=8.8.8.8","rebootOnCrash":false,"enableHA":false,"limitCpuUse":f
>>> a
>>> lse,"vncPassword":"be60d6697a342397","params":{},"disks":[{"id":1620
>>> 9 
>>> ,"name":"/home","mountPoint":"efd20cac-c832-4a57-88bf-ff939e25943b","
>>> path":"efd20cac-c832-4a57-88bf-ff939e25943b","size":139264,"type":"R
>>> O
>>> OT","storagePoolType":"SharedMountPoint","storagePoolUuid":"390801ff
>>> - 
>>> 45b9-4177-9620-b863a3358b2d","deviceId":0}],"nics":[{"deviceId":2,"n
>>> e
>>> tworkRateMbps":-1,"defaultNic":true,"ip":"172.18.1.15","netmask":"25
>>> 5
>>> .255.0.0","gateway":"172.18.1.1","mac":"06:07:fc:00:00:11","dns1":"1
>>> 0 
>>> .26.0.160","dns2":"8.8.8.8","broadcastType":"Vlan","type":"Public","
>>> b 
>>> roadcastUri":"vlan://1","isolationUri":"vlan://1","isSecurityGroupEn
>>> a 
>>> bled":false},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"
>>> i 
>>> p":"169.254.1.144","netmask":"255.255.0.0","gateway":"169.254.0.1","
>>> m 
>>> ac":"0e:00:a9:fe:01:90","broadcastType":"LinkLocal","type":"Control"
>>> , 
>>> "isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"
>>> d 
>>> efaultNic":false,"ip":"10.26.0.136","netmask":"255.255.0.0","gateway"
>>> :"10.26.0.1","mac":"06:2d:d8:00:00:07","broadcastType":"Native","typ
>>> e 
>>> ":"Management","isSecurityGroupEnabled":false},{"deviceId":3,"networ
>>> k
>>> RateMbps":-1,"defaultNic":false,"ip":"10.26.0.134","netmask":"255.25
>>> 5 
>>> .0.0","gateway":"10.26.0.1","mac":"06:a5:3e:00:00:05","broadcastType"
>>> :"Native","type":"Storage","isSecurityGroupEnabled":false}]},"result"
>>> :false,"details":"Failed to create vnet 1: Set name-type for VLAN 
>>> subsystem. Should be visible in /proc/net/vlan/configdevice eth0.1 
>>> is already a member of a bridge; can't enslave it to bridge 
>>> cloudVirBr1.Failed to add vlan: eth0.1 to 
>>> cloudVirBr1","wait":0}},{"Answer":{"result":false,"details":"Stopped
>>> by previous failure","wait":0}}] }
>>> 2013-03-11 09:56:23,743 DEBUG [agent.transport.Request]
>>> (secstorage-1:null) Seq 5-1100559375: Received:  { Ans: , MgmtId:
>>> 16279293006507, via: 5, Ver: v1, Flags: 110, { StartAnswer, Answer } 
>>> }
>>> 2013-03-11 09:56:23,743 WARN  [cloud.vm.VirtualMachineManagerImpl]
>>> (secstorage-1:null) Cleanup failed due to Failed to create vnet 1:
>>> Set name-type for VLAN subsystem. Should be visible in 
>>> /proc/net/vlan/configdevice eth0.1 is already a member of a bridge; 
>>> can't enslave it to bridge cloudVirBr1.Failed to add vlan: eth0.1 to
>>> cloudVirBr1
>>> 2013-03-11 09:56:23,743 WARN  [cloud.vm.VirtualMachineManagerImpl]
>>> (secstorage-1:null) Cleanup failed due to Stopped by previous 
>>> failure
>>> 2013-03-11 09:56:23,744 WARN  [cloud.vm.VirtualMachineManagerImpl]
>>> (AgentManager-Handler-5:null) Cleanup failed due to Failed to create 
>>> vnet
>>> 1: Set name-type for VLAN subsystem. Should be visible in 
>>> /proc/net/vlan/configdevice eth0.1 is already a member of a bridge; 
>>> can't enslave it to bridge cloudVirBr1.Failed to add vlan: eth0.1 to
>>> cloudVirBr1
>>> 2013-03-11 09:56:23,744 WARN  [cloud.vm.VirtualMachineManagerImpl]
>>> (AgentManager-Handler-5:null) Cleanup failed due to Stopped by 
>>> previous failure
>>> 2013-03-11 09:56:23,744 DEBUG [agent.manager.AgentAttache]
>>> (AgentManager-Handler-5:null) Seq 5-1100559376: Sending now.  is 
>>> current sequence.
>>> 2013-03-11 09:56:23,750 INFO  [cloud.vm.VirtualMachineManagerImpl]
>>> (secstorage-1:null) Unable to start VM on Host[-5-Routing] due to 
>>> Failed to create vnet 1: Set name-type for VLAN subsystem. Should be 
>>> visible in /proc/net/vlan/configdevice eth0.1 is already a member of 
>>> a bridge; can't enslave it to bridge cloudVirBr1.Failed to add vlan:
>>> eth0.1 to cloudVirBr1
>>> 2013-03-11 09:56:23,758 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (secstorage-1:null) Cleaning up resources for the vm 
>>> VM[SecondaryStorageVm|s-16209-VM] in Starting state
>>> 2013-03-11 09:56:23,760 DEBUG [agent.transport.Request]
>>> (secstorage-1:null) Seq 5-1100559384: Waiting for Seq 1100559376
>>> Scheduling:  { Cmd , MgmtId: 16279293006507, via: 5, Ver: v1, Flags:
>>> 100111,
>>> [{"StopCommand":{"isProxy":false,"vmName":"s-16209-VM","wait":0}}] }
>>> 2013-03-11 09:56:24,213 DEBUG
>>> [network.router.VirtualNetworkApplianceManagerImpl]
>>> (RouterStatusMonitor-1:null) Found 0 routers.
>>> 2013-03-11 09:56:25,364 DEBUG [agent.manager.AgentManagerImpl]
>>> (AgentManager-Handler-6:null) Ping from 5
>>> 2013-03-11 09:56:37,004 DEBUG [cloud.server.StatsCollector]
>>> (StatsCollector-2:null) HostStatsCollector is running...
>>> 2013-03-11 09:56:37,598 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (AgentManager-Handler-7:null) Cleanup succeeded. Details null
>>> 2013-03-11 09:56:37,598 DEBUG [agent.transport.Request]
>>> (StatsCollector-2:null) Seq 5-1100559385: Received:  { Ans: , MgmtId:
>>> 16279293006507, via: 5, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
>>>
>>>
>>>

Mime
View raw message