cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Hohman <mhoh...@newheights.org>
Subject Re: Help with advanced zone kvm networking setup.
Date Tue, 22 Jan 2013 00:22:46 GMT

I must be missing a step here. I just completed a complete reinstall of both the management
server and the 1st kvm host. Everything boots as usual except I can't connect to the console
proxy and I am unable to ssh into either the ssvm or console proxy.

I'm following the cloudstack documentation:
http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Installation_Guide/management-server-install-flow.html

I'm using a stock install of ubuntu 12.04.01 for both management server and kvm hosts.
I've attached my .bash_history from the management server

passwd root
exit
echo "deb http://cloudstack.apt-get.eu/ubuntu precise 4.0" >> /etc/apt/sources.list
wget -O - http://cloudstack.apt-get.eu/release.asc|apt-key add -
apt-get -y update
apt-get -y upgrade
apt-get install cloud-client
apt-get install mysql-server
nano /etc/mysql/my.cnf
/etc/init.d/mysql start
cloud-setup-databases cloud:######@localhost --deploy-as=root:##### -m ###### -k ######
mkdir -p /mnt/secondary
mount -t nfs 10.0.220.6:/mnt/store1/sec /mnt/secondary
cloud-setup-management
/usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary
-u http://download.cloud.com/templates/acton/acton-systemvm-02062012.qcow2.bz2 -h kvm -s ########
-F
reboot

I use the following to configure my kvm hosts.

echo "deb http://cloudstack.apt-get.eu/ubuntu precise 4.0" >> /etc/apt/sources.list
wget -O - http://cloudstack.apt-get.eu/release.asc|apt-key add -
apt-get -y update
apt-get -y upgrade
echo "127.0.0.1 localhost" > /etc/hosts
echo "127.0.1.1     1955-03.newheights.org 1955-03" >> /etc/hosts
apt-get -y install openntpd cloud-agent
echo "listen_tls = 0" >> /etc/libvirt/libvirtd.conf
echo "listen_tcp = 1" >> /etc/libvirt/libvirtd.conf
echo "tcp_port = 16059" >> /etc/libvirt/libvirtd.conf
echo "auth_tcp = \"none\"" >> /etc/libvirt/libvirtd.conf
echo "mdns_adv = 0" >> /etc/libvirt/libvirtd.conf
nano /etc/init/libvirt-bin.conf  (Add -l to opts)
ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/
ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/
apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
ufw allow proto tcp from any to any port 22
ufw allow proto tcp from any to any port 1798
ufw allow proto tcp from any to any port 16509
ufw allow proto tcp from any to any port 5900:6100
ufw allow proto tcp from any to any port 49152:49216
echo "auto lo" > /etc/network/interfaces
echo "iface lo inet loopback" >> /etc/network/interfaces
echo "" >> /etc/network/interfaces
echo "auto eth0" >> /etc/network/interfaces
echo "iface eth0 inet manual" >> /etc/network/interfaces
echo "" >> /etc/network/interfaces
echo "auto eth1" >> /etc/network/interfaces
echo "iface eth1 inet manual" >> /etc/network/interfaces
echo "" >> /etc/network/interfaces
echo "auto cloudbr0" >> /etc/network/interfaces
echo "iface cloudbr0 inet dhcp" >> /etc/network/interfaces
echo "bridge_ports eth0" >> /etc/network/interfaces
echo "" >> /etc/network/interfaces
echo "auto cloudbr1" >> /etc/network/interfaces
echo "iface cloudbr1 inet manual" >> /etc/network/interfaces
echo "bridge_ports eth1" >> /etc/network/interfaces
reboot

I'm really at a loss here.


Thanks,
Matt Hohman


On Jan 21, 2013, at 2:33 PM, Geoff Higginbottom <geoff.higginbottom@shapeblue.com> wrote:

> Hi Matt,
> 
> Just to answer an earlier question, as confirmed by Paul, yes the labels for both Public
and Guest should be 'cloudbr1' as they are both using the same bridge.
> 
> If you dropped the Storage VLAN after creating the System VMs, first restart your management
service 'service cloud-management restart' then destroy both the Sec Storage VM and Console
Proxy VM and once they have re-deployed try logging onto them again.
> 
> Regards
> 
> Geoff Higginbottom
> 
> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
> 
> geoff.higginbottom@shapeblue.com
> 
> 
> -----Original Message-----
> From: Matt Hohman [mailto:mhohman@newheights.org]
> Sent: 21 January 2013 22:27
> To: cloudstack-users@incubator.apache.org
> Subject: Re: Help with advanced zone kvm networking setup.
> 
> Well this is odd. When I try to login to the ssvm via the host it's running on I get
the following.
> 
> root@1955-01:~# ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@169.254.2.171 The authenticity
of host '[169.254.2.171]:3922 ([169.254.2.171]:3922)' can't be established.
> RSA key fingerprint is 58:de:8b:d7:2b:8e:01:2e:cd:f4:58:86:27:e4:94:39.
> Are you sure you want to continue connecting (yes/no)? yes
> Warning: Permanently added '[169.254.2.171]:3922' (RSA) to the list of known hosts.
> Permission denied (publickey).
> 
> root@1955-01:~# ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@169.254.2.171 Permission
denied (publickey).
> 
> Same happens with the console vm
> 
> 
> Thanks,
> Matt Hohman
> New Heights Church
> 
> 
> 
> On Jan 21, 2013, at 1:56 PM, Paul Angus <paul.angus@shapeblue.com> wrote:
> 
>> Matt,
>> 
>> Your network labels look fine. Have you tried connecting to the SSVM and running
the test script?
>> 
>> ssh -i /var/lib/cloud/management/.ssh/id_rsa -p 3922
>> root@[private-ip-of-SSVM] (from the KVM host which the SSVM is on)
>> 
>> Then run:
>> /usr/local/cloud/systemvm/ssvm-check.sh
>> 
>> It might give you some pointers
>> 
>> 
>> Regards,
>> 
>> Paul Angus
>> S: +44 20 3603 0540 | M: +447711418784 paul.angus@shapeblue.com
>> 
>> -----Original Message-----
>> From: Matt Hohman [mailto:mhohman@newheights.org]
>> Sent: 21 January 2013 21:30
>> To: cloudstack-users@incubator.apache.org
>> Subject: Re: Help with advanced zone kvm networking setup.
>> 
>> Well, I got the zone,cluster,pod,and host added everything looked like it was running
but I couldn't get the ssvm to download templates or iso's and I couldn't connect to the console
proxy.
>> 
>> I was able to ping both system vm's on both their private ip's and on their public
ip's.
>> 
>> I think I'm still having an issue with my network setup.
>> 
>> I went ahead and dropped the separate storage vlan. So currently my machines have
the following network connections.
>> 
>> eth0 - Management Untagged (10.0.220.0/24)
>> 
>> eth1 - Public vlan 101 (107.xxx.xxx.0/24)
>> eth1 - Guest vlan 3100 - 3199 (10.0.222.0/24)
>> 
>> Here is a copy of my /etc/network/interfaces that I created on each kvm host:
>> 
>> ### /etc/network/interfaces ###
>> auto lo
>> iface lo inet loopback
>> 
>> auto eth0
>> iface eth0 inet manual
>> 
>> auto eth1
>> iface eth1 inet manual
>> 
>> auto cloudbr0
>> iface cloudbr0 inet dhcp
>> bridge_ports eth0
>> 
>> auto cloudbr1
>> iface cloudbr1 inet manual
>> bridge_ports eth1
>> ### EOF ###
>> 
>> Here is the results of an ifconfig from the host running the 2 system vm's:
>> 
>> ### ifconfig ###
>> cloud0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00
>>         inet addr:169.254.0.1  Bcast:169.254.255.255  Mask:255.255.0.0
>>         inet6 addr: fe80::4cce:e2ff:fe44:57ad/64 Scope:Link
>>         UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>         RX packets:189 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:234 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:30329 (30.3 KB)  TX bytes:38619 (38.6 KB)
>> 
>> cloudVirBr101 Link encap:Ethernet  HWaddr 00:14:22:74:ad:36
>>         inet6 addr: fe80::214:22ff:fe74:ad36/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:3107 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:145804 (145.8 KB)  TX bytes:3048 (3.0 KB)
>> 
>> cloudbr0  Link encap:Ethernet  HWaddr 00:14:22:74:ad:34
>>         inet addr:10.0.220.102  Bcast:10.0.220.255  Mask:255.255.255.0
>>         inet6 addr: fe80::214:22ff:fe74:ad34/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:830799 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:263529 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:914312894 (914.3 MB)  TX bytes:785374708 (785.3 MB)
>> 
>> cloudbr1  Link encap:Ethernet  HWaddr 00:14:22:74:ad:36
>>         inet6 addr: fe80::214:22ff:fe74:ad36/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:391 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:41 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:19382 (19.3 KB)  TX bytes:3478 (3.4 KB)
>> 
>> eth0      Link encap:Ethernet  HWaddr 00:14:22:74:ad:34
>>         inet6 addr: fe80::214:22ff:fe74:ad34/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:1220462 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:767547 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:1000
>>         RX bytes:956567228 (956.5 MB)  TX bytes:821714262 (821.7 MB)
>>         Interrupt:16 Memory:f8000000-f8012800
>> 
>> eth1      Link encap:Ethernet  HWaddr 00:14:22:74:ad:36
>>         inet6 addr: fe80::214:22ff:fe74:ad36/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:6536 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:2012 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:1000
>>         RX bytes:568564 (568.5 KB)  TX bytes:205654 (205.6 KB)
>>         Interrupt:16 Memory:f4000000-f4012800
>> 
>> eth1.101  Link encap:Ethernet  HWaddr 00:14:22:74:ad:36
>>         inet6 addr: fe80::214:22ff:fe74:ad36/64 Scope:Link
>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>         RX packets:6060 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:1929 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:381208 (381.2 KB)  TX bytes:182890 (182.8 KB)
>> 
>> lo        Link encap:Local Loopback
>>         inet addr:127.0.0.1  Mask:255.0.0.0
>>         inet6 addr: ::1/128 Scope:Host
>>         UP LOOPBACK RUNNING  MTU:16436  Metric:1
>>         RX packets:479 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:479 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:38944 (38.9 KB)  TX bytes:38944 (38.9 KB)
>> 
>> virbr0    Link encap:Ethernet  HWaddr fa:c3:96:00:b4:14
>>         inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
>>         UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>         collisions:0 txqueuelen:0
>>         RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>> 
>> ### EOF ###
>> 
>> I'm a little confused where virbr0 is coming from as I haven't used that ip subnet
in any of the config.
>> 
>> My assumption when I'm creating the zone is to have the networks
>> tagged as follows
>> 
>> Management - cloudbr0
>> Guest - cloudbr1
>> Public cloudbr1
>> 
>> Is that correct? Should both of the Guest and Public networks be set to the same
label since they are both tagged vlans on the same physical interface?
>> 
>> Thanks,
>> Matt Hohman
>> New Heights Church
>> 
>> 
>> 
>> On Jan 21, 2013, at 9:04 AM, Geoff Higginbottom <geoff.higginbottom@shapeblue.com>
wrote:
>> 
>>> No problem Matt,
>>> 
>>> Just reach out to the community whenever you get any issues and we will try to
help you where we can.
>>> 
>>> Regards
>>> 
>>> Geoff Higginbottom
>>> 
>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
>>> 
>>> geoff.higginbottom@shapeblue.com
>>> 
>>> -----Original Message-----
>>> From: Matt Hohman [mailto:mhohman@newheights.org]
>>> Sent: 21 January 2013 15:54
>>> To: cloudstack-users@incubator.apache.org
>>> Subject: Re: Help with advanced zone kvm networking setup.
>>> 
>>> Geoff,
>>> Thanks for the clarity. I wish I would have asked this question a couple weeks
ago.
>>> 
>>> Thanks,
>>> Matt Hohman
>>> Technical Ministries
>>> New Heights Church
>>> 
>>> On Jan 21, 2013, at 1:43 AM, Geoff Higginbottom <geoff.higginbottom@shapeblue.com>
wrote:
>>> 
>>>> Hi Nate,
>>>> 
>>>> The main problem I see with your design is with relation to the Storage Network
using a VLAN.
>>>> 
>>>> There is a lot of confusion within CloudStack relating to the 'Storage Network'
as it relates to Secondary and NOT Primary Storage.
>>>> 
>>>> If you choose to use the Storage Network, it is only worthwhile doing so
if you have NIC(s) you can dedicate to it, once configured it is used by the SSVM to transfer
Snapshots, Templates etc to and from Secondary Storage.  If you place it on a unique VLAN
and IP Range, the SSVM AND the CloudStack management Server must be able to communicate with
it.
>>>> 
>>>> With regard to VLANs, just make sure the switch ports are configured
>>>> for the VLAN IDs you want to use, CloucStack will configure the
>>>> VLANs on the Hypervisors for you
>>>> 
>>>> I suggest you take a look at the following
>>>> 
>>>> Paul Angus' Blog on understanding the physical network
>>>> http://www.shapeblue.com/2013/01/07/understanding-cloudstacks-physic
>>>> a
>>>> l
>>>> -networking-architecture/
>>>> 
>>>> You may also find My blog on CloudStack Networking usefull, a little
>>>> old now but still pertinent
>>>> http://www.shapeblue.com/2012/05/01/cloudstack-networking-considerat
>>>> i
>>>> o
>>>> ns/
>>>> 
>>>> There is also a recording of the 'Introduction to Networking' talk I
>>>> gave at the CloudStack Collaboration Conference in Las Vegas
>>>> recently
>>>> http://www.youtube.com/watch?v=GIV9D2jYid0
>>>> 
>>>> 
>>>> Regards
>>>> 
>>>> Geoff Higginbottom
>>>> 
>>>> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
>>>> 
>>>> geoff.higginbottom@shapeblue.com
>>>> 
>>>> -----Original Message-----
>>>> From: Matt Hohman [mailto:mhohman@newheights.org]
>>>> Sent: 21 January 2013 03:45
>>>> To: cloudstack-users@incubator.apache.org
>>>> Cc: Nate Vick
>>>> Subject: Help with advanced zone kvm networking setup.
>>>> 
>>>> Hey All,
>>>> 
>>>> I'm looking for some advice on setting up our advanced zone for our small
private cloud. Here is a rundown of how we are trying to get this going.
>>>> 
>>>> 5 KVM Ubuntu 12.03 Hypervisors with 2 nics each
>>>> 
>>>> on eth0
>>>> 
>>>> management network is untagged 10.0.220.0/24 vlan 3001 tagged
>>>> storage
>>>> 10.0.221.0/24
>>>> 
>>>> on eth1
>>>> vlan 101 tagged Public 107.xxx.xxx.0/24 vlan 3100 - 3199 tagged
>>>> Guest
>>>> 10.0.223.0/24
>>>> 
>>>> KVM Hots are on:
>>>> 10.0.220.101 mngt 10.0.221.101 storage
>>>> 10.0.220.102 mngt 10.0.221.102 storage
>>>> 10.0.220.103 mngt 10.0.221.103 storage
>>>> 10.0.220.104 mngt 10.0.221.104 storage
>>>> 10.0.220.105 mngt 10.0.221.105 storage
>>>> 
>>>> Cloudstack 4.0 is installed on Ubuntu 12.04 on ip 10.0.220.2 NFS is
>>>> 10.0.221.200
>>>> 
>>>> Does this network scheme work well for cloudstack? Is there something that
should be changed?
>>>> 
>>>> I'm a little confused on how to set this setup properly in /etc/network/interfaces.
Do I need to manually define all 100 guest vlans? How do I setup the guest bridge?
>>>> 
>>>> When I'm adding the zone to cloudstack what should my network labels be and
do I assign these in the /etc/network/interfaces file?
>>>> 
>>>> Here is my attempt at /etc/network/interfaces for host1
>>>> 
>>>> ### /etc/network/interfaces
>>>> auto lo
>>>> iface lo inet loopback
>>>> 
>>>> # Hypervisor Management
>>>> auto eth0
>>>> iface eth0 inet static
>>>> address 10.0.220.101
>>>> netmask 255.255.255.0
>>>> gateway 10.0.220.1
>>>> dns-nameservers 8.8.8.8 8.8.4.4
>>>> 
>>>> #cloud-store storage network
>>>> auto eth0.3001
>>>> iface eth0.3001 inet static
>>>> address 10.0.221.101
>>>> netmask 255.255.255.0
>>>> 
>>>> 
>>>> # Public network
>>>> auto cloudbr0
>>>> iface cloudbr0 inet manual
>>>> bridge_ports eth1.101
>>>> bridge_fd 5
>>>> bridge_stp off
>>>> bridge_maxwait 1
>>>> ### EOF
>>>> 
>>>> Any help would be greatly appreciated!
>>>> 
>>>> Thanks,
>>>> Matt Hohman
>>>> Technical Ministries
>>>> New Heights Church
>>>> ShapeBlue provides a range of strategic and technical consulting and implementation
services to help IT Service Providers and Enterprises to build a true IaaS compute cloud.
ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers
and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
>>>> 
>>>> ________________________________
>>>> 
>>>> This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or opinions expressed
are solely those of the author and do not necessarily represent those of Shape Blue Ltd. If
you are not the intended recipient of this email, you must neither take any action based upon
its contents, nor copy or show it to anyone. Please contact the sender if you believe you
have received this email in error. Shape Blue Ltd is a company incorporated in England &
Wales.
>>> 
>>> 
>>> ShapeBlue provides a range of strategic and technical consulting and implementation
services to help IT Service Providers and Enterprises to build a true IaaS compute cloud.
ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers
and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
>>> 
>>> ________________________________
>>> 
>>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you
are not the intended recipient of this email, you must neither take any action based upon
its contents, nor copy or show it to anyone. Please contact the sender if you believe you
have received this email in error. Shape Blue Ltd is a company incorporated in England &
Wales.
>> 
>> 
>> ShapeBlue provides a range of strategic and technical consulting and implementation
services to help IT Service Providers and Enterprises to build a true IaaS compute cloud.
ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers
and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
>> 
>> ________________________________
>> 
>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you
are not the intended recipient of this email, you must neither take any action based upon
its contents, nor copy or show it to anyone. Please contact the sender if you believe you
have received this email in error. Shape Blue Ltd is a company incorporated in England &
Wales.
> 
> 
> ShapeBlue provides a range of strategic and technical consulting and implementation services
to help IT Service Providers and Enterprises to build a true IaaS compute cloud. ShapeBlue’s
expertise, combined with CloudStack technology, allows IT Service Providers and Enterprises
to deliver true, utility based, IaaS to the customer or end-user.
> 
> ________________________________
> 
> This email and any attachments to it may be confidential and are intended solely for
the use of the individual to whom it is addressed. Any views or opinions expressed are solely
those of the author and do not necessarily represent those of Shape Blue Ltd. If you are not
the intended recipient of this email, you must neither take any action based upon its contents,
nor copy or show it to anyone. Please contact the sender if you believe you have received
this email in error. Shape Blue Ltd is a company incorporated in England & Wales.


Mime
View raw message