Return-Path: X-Original-To: apmail-incubator-cloudstack-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-cloudstack-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5D1B595DD for ; Wed, 16 May 2012 16:24:42 +0000 (UTC) Received: (qmail 87573 invoked by uid 500); 16 May 2012 16:24:42 -0000 Delivered-To: apmail-incubator-cloudstack-dev-archive@incubator.apache.org Received: (qmail 87544 invoked by uid 500); 16 May 2012 16:24:42 -0000 Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cloudstack-dev@incubator.apache.org Delivered-To: mailing list cloudstack-dev@incubator.apache.org Received: (qmail 87536 invoked by uid 99); 16 May 2012 16:24:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 May 2012 16:24:42 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mhartmann@tls.net designates 65.124.104.104 as permitted sender) Received: from [65.124.104.104] (HELO smtp1.tls.net) (65.124.104.104) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 May 2012 16:24:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tls.net; s=tlsnet-v1; h=Content-Type:In-Reply-To:References:Subject:To:MIME-Version:From:Date:Message-ID; bh=G/5gwjsFOmPBwW4L13vmlUyZmxy8sYf8WOTE8ooRcno=; b=oAlOzhVzJkzjuwidr13y8NAA0Ie1x/Um6tYbdEX5MIZOYJu5oobmYfRwvS26cbumRvfBwVhhtvauJF2gD4+4FkOCzXpVUAjRftz+d92al15p7vgoIBrQ10nPtAMoqRU8rJV4C9vAObbcrZ8VZJdF1hxfw33fQxZlSV+XD6CKS8I=; Received: from tls-office.tls.net ([65.124.104.200] helo=[192.168.32.50]) by smtp1.tls.net with esmtpa (Exim 4.76) (envelope-from ) id 1SUh1I-0004Uo-EJ for cloudstack-dev@incubator.apache.org; Wed, 16 May 2012 12:24:15 -0400 Message-ID: <4FB3D4AB.4060906@tls.net> Date: Wed, 16 May 2012 12:24:11 -0400 From: Matthew Hartmann Organization: TLS.NET, Inc. User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: cloudstack-dev@incubator.apache.org Subject: Re: Two Issues with XenServer 6.0.2 and CS 3.0.1 References: <4FB150C9.5000905@tls.net> <4FB151FB.1040100@tls.net> <4FB15C8F.6010602@tls.net> ,<4FB16722.1030703@tls.net> <5903BB99-B5B4-4840-A5F0-CE67606594D2@shapeblue.com>,<4FB172A3.20801@tls.net> <3A7910D6-4B48-40D4-AAEA-785D5FAF14D2@shapeblue.com> <4FB3B2E9.7010704@tls.net> In-Reply-To: <4FB3B2E9.7010704@tls.net> Content-Type: multipart/alternative; boundary="------------030501000602080609080608" X-TLSNET-SMTP-Spam-Score: -1.0 (-) X-TLSNET-SMTP-Spam-Report: Content analysis details: (-1.0 points, 10.0 required) pts rule name description ---- ---------------------- ----------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 0.0 HTML_MESSAGE BODY: HTML included in message X-Virus-Checked: Checked by ClamAV on apache.org --------------030501000602080609080608 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Just an FYI, I had to manually scrub the database then re-add them via the CS UI. Things are coming back to life very slowly. Thanks again for your assistance! :) Cheers, Matthew On 5/16/2012 10:00 AM, Matthew Hartmann wrote: > Well, I'm at the point I'm ready to remove Primary & Secondary > Storage, however CloudStack refuses to remove them even though there > are 0 hosts configured. CloudStack reports when trying to remove > Primary Storage: "Failed to delete storage pool" > > When trying to delete Secondary Storage CloudStack reports: "Can not > delete this secondary storage due to there are still snapshots on it" > > I disabled the Pod and Zone before attempting to remove Primary and > Secondary Storage, as well as putting Primary Storage in Maintenance > Mode. > > Management Server log reports when trying to delete Secondary Storage: > > 2012-05-16 09:57:29,553 ERROR [cloud.api.ApiDispatcher] > (catalina-exec-24:null) Exception while executing DeleteHostCmd: > com.cloud.utils.exception.CloudRuntimeException: Can not delete this > secondary storage due to there are still snapshots on it > > Management Server log reports when trying to delete Primary Storage: > > 2012-05-16 09:58:53,600 WARN [cloud.storage.StorageManagerImpl] > (catalina-exec-2:null) Cannot delete pool > san-p1n1c1.iedc.cloud.tls.net as there are associated vols for this pool > 2012-05-16 09:58:53,601 WARN [cloud.api.ApiDispatcher] > (catalina-exec-2:null) class com.cloud.api.ServerApiException : Failed > to delete storage pool > > I suspect manual scrubbing of the database is in order? > > Cheers, > > Matthew > > > On 5/14/2012 5:17 PM, Geoff Higginbottom wrote: >> Matthew, >> >> Removing and then re-adding Pri Storage will not wipe the LUNs and >> the existing VHD files which are there, assuming the associated Guest >> Instances are still in the database should also remap. >> >> How to avoid this in the future? It's pretty rare for a XenServer >> cluster to bomb out in this way, but a 'standby cluster' would not >> really help as that would have its own Storage Pool. However having >> at least two Primary Storage pools in the Cluster (eg two iSCSI LUNs) >> will allow you to migrate Guest Instances between them should you >> detect any problems. >> >> Also having multiple PODs and Clusters and forcing the distribution >> of Guest Instances for a particular Account to be spread amongst >> them, rather than relying on the default 'Random' allocation policies >> reduces risks in the event of any failures. >> >> If you do have data on the existing Primary Storage and want to be >> �ber safe (my preferred stance) then copy the VHDs off first, you can >> always re-import them back into CloudStack via Templates if you have >> to, but I assume as there are no clients on this system yet the LUNs >> should be empty. >> >> Kind Regards >> >> Geoff Higginbottom >> CTO / Senior Consultant >> ShapeBlue Ltd >> >> geoff.higginbottom@shapeblue.com >> | www.shapeblue.com >> >> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS >> >> >> On 14 May 2012, at 22:02, "Matthew >> Hartmann"> wrote: >> >> On 5/14/2012 4:48 PM, Geoff Higginbottom wrote: >> I had assumed you had Primary and Secondary storage on different IP >> ranges to your Management Server which does need to be able to access >> Secondary Storage but not Primary Storage. >> >> That assumption is correct, Primary Storage is on a different subnet >> than Secondary Storage. Secondary Storage is sitting on the >> Management Network subnet. >> >> >> I now note your Primary Storage is iSCSI, and that you had a failure >> of all Hosts. >> >> When adding Storage and Hosts, you have to add Hosts, before you add >> Primary Storage, and you have to add Primary Storage before Secondary >> Storage. >> >> This is all taken care of by the Add Zone Wizard, but if you have a >> failure of all Hosts in a Zone then you need to remove both Primary >> and Secondary Storage, then add a Host, then add Primary Storage, >> then add Secondary storage. >> >> Templates, ISOs etc that are on Secondary will disappear from the >> GUI, but don't worry, they will be detected by the SSVM once it comes >> online, and they will then reappear in the GUI. >> >> If this is a simple test system then it may be easier to simply drop >> the DB Schema, and recreate it from scratch, or create a new Zone and >> let the wizard take care of everything. >> >> This actually was a system we had *just* placed into production. We >> luckily did not have any clients setup on this cloud yet. However, >> won't removing Primary Storage then re-adding it cause all data on >> the LUN's to be effectively wiped? >> >> How would you propose I avoid something like this in the future? A >> standby cluster? >> >> Thanks again for your help! >> >> Cheers, >> >> Matthew >> >> >> >> On 14 May 2012, at 21:12, "Matthew >> Hartmann"> wrote: >> >> >> On 5/14/2012 3:35 PM, Geoff Higginbottom wrote: >> Matthew, >> >> If I understand correctly, your storage is on a different IP Range to >> your Management Server and Hosts etc, if this is the case you need a >> Storage Network, which also needs to be accessible from the >> Management Server. If no Storage Network is configured, CloudStack >> will use the Management Network when trying to access Storage. >> Odd, did something change? The Management Server has never needed to >> access the storage network before. I have tried using XenServer name >> labels for the Storage Network, but to no avail. Why would the >> Management Server need to access my iSCSI SAN (Primary Storage)? I >> have confirmed my my Management Server can still access Secondary >> Storage (NFS Server). All devices have an FQDN which can be resolved >> from the XenServer and Management Server. >> >> Re http://bugs.cloudstack.org/browse/CS-14655 this is only required >> if you are using NIC Bonding and I do not believe you are >> Yes, we are using bonding but we are not using CSP. The bug report >> refers to bonding and XenServer 6.0.x but it is not clear whether the >> steps outlined should be taken regardless of whether or not CSP is >> being used in ones setup. >> >> Cheers, >> >> Matthew >> >> Regards >> >> Geoff >> >> >> -----Original Message----- >> From: Matthew Hartmann [mailto:mhartmann@tls.net] >> Sent: 14 May 2012 20:27 >> To: >> cloudstack-dev@incubator.apache.org >> Subject: Re: Two Issues with XenServer 6.0.2 and CS 3.0.1 >> >> Please see my responses inline, below: >> >> On 5/14/2012 3:18 PM, Geoff Higginbottom wrote: >> Matthew, >> >> It looks like you have forgot to label your physical networks and map >> them within the Zone Setup process, this is vital if you want to use >> dedicated NICs for any of the four network types, ie Management, >> Guest, Public or Storage. >> This was a pre-existing zone that was operating well. The XenServer >> pool freaked out and unraveled quicker than the fall of Rome. I >> performed a clean install on my XenServer's and am simply trying to >> re-add them back into the Pod. I'm only using one Node at this moment >> just so I can get things going. When I initially setup my XenServer's >> and the pool, I made copious note should something like this happen. >> Now that it has unfortunately happened, configuring my XenServer's >> the same as they were initially when everything was functional >> doesn't seem to have helped. >> >> Re the Cloud Supplemental Pack, this is only required if you are using >> Security Groups with a Basic Zone >> Right, but is the patch required regardless of whether or not CSP is >> installed? I have experienced a similar issue and I am *not* using CSP. >> >> Cheers, >> >> Matthew >> >> Regards >> >> Geoff >> >> >> -----Original Message----- >> From: Matthew Hartmann [mailto:mhartmann@tls.net] >> Sent: 14 May 2012 19:42 >> To: CloudStack user/admin discussions >> Cc: CloudStack Devs >> Subject: Fwd: Two Issues with XenServer 6.0.2 and CS 3.0.1 >> >> >> I forgot to include the following information that I found in the >> database after the host was added: >> >> mysql> select * from host where id=22 \G; >> [...] >> private_ip_address: 10.0.0.16 >> private_netmask: 255.255.254.0 >> private_mac_address: e8:9a:8f:22:9b:c4 >> storage_ip_address: 10.0.0.16 >> storage_netmask: 255.255.254.0 >> storage_mac_address: e8:9a:8f:22:9b:c4 >> storage_ip_address_2: 10.0.0.16 >> storage_mac_address_2: e8:9a:8f:22:9b:c4 >> storage_netmask_2: 255.255.254.0 >> [...] >> >> That information is incorrect, as the storage_ip_address and >> storage_ip_address_2 are two completely different subnets and >> CloudStack does not see these network devices. >> >> Thanks! >> >> Matthew >> >> -------- Original Message -------- >> Subject: Two Issues with XenServer 6.0.2 and CS 3.0.1 >> Date: Mon, 14 May 2012 14:36:57 -0400 >> From: Matthew Hartmann> >> Organization: TLS.NET, Inc. >> To: CloudStack user/admin discussions >> > >> >> CC: CloudStack >> Devs> >> >> >> >> First of all, it is not very clear whether or not this patch is >> required for XenServer with regard to the Cloud Supplemental Pack: >> http://bugs.cloudstack.org/browse/CS-14655 >> >> Could someone shed some light on this? >> >> Secondly, I'm having an issue with CloudStack trying to use my >> "cloud-private" network for Public, Guest, Management and Storage. >> >> Here is what my CS 3.0.1 Management Server log is showing: >> >> [java] INFO [xen.resource.CitrixResourceBase] (http-8080-6:) Private >> Network is cloud-private for host 10.0.0.16 [java] INFO >> [xen.resource.CitrixResourceBase] (http-8080-6:) Guest Network is >> cloud-private for host 10.0.0.16 [java] INFO >> [xen.resource.CitrixResourceBase] (http-8080-6:) Public Network is >> cloud-private for host 10.0.0.16 [java] INFO >> [xen.resource.CitrixResourceBase] (http-8080-6:) Storage Network 1 is >> cloud-private for host 10.0.0.16 [java] INFO >> [xen.resource.CitrixResourceBase] (http-8080-6:) Storage Network 2 is >> cloud-private for host 10.0.0.16 >> >> On my XenServer 6.0.2 machine I have my "cloud-private" and >> "cloud-public" networks. In CS, I have my Storage network set to "Use >> default gateway". The XenServer has access to the Internet and can >> resolve the FQDN of all hosts, management servers and the SAN on the >> storage network. I can ping the IP's of the SAN on both Storage >> subnets, yet CloudStack reports after adding the host that it can't >> see the storage network; obviously because it is incorrectly >> discovering my networks. >> >> I have verified in the config in my database for my networks: >> >> mysql> select * from physical_network_traffic_types; >> +----+--------------------------------------+---------------------+--------------+-------------------+-------------------+----------------------+-------------------------+-------------------+------+ >> >> | id | uuid | physical_network_id | >> traffic_type | xen_network_label | kvm_network_label | >> vmware_network_label | simulator_network_label | ovm_network_label | >> vlan | >> +----+--------------------------------------+---------------------+--------------+-------------------+-------------------+----------------------+-------------------------+-------------------+------+ >> >> | 1 | 3a368a1d-5d33-4e11-80ea-02da222d3930 | 1 | >> Management | cloud-private | NULL | >> NULL | NULL | NULL | >> NULL | >> | 2 | 3bf7b069-355e-4171-8ac5-c6f84777fc9d | 2 | >> Guest | cloud-public | NULL | >> NULL | NULL | NULL | >> NULL | >> | 3 | bb19c965-d726-451e-98fe-e0363c3bbd19 | 2 | >> Public | cloud-public | NULL | >> NULL | NULL | NULL | >> NULL | >> | 4 | 9df97cc6-f4e6-4d5f-b148-99f0dfc64dac | 1 | >> Storage | NULL | NULL | >> NULL | NULL | NULL | >> NULL | >> +----+--------------------------------------+---------------------+--------------+-------------------+-------------------+----------------------+-------------------------+-------------------+------+ >> >> 4 rows in set (0.00 sec) >> >> Thoughts? Any assistance would be greatly appreciated! >> >> Cheers, >> >> Matthew >> >> >> ShapeBlue provides a range of strategic and technical consulting >> services to help IT Service Providers and enterprises to build a true >> IaaS compute cloud. ShapeBlue�s expertise, combined with CloudStack >> technology, allows IT Service providers and enterprises to deliver >> true, utility based, IaaS to the customer without reengineering >> existing physical, virtual or storage layers. >> >> ________________________________ >> >> This email and any attachments to it may be confidential and are >> intended solely for the use of the individual to whom it is >> addressed. Any views or opinions expressed are solely those of the >> author and do not necessarily represent those of Shape Blue Ltd. If >> you are not the intended recipient of this email, you must neither >> take any action based upon its contents, nor copy or show it to >> anyone. Please contact the sender if you believe you have received >> this email in error. Shape Blue Ltd is a company incorporated in >> England& Wales. >> ShapeBlue provides a range of strategic and technical consulting >> services to help IT Service Providers and enterprises to build a true >> IaaS compute cloud. ShapeBlue�s expertise, combined with CloudStack >> technology, allows IT Service providers and enterprises to deliver >> true, utility based, IaaS to the customer without reengineering >> existing physical, virtual or storage layers. >> >> ________________________________ >> >> This email and any attachments to it may be confidential and are >> intended solely for the use of the individual to whom it is >> addressed. Any views or opinions expressed are solely those of the >> author and do not necessarily represent those of Shape Blue Ltd. If >> you are not the intended recipient of this email, you must neither >> take any action based upon its contents, nor copy or show it to >> anyone. Please contact the sender if you believe you have received >> this email in error. Shape Blue Ltd is a company incorporated in >> England& Wales. >> >> ShapeBlue provides a range of strategic and technical consulting >> services to help IT Service Providers and enterprises to build a true >> IaaS compute cloud. ShapeBlue�s expertise, combined with CloudStack >> technology, allows IT Service providers and enterprises to deliver >> true, utility based, IaaS to the customer without reengineering >> existing physical, virtual or storage layers. >> >> ________________________________ >> >> This email and any attachments to it may be confidential and are >> intended solely for the use of the individual to whom it is >> addressed. Any views or opinions expressed are solely those of the >> author and do not necessarily represent those of Shape Blue Ltd. If >> you are not the intended recipient of this email, you must neither >> take any action based upon its contents, nor copy or show it to >> anyone. Please contact the sender if you believe you have received >> this email in error. Shape Blue Ltd is a company incorporated in >> England& Wales. >> >> >> >> >> >> ShapeBlue provides a range of strategic and technical consulting >> services to help IT Service Providers and enterprises to build a true >> IaaS compute cloud. ShapeBlue�s expertise, combined with CloudStack >> technology, allows IT Service providers and enterprises to deliver >> true, utility based, IaaS to the customer without reengineering >> existing physical, virtual or storage layers. >> >> ________________________________ >> >> This email and any attachments to it may be confidential and are >> intended solely for the use of the individual to whom it is >> addressed. Any views or opinions expressed are solely those of the >> author and do not necessarily represent those of Shape Blue Ltd. If >> you are not the intended recipient of this email, you must neither >> take any action based upon its contents, nor copy or show it to >> anyone. Please contact the sender if you believe you have received >> this email in error. Shape Blue Ltd is a company incorporated in >> England& Wales. >> > > --------------030501000602080609080608--