cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjeev Neelarapu <sanjeev.neelar...@accelerite.com>
Subject RE: System VMs keeps failing to start
Date Tue, 10 May 2016 06:43:33 GMT
Hi Josh,

Since you are using clustered management server with HA, make sure that you have set the global
setting parameter "host" to the load balance/virtual IP address. If not, please set , restart
management server and destroy system vms.

Best Regards,
Sanjeev N
Chief Product Engineer, Accelerite
Off: +91 40 6722 9368 | EMail: sanjeev.neelarapu@accelerite.com 



-----Original Message-----
From: Abhinandan Prateek [mailto:abhinandan.prateek@shapeblue.com] 
Sent: Monday, May 09, 2016 4:46 PM
To: users@cloudstack.apache.org
Subject: Re: System VMs keeps failing to start

System VM connect back to management server on port 8250. Some of the other ports to be aware
of are here : https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ports+used+by+CloudStack




On 04/03/16, 4:28 PM, "cloudstackhelp@outlook.com" <cloudstackhelp@outlook.com> wrote:

>
>
>Hi,
>
>
>Before I dig through that I was just wondering if maybe it's really just a networking
issue. Here's how my network is setup right now:
>
>
>a. Management server cluster running galera sitting behind a gateway that NATs port 8080
to a HAproxy load balancer. All public traffic that is a result from MS requests get allowed
through. Everything else is rejected.
>
>
>b. Xenserver HVs with 4 NICs connected to 4 different switches:
>
>1. L3 switch with connection to internet (public subnet)
>
>2. L2 switch for management network which is connected to the MS 
>cluster and the secondary NFS share (192.168.2.0/24)
>
>3. L2 switch for storage network where the EQL SAN sits on 
>(192.168.10.0/24)
>
>4. L2 switch for the guest network (10.10.1.0/16)
>
>
>I'm not too sure if I'm doing stuff wrongly.
>
>
>Josh
>
>
>
>
>
>
>On Thu, Mar 3, 2016 at 2:07 PM -0800, "Ahmad Emneina" <aemneina@gmail.com> wrote:
>
>
>
>
>
>If you see any exceptions, I'd ideally like to see a few hundred lines 
>above and below. It might be easiest to stop the management service, 
>rename the log file. Restart the service, observe the system vm's go 
>through their life cycle... stop the management server and post that. 
>Whatever works best for you.
>
>On Thu, Mar 3, 2016 at 1:43 PM, <cloudstackhelp@outlook.com> wrote:
>
>>
>>
>> It's really huge. Which part am I looking for exactly?
>>
>>
>>
>>
>>
>>
>> On Thu, Mar 3, 2016 at 1:38 PM -0800, "Ahmad Emneina" 
>> <aemneina@gmail.com>
>> wrote:
>>
>>
>>
>>
>>
>> Josh, can you share the logs off the management server. Namely:
>> /var/log/cloudstack/management/management-server.log
>> Post as much as you can to pastebin or similar. That'll help identify 
>> what part of the process is failing...
>>
>> Ahmad E
>>
>> > On Mar 3, 2016, at 12:44 PM, <cloudstackhelp@outlook.com> <
>> cloudstackhelp@outlook.com> wrote:
>> >
>> >
>> >
>> > Nothing's being spun up on the HVs. I'm using separate networks for 
>> > each
>> component (public, management, guest, storage). They all have a 
>> dedicated NIC each. On the HVs it seems like CS created its own cloud 
>> link local network but the link status is <none>
>> >
>> >
>> > Thanks
>> >
>> > Josh
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Mar 3, 2016 at 12:39 PM -0800, "Ahmad Emneina" <
>> aemneina@gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> > Hey Josh,
>> >
>> > Are you seeing the VM's being spun up on the backend (hypervisor)? 
>> > If so, I'd imagine its a communication issue between the management 
>> > server and
>> the
>> > public interface on the system vm. If you use VLAN tagging for your
>> public
>> > network, make sure the VLAN is trunked to your hypervisors in the cloud.
>> I
>> > recommend you stop the management service. Once restarted 
>> > CloudStack will try to recycle those vm's and spin them up again 
>> > (so no worries should be had there). If you're able to time it 
>> > correctly, you can stop the management service before the system 
>> > vm's get shut down and log into them... make sure the respective interfaces
can reach their next hops...
>> > that would be a good first step.
>> >
>> >> On Thu, Mar 3, 2016 at 12:32 PM, <cloudstackhelp@outlook.com> wrote:
>> >>
>> >>
>> >>
>> >> Hi Ron and all,
>> >>
>> >>
>> >>
>> >> You mentioned that SVMs need to be started. I'm finding that my 
>> >> installation keeps looping the command "Creating system VMs (this 
>> >> may
>> take
>> >> a while)". Right now, it's done its 70th cycle (s-70-VM) and seems 
>> >> to
>> want
>> >> to keep going on until it crashes.
>> >>
>> >>
>> >>
>> >> I'm thinking I should just kill the process but I'm worried I 
>> >> can't
>> start
>> >> the process again later. Is there a way to re-run this again later on?
>> >>
>> >>
>> >>
>> >> Why is it failing to start the VMs? Why is everything null? My 
>> >> networks are starting fine. Apologies for the lack of formating. 
>> >> Sending this via phone.
>> >>
>> >>
>> >> Console proxy up in zone: Public Cloud, proxy: v-72-VM, public IP: 
>> >> null, private IP: N/A1004 Mar 2016 04:27:04
>> >>
>> >> Console proxy creation failure. zone: Public Cloud, error details:
>> >> null1004 Mar 2016 04:27:04
>> >>
>> >> Secondary Storage Vm creation failure. zone: Public Cloud, error
>> details:
>> >> null1904 Mar 2016 04:27:00
>> >>
>> >> Console proxy up in zone: Public Cloud, proxy: v-72-VM, public IP: 
>> >> null, private IP: N/A1004 Mar 2016 04:26:34
>> >>
>> >> Console proxy creation failure. zone: Public Cloud, error details:
>> >> null1004 Mar 2016 04:26:34
>> >>
>> >>
>> >> Thanks in advance!
>> >>
>> >> Josh
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Mar 3, 2016 at 11:02 AM -0800, "Ron Wheeler" < 
>> >> rwheeler@artifact-software.com> wrote:
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> I have been using Linux and the Internet since the mid 1990s.
>> >> There are still 3 consecutively numbered C class registered to me 
>> >> for different clients back in the days when  c-class networks were 
>> >> given out from massive ranges of free numbers.
>> >> I have set up small ISP operations for clients with multiple 
>> >> domains including web sites, e-mail servers, fileservers, etc.
>> >> I have done this on SCO , Mandrake, CentOS 4 to 7.
>> >>
>> >> I should not have to struggle to figure out how set up Cloudstack 
>> >> in a small configuration with a few servers and a single public IP.
>> >>
>> >> The documentation on networking is jumbled about and so unclear 
>> >> that I can only point out why it is not clear but can not figure 
>> >> out the truth sufficiently well to actually fix it.
>> >> I still don't know where the sources for the drawings are kept 
>> >> even though I have asked several times.
>> >>
>> >> It needs a team approach with someone who knows the truth and 
>> >> someone who can write it down so that someone who did not write 
>> >> the code can figure out what to do.
>> >>
>> >> The biggest problem with programmers writing the user 
>> >> documentation is that they are so caught up in the exceptions and special
cases.
>> >> They spent a lot of time figuring out how to handle these oddball 
>> >> cases that they feel that these triumphs must be on the front page.
>> >> They forget to explain the 95% case and lace the description of 
>> >> the main flow with notes about these interesting exceptions.
>> >>
>> >> That is not just true for Cloudstack but is a general problem with 
>> >> documentation just because we are all human.
>> >>
>> >> They also forget that the user does not want to be an expert in 
>> >> the topic but wants to know enough to get the thing running.
>> >> The user has a lot of other problems and does not to become a 
>> >> developer in order to get this to work.
>> >>
>> >> In my case, I really need to get some internal applications 
>> >> (accounting, SCM, issue tracking, Maven repo, 20 web sites etc.) 
>> >> running on virtual machines in an environment that is easy to 
>> >> manage.  I want to support clients who I am supporting as users of 
>> >> other systems - just want simple low volume services to support my supporting
of their users.
>> >>
>> >> I only expect to have 4 servers, one NIC per machine to support 1 
>> >> transaction per second on a busy day I may get down to 2 servers  
>> >> if Cloudstack works well and allows me to manage test servers and 
>> >> run docker nicely.
>> >>
>> >> I do not want to know enough to be the network administrator at 
>> >> Google or Amazon.
>> >>
>> >> This should not be hard to implement and from what I have seen it 
>> >> is not but the networking docs are a major barrier to acceptance 
>> >> by mid-market companies - 300-1000 users with 1 or 2 System Admins 
>> >> who have to support all of the operations requirements and help 
>> >> developers and application support teams test and keep production systems
running.
>> >>
>> >> Ron
>> >>
>> >>
>> >>> On 03/03/2016 6:22 AM, Mario Giammarco wrote:
>> >>> Simon Weller <sweller@...> writes:
>> >>>
>> >>>> I do agree that the docs are confusing, especially if you have a
>> limited
>> >>> knowledge of networking concepts.
>> >>>> In terms of the complexity, a lot of that has to do with the 
>> >>>> fact that
>> >>> every company has different service
>> >>>> requirements and ACS needs to be flexible enough to accommodate

>> >>>> very
>> >>> different underlying needs.
>> >>> Not agree. Even with good knowledge documentation is confusing because:
>> >>>
>> >>> - it assumes  you are always in the use case of "I have plenty of
>> >> routable ips"
>> >>> - it forgets to say that two system vms are create to manage 
>> >>> routing
>> and
>> >>> secondary storage
>> >>> - it does not say that cloudstack manager can rewrite your host
>> >> configuration
>> >>>
>> >>>
>> >>>
>> >>>> It's always best to start with a basic zone, unless you REALLY 
>> >>>> need
>> some
>> >>> functionality within an advanced
>> >>>> zone. As soon as you move into advanced zone networking, you 
>> >>>> need to
>> >> have
>> >>> a good understanding of layer 2/3
>> >>>> networking.
>> >>> I was able to make my cloudstack network working only when I 
>> >>> skipped
>> >> basic
>> >>> zone and used advanced zone
>> >>
>> >>
>> >> --
>> >> Ron Wheeler
>> >> President
>> >> Artifact Software Inc
>> >> email: rwheeler@artifact-software.com
>> >> skype: ronaldmwheeler
>> >> phone: 866-970-2435, ext 102
>> >>
>> >>
>>

Regards,

AbhinandanĀ Prateek

abhinandan.prateek@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue



DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite,
a Persistent Systems business. It is intended only for the use of the individual or entity
to which it is addressed. If you are not the intended recipient, you are not authorized to
read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent
Systems business does not accept any liability for virus infected mails.
Mime
View raw message