cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhinandan Prateek <abhinandan.prat...@shapeblue.com>
Subject Re: System VMs keeps failing to start
Date Mon, 09 May 2016 11:15:45 GMT
System VM connect back to management server on port 8250. Some of the other ports to be aware
of are here : https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ports+used+by+CloudStack




On 04/03/16, 4:28 PM, "cloudstackhelp@outlook.com" <cloudstackhelp@outlook.com> wrote:

>
>
>Hi,
>
>
>Before I dig through that I was just wondering if maybe it's really just a networking
issue. Here's how my network is setup right now:
>
>
>a. Management server cluster running galera sitting behind a gateway that NATs port 8080
to a HAproxy load balancer. All public traffic that is a result from MS requests get allowed
through. Everything else is rejected.
>
>
>b. Xenserver HVs with 4 NICs connected to 4 different switches:
>
>1. L3 switch with connection to internet (public subnet)
>
>2. L2 switch for management network which is connected to the MS cluster and the secondary
NFS share (192.168.2.0/24)
>
>3. L2 switch for storage network where the EQL SAN sits on (192.168.10.0/24)
>
>4. L2 switch for the guest network (10.10.1.0/16)
>
>
>I'm not too sure if I'm doing stuff wrongly.
>
>
>Josh
>
>
>
>
>
>
>On Thu, Mar 3, 2016 at 2:07 PM -0800, "Ahmad Emneina" <aemneina@gmail.com> wrote:
>
>
>
>
>
>If you see any exceptions, I'd ideally like to see a few hundred lines
>above and below. It might be easiest to stop the management service, rename
>the log file. Restart the service, observe the system vm's go through their
>life cycle... stop the management server and post that. Whatever works best
>for you.
>
>On Thu, Mar 3, 2016 at 1:43 PM, <cloudstackhelp@outlook.com> wrote:
>
>>
>>
>> It's really huge. Which part am I looking for exactly?
>>
>>
>>
>>
>>
>>
>> On Thu, Mar 3, 2016 at 1:38 PM -0800, "Ahmad Emneina" <aemneina@gmail.com>
>> wrote:
>>
>>
>>
>>
>>
>> Josh, can you share the logs off the management server. Namely:
>> /var/log/cloudstack/management/management-server.log
>> Post as much as you can to pastebin or similar. That'll help identify what
>> part of the process is failing...
>>
>> Ahmad E
>>
>> > On Mar 3, 2016, at 12:44 PM, <cloudstackhelp@outlook.com> <
>> cloudstackhelp@outlook.com> wrote:
>> >
>> >
>> >
>> > Nothing's being spun up on the HVs. I'm using separate networks for each
>> component (public, management, guest, storage). They all have a dedicated
>> NIC each. On the HVs it seems like CS created its own cloud link local
>> network but the link status is <none>
>> >
>> >
>> > Thanks
>> >
>> > Josh
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Mar 3, 2016 at 12:39 PM -0800, "Ahmad Emneina" <
>> aemneina@gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> > Hey Josh,
>> >
>> > Are you seeing the VM's being spun up on the backend (hypervisor)? If so,
>> > I'd imagine its a communication issue between the management server and
>> the
>> > public interface on the system vm. If you use VLAN tagging for your
>> public
>> > network, make sure the VLAN is trunked to your hypervisors in the cloud.
>> I
>> > recommend you stop the management service. Once restarted CloudStack will
>> > try to recycle those vm's and spin them up again (so no worries should be
>> > had there). If you're able to time it correctly, you can stop the
>> > management service before the system vm's get shut down and log into
>> > them... make sure the respective interfaces can reach their next hops...
>> > that would be a good first step.
>> >
>> >> On Thu, Mar 3, 2016 at 12:32 PM, <cloudstackhelp@outlook.com> wrote:
>> >>
>> >>
>> >>
>> >> Hi Ron and all,
>> >>
>> >>
>> >>
>> >> You mentioned that SVMs need to be started. I'm finding that my
>> >> installation keeps looping the command "Creating system VMs (this may
>> take
>> >> a while)". Right now, it's done its 70th cycle (s-70-VM) and seems to
>> want
>> >> to keep going on until it crashes.
>> >>
>> >>
>> >>
>> >> I'm thinking I should just kill the process but I'm worried I can't
>> start
>> >> the process again later. Is there a way to re-run this again later on?
>> >>
>> >>
>> >>
>> >> Why is it failing to start the VMs? Why is everything null? My networks
>> >> are starting fine. Apologies for the lack of formating. Sending this via
>> >> phone.
>> >>
>> >>
>> >> Console proxy up in zone: Public Cloud, proxy: v-72-VM, public IP: null,
>> >> private IP: N/A1004 Mar 2016 04:27:04
>> >>
>> >> Console proxy creation failure. zone: Public Cloud, error details:
>> >> null1004 Mar 2016 04:27:04
>> >>
>> >> Secondary Storage Vm creation failure. zone: Public Cloud, error
>> details:
>> >> null1904 Mar 2016 04:27:00
>> >>
>> >> Console proxy up in zone: Public Cloud, proxy: v-72-VM, public IP: null,
>> >> private IP: N/A1004 Mar 2016 04:26:34
>> >>
>> >> Console proxy creation failure. zone: Public Cloud, error details:
>> >> null1004 Mar 2016 04:26:34
>> >>
>> >>
>> >> Thanks in advance!
>> >>
>> >> Josh
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Mar 3, 2016 at 11:02 AM -0800, "Ron Wheeler" <
>> >> rwheeler@artifact-software.com> wrote:
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> I have been using Linux and the Internet since the mid 1990s.
>> >> There are still 3 consecutively numbered C class registered to me for
>> >> different clients back in the days when  c-class networks were given out
>> >> from massive ranges of free numbers.
>> >> I have set up small ISP operations for clients with multiple domains
>> >> including web sites, e-mail servers, fileservers, etc.
>> >> I have done this on SCO , Mandrake, CentOS 4 to 7.
>> >>
>> >> I should not have to struggle to figure out how set up Cloudstack in a
>> >> small configuration with a few servers and a single public IP.
>> >>
>> >> The documentation on networking is jumbled about and so unclear that I
>> >> can only point out why it is not clear but can not figure out the truth
>> >> sufficiently well to actually fix it.
>> >> I still don't know where the sources for the drawings are kept even
>> >> though I have asked several times.
>> >>
>> >> It needs a team approach with someone who knows the truth and someone
>> >> who can write it down so that someone who did not write the code can
>> >> figure out what to do.
>> >>
>> >> The biggest problem with programmers writing the user documentation is
>> >> that they are so caught up in the exceptions and special cases.
>> >> They spent a lot of time figuring out how to handle these oddball cases
>> >> that they feel that these triumphs must be on the front page.
>> >> They forget to explain the 95% case and lace the description of the main
>> >> flow with notes about these interesting exceptions.
>> >>
>> >> That is not just true for Cloudstack but is a general problem with
>> >> documentation just because we are all human.
>> >>
>> >> They also forget that the user does not want to be an expert in the
>> >> topic but wants to know enough to get the thing running.
>> >> The user has a lot of other problems and does not to become a developer
>> >> in order to get this to work.
>> >>
>> >> In my case, I really need to get some internal applications (accounting,
>> >> SCM, issue tracking, Maven repo, 20 web sites etc.) running on virtual
>> >> machines in an environment that is easy to manage.  I want to support
>> >> clients who I am supporting as users of other systems - just want simple
>> >> low volume services to support my supporting of their users.
>> >>
>> >> I only expect to have 4 servers, one NIC per machine to support 1
>> >> transaction per second on a busy day
>> >> I may get down to 2 servers  if Cloudstack works well and allows me to
>> >> manage test servers and run docker nicely.
>> >>
>> >> I do not want to know enough to be the network administrator at Google
>> >> or Amazon.
>> >>
>> >> This should not be hard to implement and from what I have seen it is not
>> >> but the networking docs are a major barrier to acceptance by mid-market
>> >> companies - 300-1000 users with 1 or 2 System Admins who have to support
>> >> all of the operations requirements and help developers and application
>> >> support teams test and keep production systems running.
>> >>
>> >> Ron
>> >>
>> >>
>> >>> On 03/03/2016 6:22 AM, Mario Giammarco wrote:
>> >>> Simon Weller <sweller@...> writes:
>> >>>
>> >>>> I do agree that the docs are confusing, especially if you have a
>> limited
>> >>> knowledge of networking concepts.
>> >>>> In terms of the complexity, a lot of that has to do with the fact
that
>> >>> every company has different service
>> >>>> requirements and ACS needs to be flexible enough to accommodate
very
>> >>> different underlying needs.
>> >>> Not agree. Even with good knowledge documentation is confusing because:
>> >>>
>> >>> - it assumes  you are always in the use case of "I have plenty of
>> >> routable ips"
>> >>> - it forgets to say that two system vms are create to manage routing
>> and
>> >>> secondary storage
>> >>> - it does not say that cloudstack manager can rewrite your host
>> >> configuration
>> >>>
>> >>>
>> >>>
>> >>>> It's always best to start with a basic zone, unless you REALLY need
>> some
>> >>> functionality within an advanced
>> >>>> zone. As soon as you move into advanced zone networking, you need
to
>> >> have
>> >>> a good understanding of layer 2/3
>> >>>> networking.
>> >>> I was able to make my cloudstack network working only when I skipped
>> >> basic
>> >>> zone and used advanced zone
>> >>
>> >>
>> >> --
>> >> Ron Wheeler
>> >> President
>> >> Artifact Software Inc
>> >> email: rwheeler@artifact-software.com
>> >> skype: ronaldmwheeler
>> >> phone: 866-970-2435, ext 102
>> >>
>> >>
>>

Regards,

AbhinandanĀ Prateek

abhinandan.prateek@shapeblue.comĀ 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
Mime
View raw message