cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayapal Reddy Uradi <jayapalreddy.ur...@citrix.com>
Subject Re: InsufficientServerCapacityException - Unsure Why
Date Fri, 11 Jul 2014 09:19:49 GMT
Hi Elliot,

Try recreating router (destroy the router and deploy new vm, router get recreated).
After recreation if the problem still exists, check the router logs to see why the interfaces
are brought up.


Thanks,
jayapal

On 11-Jul-2014, at 1:38 PM, Elliot Berg <elliot.berg@avcosystems.com> wrote:

> So, I'm wondering whether the guest not having the interfaces configured correctly (i.e.
not having an IP) is just a symptom of more generally broken networking - my interfaces file
for the KVM host is below, does anyone spot any issues?
> 
> auto lo
> iface lo inet loopback
> 
> auto eth0
> iface eth0 inet manual
> 
> auto cloudbr0
> iface cloudbr0 inet static
>        bridge_ports eth0
>        bridge_fd 5
>        bridge_stp off
>        bridge_maxwait 1
>        address 10.4.0.2
>        netmask 255.0.0.0
>        network 10.0.0.0
>        broadcast 10.255.255.255
>        gateway 10.0.0.1
>        # dns-* options are implemented by the resolvconf package, if installed
>        dns-nameservers 10.0.0.12
>        dns-search avco
> 
> auto cloudbr1
> iface cloudbr1 inet manual
>        bridge_ports eth0
>        bridge_fd 5
>        bridge_stp off
>        bridge_maxwait 1
> 
> Thanks,
> 
> Elliot
> 
> Elliot Berg wrote:
>> Doh! I did, but forgot about it being on a funny port. Now that I'm into the VM I
can see that it's not running, and fails to start when it tries to bind to the address that
it should have on the guest range. I notice that "ifconfig -a" shows two NICs, only one of
which is up (the one with the link local IP).  I'm guessing that indicates a more general
networking issue?
>> 
>> I think how it's laid out is 10.4.0.0-255 for physical machines (1 is the management
server, 2 is the first host), 10.4.1.0-255 is the management network and 10.4.2.0-255 is the
guest network...but it's possible I've misunderstood the networking config during setup? What
I really wanted was hosts on 10.4.0.0-255 and guests on 10.4.1.0-255 (and beyond), as in the
future I'd like it to co-exist with our existing infrastructure while we migrate things -
but I kept being told about conflicts etc when I tried to set up cloudstack like that during
the initial set up process?
>> 
>> Thanks,
>> 
>> Elliot
>> 
>> Jayapal Reddy Uradi wrote:
>>> Hi Elliot,
>>> 
>>> Did you ssh to VR using the ssh key ?
>>> Ex: ssh -i /root/.ssh/id_rsa.cloud -p 3922root@169.254.3.196
>>> 
>>> If it is failed to ssh, then there is issue with the ssh keys.
>>> 
>>> Thanks,
>>> Jayapal
>>> 
>>> 
>>> On 09-Jul-2014, at 4:43 PM, Harikrishna Patnala<harikrishna.patnala@citrix.com>
 wrote:
>>> 
>>>> 1) Log into your KVM host.
>>>> 2) Use command “virsh list”. This gives the list of VMs on the host.
>>>> 3) Use command “virsh console<VirtualRouterId>” to log into the
VR.
>>>> 
>>>> 
>>>> -Harikrishna
>>>> 
>>>> On 09-Jul-2014, at 3:52 pm, Elliot Berg<elliot.berg@avcosystems.com>
 wrote:
>>>> 
>>>>> I don't know - I can't seem to ssh to the link local IP. It pings, but
ssh times out. If I try and use the "connect to console" button in the gui, that too times
out :(
>>>>> 
>>>>> Elliot
>>>>> 
>>>>> Harikrishna Patnala wrote:
>>>>>>> From the logs
>>>>>> 2014-07-08 12:08:56,218 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null)
Seq 1-277348416: Processing:  { Ans: , MgmtId: 159320647860937, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"grep: /var/lib/misc/dnsmasq.leases:
No such file or directory","wait":0}}] }
>>>>>> 
>>>>>> Can you check whether dnsmasq service is running in the Virtual Router
? if not, start the service and check for “/var/lib/misc/dnsmasq.leases”
>>>>>> 
>>>>>> -Harikrishna
>>>>>> 
>>>>>> On 08-Jul-2014, at 3:47 pm, Elliot Berg<elliot.berg@avcosystems.com>
  wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I've done that, and now there's a new virtual router which says
it's running, however a deployment still fails. My latest lot of logs are available athttps://dl.dropboxusercontent.com/u/47728104/management-server.log.gz,
and there's now one thing in the op_it_work table with a step != 'Done', which is a ConsoleProxy.
>>>>>>> 
>>>>>>> Interestingly if I look at the console proxy vm in the cloudstack
management gui it says it's running, though.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> Elliot
>>>>>>> 
>>>>>>> Harikrishna Patnala wrote:
>>>>>>>> Yes mark the VR to stopped, destroy VR, mark the VR entry
in op_it_work to “Done” and try deploying VM.
>>>>>>>> 
>>>>>>>> -Harikrishna
>>>>>>>> 
>>>>>>>> On 08-Jul-2014, at 12:44 pm, Elliot Berg<elliot.berg@avcosystems.com>
   wrote:
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> It appears to be stuck in the "starting" state - so I
don't get the option to reboot it or anything. If I change the state to stopped in the database
directly will the management server attempt to start it again or do I need to do something
more?
>>>>>>>>> 
>>>>>>>>> Thanks!
>>>>>>>>> 
>>>>>>>>> Elliot
>>>>>>>>> 
>>>>>>>>> Harikrishna Patnala wrote:
>>>>>>>>>> Is your Virtual Router up and running ? If is in
running state you can mark it Done and deploy a VM.
>>>>>>>>>> If it is in stopped state try restarting it. You
can try updating the field as well.
>>>>>>>>>> 
>>>>>>>>>> -Harikrishna
>>>>>>>>>> 
>>>>>>>>>> On 07-Jul-2014, at 7:10 pm, Elliot Berg<elliot.berg@avcosystems.com>
    wrote:
>>>>>>>>>> 
>>>>>>>>>>> I can see two entries that have the "step" field
set to something other than "Done", one of them is
>>>>>>>>>>> 
>>>>>>>>>>> ConsoleProxy | Starting
>>>>>>>>>>> 
>>>>>>>>>>> and the other is
>>>>>>>>>>> 
>>>>>>>>>>> DomainRouter | Prepare
>>>>>>>>>>> 
>>>>>>>>>>> Am I safe to just delete the rows, or should
I just update the field?
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> 
>>>>>>>>>>> Elliot
>>>>>>>>>>> 
>>>>>>>>>>> Harikrishna Patnala wrote:
>>>>>>>>>>>> Do you see any work item pending for Virtual
Router r-4-VM in “op_it_work” table ?
>>>>>>>>>>>> If there are any, remove those entries and
try VM deployment again.
>>>>>>>>>>>> 
>>>>>>>>>>>> I see in the logs that VR has a task pending
>>>>>>>>>>>> 2014-07-07 10:28:15,934 WARN  [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-5:job-48 = [ 22369802-b5aa-4b5a-a26d-1fab11241551 ]) The task item for vm VM[DomainRouter|r-4-VM]
has been inactive for 418531
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> -Harikrishna
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On 07-Jul-2014, at 2:18 pm, Elliot Berg<elliot.berg@avcosystems.com<mailto:elliot.berg@avcosystems.com>>
     wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm still not really spotting anything indicating
why it's not using the host, but I suspect that's just because I don't really know what I'm
looking for - so I've zipped the whole log for today and stuffed it on dropbox athttps://dl.dropboxusercontent.com/u/47728104/management-server.log.gz.
>>>>>>>>>>>> 
>>>>>>>>>>>> Hopefully someone who's used cloudstack a
lot more will have more success!
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot Berg wrote:
>>>>>>>>>>>> I'm going back over everything and I've noticed
something else - everywhere I've looked for how to use local storage says I should change
two global settings;
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>  *   system.vm.use.local.storage = true
>>>>>>>>>>>>  *   use.local.storage = true
>>>>>>>>>>>> 
>>>>>>>>>>>> However I'm looking at my global settings
and only the first exists (which I have set to true).
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot Berg wrote:
>>>>>>>>>>>> Ah, so when looking back a bit further before
(I was kind of only looking for exceptions higher up before now), I've just spotted this...
>>>>>>>>>>>> 
>>>>>>>>>>>> 2014-07-03 10:48:28,765 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-3:job-46 = [ 92fb959d-edc5-4fe2-84a0-5
>>>>>>>>>>>> 6001226e4ac ] FirstFitRoutingAllocator) Looking
for speed=1000Mhz, Ram=1024
>>>>>>>>>>>> 2014-07-03 10:48:28,765 DEBUG [allocator.impl.FirstFitAllocator]
(Job-Executor-3:job-46 = [ 92fb959d-edc5-4fe2-84a0-5
>>>>>>>>>>>> 6001226e4ac ] FirstFitRoutingAllocator) Host
name: cloudstack-host1, hostId: 1 is in avoid set, skipping this and try
>>>>>>>>>>>> ing other available hosts
>>>>>>>>>>>> 
>>>>>>>>>>>> That's the one and only host - so I'm guessing
that has something to do with it!
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot
>>>>>>>>>>>> --
>>>>>>>>>>>> Elliot Berg  |  Analyst Programmer/Network
Team
>>>>>>>>>>>> Email:elliot.berg@avcosystems.com<mailto:elliot.berg@avcosystems.com>
     | Tel: 01753 213700 | Web:www.avcosystems.com<http://www.avcosystems.com/>
>>>>>>>>>>>> <image.png>
>>>>>>>>>>>> 
>>>>>>>>>>>> Avco Systems Ltd, Registered in England&
     Wales, Registration Number 1976620
>>>>>>>>>>>> Registered Office: Avco Systems | 17 Bath
Road | Slough | SL1 3UF
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> ilya musayev wrote:
>>>>>>>>>>>> Elliot,
>>>>>>>>>>>> 
>>>>>>>>>>>> When you see such an error - there usually
a predecessor message that says CloudStack checked for X, Y and Z and found no suitable resources
based on your configuration.
>>>>>>>>>>>> 
>>>>>>>>>>>> Put the logs on pastebin or some other site
(strip out any private info you dont want to share). I would also  recommend cloudstack 4.3.1
(which is not officially out yet) but should come thru in the next several weeks. Its latest
stable release of CloudStack 4.3.0 - with latest bug fixes.
>>>>>>>>>>>> 
>>>>>>>>>>>> I've put a build for folks who want to try
it out until we complete official release of ACS 4.3.1 process.
>>>>>>>>>>>> 
>>>>>>>>>>>> Unzip tgz and it should have required RPMs
with both Open Source and Non-Open Source modules.
>>>>>>>>>>>> 
>>>>>>>>>>>> http://www.cloudsand.com/cloudstack-4.3.0-1.tgz
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards
>>>>>>>>>>>> ilya
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On 7/2/14, 1:06 AM, Elliot Berg wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> 
>>>>>>>>>>>> I've been putting together a cloudstack set-up
for experimentation purposes - right now we're just trying to compare different platforms
for private cloud infrastructure before we start getting too in depth with any of them.
>>>>>>>>>>>> 
>>>>>>>>>>>> I've added the cloudstack 4.2 apt repository,
and I'm running on Ubuntu 12.04 LTS, and I believe I've followed all the installation guides
correctly at the various stages.
>>>>>>>>>>>> 
>>>>>>>>>>>> We've set up a management server, which is
also an NFS server, however we're interested in using local storage for the majority of things,
and have also set up a single KVM host which I believe is all configured correctly to use
local storage. If I look at the dashboard, I'm told I have more than enough resource in every
section to create an instance the size I want to - which is a small offering I've created
with just 1.0GHz and 1GB of RAM, with local storage. The host's not very powerful, but according
to the dashboard I am using 1.50GHz/5.87GHz, 1.38GB/7.80GB, 3.55GB/285.95GB Secondary Storage,
1.03GB/450.99GB Local Storage and 0.00KB/571.90GB Primary Storage (I'm assuming that's meant
to be a combination of the NFS server's primary storage offering and the local storage on
the host, though the numbers don't quite make sense at first glance).
>>>>>>>>>>>> 
>>>>>>>>>>>> However, when I try to add an instance, I
receive an InsufficientServerCapacityException and I'm struggling to work out why. I can't
add an instance using a small shared storage offering either, but if I'm not mistaken that's
expected because the zone and host are configured to use local storage. The only thing I can
think of is that the local storage isn't properly configured, but when I've looked it seems
to be.
>>>>>>>>>>>> 
>>>>>>>>>>>> Any pointers for how I can further diagnose
this would be great - thanks in advance!
>>>>>>>>>>>> 
>>>>>>>>>>>> Elliot
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>> 
>>> 
>>> 


Mime
View raw message