Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: local policy)
References: 
 <CAJvmrNNZb=6ZFforqkLtofFr5KuMwphrYyE=jF6agfPFF1c1GQ@mail.gmail.com>
 <CC8DB8DC.16D1A%Ahmad.Emneina@citrix.com>
 <CAJvmrNN5X_aVGg2Z3X1cijzkGxMzvOfyFWCRoC5iKP64yDZQeA@mail.gmail.com>
 <F769EE03-2A82-48BE-88AE-59C6BAEB3449@bbits.ca>
 <CAJvmrNOUevt8-FxAi69OjYrBLRhqcFwhxGPz5uiam5-1Vn5Q8w@mail.gmail.com>
 <285FDC49-1D87-41C9-B9D6-E960299D2D31@bbits.ca>
 <CAJvmrNNBcNy1doTV3-02=Neki4pXnobJkD5jiwjVZVGtPgr3cw@mail.gmail.com>
In-Reply-To: 
 <CAJvmrNNBcNy1doTV3-02=Neki4pXnobJkD5jiwjVZVGtPgr3cw@mail.gmail.com>
Mime-Version: 1.0 (1.0)
Content-Type: text/plain;
	charset=utf-8
Message-Id: <3144FEB7-CB17-4181-9C26-38B2E4993ED6@bbits.ca>
Content-Transfer-Encoding: quoted-printable
From: "Kelceydamage@bbits" <kelcey@bbits.ca>
Subject: Re: "Primary storage is not ready,
 wait until it is ready to launch secondary storage vm"
Date: Sun, 30 Sep 2012 12:20:02 -0700
To: "cloudstack-dev@incubator.apache.org"
 <cloudstack-dev@incubator.apache.org>

Ahh, many times errant operations can occur and not be shown in the GUI.

It is best to familiarize yourself with the database.

Go to=20

wiki.cloudstack.org/display/~kdamage/Home

And download the database-resource-guide (still in progress)

That guide should help you learn the database.

The GUI only takes you do far in troubleshooting.

Sent from my iPhone

On Sep 30, 2012, at 12:13 PM, James Martin <jmartin@basho.com> wrote:

> =46rom my limited understanding and from the GUI,  everything is ok from
> that side.  I don't know it well enough to dig deeper than that.
>=20
> James
>=20
> On Sun, Sep 30, 2012 at 2:58 PM, Kelceydamage@bbits <kelcey@bbits.ca> wrot=
e:
>> The database check the IP allocation and network pools alongside the stor=
age pool
>>=20
>> Sent from my iPhone
>>=20
>> On Sep 30, 2012, at 11:56 AM, James Martin <jmartin@basho.com> wrote:
>>=20
>>> Primary storage and Secondary storage are good according to the GUI.
>>> Is that what you are referring to?
>>>=20
>>> James
>>>=20
>>> On Sun, Sep 30, 2012 at 2:48 PM, Kelceydamage@bbits <kelcey@bbits.ca> wr=
ote:
>>>> Just winging it here but for scope interface errors check that your poo=
ls are not completely allocated.
>>>>=20
>>>> Sent from my iPhone
>>>>=20
>>>> On Sep 30, 2012, at 11:21 AM, James Martin <jmartin@basho.com> wrote:
>>>>=20
>>>>> I think I fixed one problem but solved another..  I had to add
>>>>>=20
>>>>>      {
>>>>>          "name":"system.vm.use.local.storage",
>>>>>          "value":"false"
>>>>>      },
>>>>>=20
>>>>> to the devcloud.cfg and no longer receive that error.
>>>>>=20
>>>>> My SSVMs and CPVMs still aren't starting.  I've attached a big log
>>>>> that should capture the context of the errors.
>>>>>=20
>>>>> Thanks!
>>>>>=20
>>>>> James
>>>>>=20
>>>>> On Sun, Sep 30, 2012 at 11:54 AM, Ahmad Emneina
>>>>> <Ahmad.Emneina@citrix.com> wrote:
>>>>>> Hey James, can you post more logs. I'm interested to see the logs abo=
ve
>>>>>> what was pasted in the previous email. I think the real reason hides
>>>>>> there...
>>>>>>=20
>>>>>> On 9/30/12 8:50 AM, "James Martin" <jmartin@basho.com> wrote:
>>>>>>=20
>>>>>>> This was because I did have the nfs service subscribing to
>>>>>>> /etc/exports in the puppet recipe.
>>>>>>>=20
>>>>>>> I'm back to the same error as before:
>>>>>>>=20
>>>>>>>=20
>>>>>>> 2012-09-30 10:16:49,695 DEBUG [cloud.capacity.CapacityManagerImpl]
>>>>>>> (consoleproxy-1:null) VM state transitted from :Starting to Stopped
>>>>>>> with event: OperationFailedvm's original host id: null new host id:
>>>>>>> null host id before state transition: null
>>>>>>> 2012-09-30 10:16:49,703 WARN
>>>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>>>> Exception while trying to start console proxy
>>>>>>> com.cloud.exception.InsufficientServerCapacityException: Unable to
>>>>>>> create a deployment for VM[ConsoleProxy|v-2-TEST]Scope=3Dinterface
>>>>>>> com.cloud.dc.DataCenter; id=3D1
>>>>>>> at
>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineMa=
nagerI
>>>>>>> mpl.java:734)
>>>>>>> at
>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerIm=
pl.jav
>>>>>>> a:472)
>>>>>>> at
>>>>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerIm=
pl.jav
>>>>>>> a:465)
>>>>>>> at
>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsolePro=
xyMana
>>>>>>> gerImpl.java:624)
>>>>>>> at
>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(Console=
ProxyM
>>>>>>> anagerImpl.java:1154)
>>>>>>> at
>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsolePro=
xyMana
>>>>>>> gerImpl.java:1961)
>>>>>>> at
>>>>>>> com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsolePro=
xyMana
>>>>>>> gerImpl.java:170)
>>>>>>> at
>>>>>>> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:1=
13)
>>>>>>> at
>>>>>>> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java=
:34)
>>>>>>> at
>>>>>>> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.jav=
a:83)
>>>>>>> at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:7=
3)
>>>>>>> at
>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:4=
71)
>>>>>>> at
>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.jav=
a:351)
>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask=
.acces
>>>>>>> s$201(ScheduledThreadPoolExecutor.java:165)
>>>>>>> at
>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask=
.run(S
>>>>>>> cheduledThreadPoolExecutor.java:267)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor=
.java:
>>>>>>> 1110)
>>>>>>> at
>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecuto=
r.java
>>>>>>> :603)
>>>>>>> at java.lang.Thread.run(Thread.java:679)
>>>>>>> 2012-09-30 10:16:49,730 DEBUG [cloud.storage.StorageManagerImpl]
>>>>>>> (secstorage-1:null) Expunging Vol[1|vm=3D1|ROOT]
>>>>>>> 2012-09-30 10:16:49,742 DEBUG [cloud.storage.StorageManagerImpl]
>>>>>>> (secstorage-1:null) Marking volume that was never created as
>>>>>>> destroyed: Vol[1|vm=3D1|ROOT]
>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>> On Sun, Sep 30, 2012 at 11:28 AM, James Martin <jmartin@basho.com> w=
rote:
>>>>>>>> My SSVM never starts (or the CPVM for the matter).   In the GUI I h=
ave
>>>>>>>>=20
>>>>>>>> Zones: 1
>>>>>>>> Pods: 1
>>>>>>>> Clusters: 1
>>>>>>>> Hosts : 1
>>>>>>>> Primary Storage : 0
>>>>>>>> Secondary Storage : 1
>>>>>>>> System VMs  : 0
>>>>>>>> Virtual Routers : 0
>>>>>>>>=20
>>>>>>>> Also,  These messages keep repeating in the logs:
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> 2012-09-30 09:51:01,012 DEBUG [storage.snapshot.SnapshotSchedulerIm=
pl]
>>>>>>>> (SnapshotPollTask:null) Snapshot scheduler.poll is being called at
>>>>>>>> 2012-09-30 09:51:01 GMT
>>>>>>>> 2012-09-30 09:51:01,013 DEBUG [storage.snapshot.SnapshotSchedulerIm=
pl]
>>>>>>>> (SnapshotPollTask:null) Got 0 snapshots to be executed at 2012-09-3=
0
>>>>>>>> 09:51:01 GMT
>>>>>>>> 2012-09-30 09:51:01,055 DEBUG
>>>>>>>> [cloud.network.ExternalLoadBalancerUsageManagerImpl]
>>>>>>>> (ExternalNetworkMonitor-1:null) External load balancer devices stat=
s
>>>>>>>> collector is running...
>>>>>>>> 2012-09-30 09:51:01,106 DEBUG
>>>>>>>> [network.router.VirtualNetworkApplianceManagerImpl]
>>>>>>>> (RouterMonitor-1:null) Found 0 running routers.
>>>>>>>> 2012-09-30 09:51:01,120 DEBUG
>>>>>>>> [network.router.VirtualNetworkApplianceManagerImpl]
>>>>>>>> (RouterStatusMonitor-1:null) Found 0 routers.
>>>>>>>> 2012-09-30 09:51:15,064 DEBUG [cloud.server.StatsCollector]
>>>>>>>> (StatsCollector-1:null) StorageCollector is running...
>>>>>>>> 2012-09-30 09:51:15,066 DEBUG [cloud.server.StatsCollector]
>>>>>>>> (StatsCollector-1:null) There is no secondary storage VM for second=
ary
>>>>>>>> storage host nfs://10.0.2.15:/opt/storage/secondary
>>>>>>>> 2012-09-30 09:51:15,072 DEBUG [cloud.server.StatsCollector]
>>>>>>>> (StatsCollector-2:null) VmStatsCollector is running...
>>>>>>>> 2012-09-30 09:51:15,943 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>> (DirectAgent-1:null) Ping from 1
>>>>>>>> 2012-09-30 09:51:16,706 DEBUG [cloud.server.StatsCollector]
>>>>>>>> (StatsCollector-1:null) HostStatsCollector is running...
>>>>>>>> 2012-09-30 09:51:16,714 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>> (DirectAgent-14:null) Seq 1-164560912: Executing request
>>>>>>>> 2012-09-30 09:51:16,874 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>> (DirectAgent-14:null) Seq 1-164560912: Response Received:
>>>>>>>> 2012-09-30 09:51:16,875 DEBUG [agent.transport.Request]
>>>>>>>> (StatsCollector-1:null) Seq 1-164560912: Received:  { Ans: , MgmtId=
:
>>>>>>>> 8796761203466, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }=

>>>>>>>> 2012-09-30 09:51:18,640 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Executing request
>>>>>>>> 2012-09-30 09:51:18,797 DEBUG [agent.manager.DirectAgentAttache]
>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Response Received:
>>>>>>>> 2012-09-30 09:51:18,798 DEBUG [agent.transport.Request]
>>>>>>>> (DirectAgent-31:null) Seq 1-164560898: Processing:  { Ans: , MgmtId=
:
>>>>>>>> 8796761203466, via: 1, Ver: v1, Flags: 10,
>>>>>>>>=20
>>>>>>>> [{"ClusterSyncAnswer":{"_clusterId":1,"_newStates":{},"_isExecuted"=
:false
>>>>>>>> ,"result":true,"wait":0}}]
>>>>>>>> }
>>>>>>>> 2012-09-30 09:51:30,279 DEBUG
>>>>>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)=

>>>>>>>> Primary storage is not ready, wait until it is ready to launch
>>>>>>>> secondary storage vm
>>>>>>>> 2012-09-30 09:51:30,279 DEBUG
>>>>>>>> [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null)=

>>>>>>>> Zone 1 is not ready to launch secondary storage VM yet
>>>>>>>> 2012-09-30 09:51:30,568 DEBUG
>>>>>>>> [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null)
>>>>>>>> Skip capacity scan due to there is no Primary Storage UPintenance m=
ode
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> Thanks,
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> James
>>>>>>>=20
>>>>>>=20
>>>>>>=20
>>>>>> --
>>>>>> =C3=86
>>>>>>=20
>>>>>>=20
>>>>>>=20
>>>>> <log.txt>