cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nik Martin <nik.mar...@nfinausa.com>
Subject Re: Error Creating VMs and viewing hosts
Date Mon, 15 Oct 2012 17:17:08 GMT
On 10/15/2012 12:00 PM, Alena Prokharchyk wrote:
> Nic, looks like when storage pool was removed, cloudStack kept references
> host-> removed storage pool in the DB - its a bug.
>
> To workaround on your current setup:
>
> * save the db dump of cloud db just in case
> * delete all the stale storageHost refs to remove:
>
> "delete from storage_pool_host_ref where pool_id not in (select id from
> storage_pool)"
>
> Now listHosts should execute w/o any problems. Let me know if it works.
>

It worked, but there are some fk constraints, so setting deleted=now() 
on offline primary storage in storage_pool isn't enough.  I deleted the 
offline primary storage from storage_pool then deleted associated rows 
from storage_pool_host_ref, then restarted cloud-management, and hosts 
are now showing up;

> -Alena.
>
>
> On 10/15/12 9:45 AM, "Nik Martin" <nik.martin@nfinausa.com> wrote:
>
>> On 10/15/2012 11:40 AM, Alena Prokharchyk wrote:
>>> Nik, the error you see, doesn't state that the VM migration failed. NPE
>>> happens on listHosts command, when we are building the response and
>>> trying to figure out if local storage is enabled on the host.
>>>
>>> What version of cloudStack you are running on? Did you remove any of
>>> the primary storages in your setup?
>>>
>>> -Alena.
>>>
>>
>> Alena,
>>
>> This is a 3.02 CS with Xenserver hosts.  I have removed and re-added
>> primary storage, to work around bug 251, which takes an entire cloud
>> offline if any primary storage is put into maintenance mode and
>> shutdown.  I guess the fallout from that is more serious than earlier
>> expected. I'll need further direction on how to recover from bug 251,
>> which knocks me entirely offline.
>>
>> Nik
>>
>>
>>
>>> From: Nik Martin
>>> <nik.martin@nfinausa.com<mailto:nik.martin@nfinausa.com>>
>>> Reply-To: "'cloudstack-users@incubator. org'"
>>> <cloudstack-users@incubator.apache.org<mailto:cloudstack-users@incubator.
>>> apache.org>>
>>> To: "'cloudstack-users@incubator. org'"
>>> <cloudstack-users@incubator.apache.org<mailto:cloudstack-users@incubator.
>>> apache.org>>
>>> Subject: Error Creating VMs and viewing hosts
>>>
>>> I migrated some VMs around to different storage tags (I have two primary
>>> SANs, HD and SSD), and at some point, a migration failed, with this
>>> error:
>>>
>>>
>>> 2012-10-15 09:34:32,399 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (StatsCollector-1:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:32,408 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-188:null) Seq 4-857023789: Executing request
>>> 2012-10-15 09:34:32,626 ERROR [cloud.api.ApiDispatcher]
>>> (catalina-exec-2:null) Exception while executing ListHostsCmd:
>>> java.lang.NullPointerException
>>> at
>>>
>>> com.cloud.storage.StorageManagerImpl.isLocalStorageActiveOnHost(StorageMa
>>> nagerImpl.java:409)
>>> at
>>> com.cloud.api.ApiDBUtils.isLocalStorageActiveOnHost(ApiDBUtils.java:340)
>>> at
>>>
>>> com.cloud.api.ApiResponseHelper.createHostResponse(ApiResponseHelper.java
>>> :640)
>>> at com.cloud.api.commands.ListHostsCmd.execute(ListHostsCmd.java:167)
>>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
>>> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:509)
>>> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:416)
>>> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:300)
>>> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:59)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>> at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicat
>>> ionFilterChain.java:290)
>>> at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilte
>>> rChain.java:206)
>>> at
>>>
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve
>>> .java:233)
>>> at
>>>
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve
>>> .java:191)
>>> at
>>>
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
>>> 127)
>>> at
>>>
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
>>> 102)
>>> at
>>>
>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>>> at
>>>
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.j
>>> ava:109)
>>> at
>>>
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:29
>>> 8)
>>> at
>>>
>>> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.ja
>>> va:889)
>>> at
>>>
>>> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.proces
>>> s(Http11NioProtocol.java:721)
>>> at
>>>
>>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.ja
>>> va:2268)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java
>>> :1110)
>>> at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>> a:603)
>>> at java.lang.Thread.run(Thread.java:679)
>>> 2012-10-15 09:34:33,597 DEBUG [agent.manager.AgentManagerImpl]
>>> (AgentManager-Handler-14:null) Ping from 14
>>> 2012-10-15 09:34:33,658 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-188:null) Seq 4-857023789: Response Received:
>>> 2012-10-15 09:34:33,658 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (DirectAgent-188:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:33,658 DEBUG [agent.transport.Request]
>>> (StatsCollector-1:null) Seq 4-857023789: Received:  { Ans: , MgmtId:
>>> 130577622632, via: 4, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>>> 2012-10-15 09:34:33,658 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (StatsCollector-1:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:33,665 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-70:null) Seq 4-857023790: Executing request
>>> 2012-10-15 09:34:34,455 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-70:null) Seq 4-857023790: Response Received:
>>> 2012-10-15 09:34:34,456 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (DirectAgent-70:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:34,456 DEBUG [agent.transport.Request]
>>> (StatsCollector-1:null) Seq 4-857023790: Received:  { Ans: , MgmtId:
>>> 130577622632, via: 4, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>>> 2012-10-15 09:34:34,456 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (StatsCollector-1:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:34,461 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-459:null) Seq 4-857023791: Executing request
>>> 2012-10-15 09:34:35,387 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-459:null) Seq 4-857023791: Response Received:
>>> 2012-10-15 09:34:35,387 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (DirectAgent-459:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:35,388 DEBUG [agent.transport.Request]
>>> (StatsCollector-1:null) Seq 4-857023791: Received:  { Ans: , MgmtId:
>>> 130577622632, via: 4, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
>>> 2012-10-15 09:34:35,388 DEBUG [cloud.vm.VirtualMachineManagerImpl]
>>> (StatsCollector-1:null) Cleanup succeeded. Details null
>>> 2012-10-15 09:34:35,547 DEBUG [cloud.server.StatsCollector]
>>> (StatsCollector-1:null) HostStatsCollector is running...
>>> 2012-10-15 09:34:35,556 DEBUG [agent.manager.DirectAgentAttache]
>>> (DirectAgent-137:null) Seq 4-857023792: Executing request
>>> 2
>>>
>>> Now, I can't view my hosts under Infrastructure, and can't create new
>>> VMs.  I don't have any local storage in my cloud, so I don't understand
>>> the reference to local storage in the log. When I go to view hosts I get
>>> an empty popup box.  All my storage and hosts are online, so my guess
>>> this is a database error, but I don;t know where to look.
>>>
>>> --
>>> Regards,
>>>
>>> Nik
>>>
>>> Nik Martin
>>> VP Business Development
>>> Nfina Technologies, Inc.
>>> +1.251.243.0043 x1003
>>> Relentless Reliability
>>>
>>>
>>
>>
>> --
>> Regards,
>>
>> Nik
>>
>> Nik Martin
>> VP Business Development
>> Nfina Technologies, Inc.
>> +1.251.243.0043 x1003
>> Relentless Reliability
>>
>
>


-- 
Regards,

Nik

Nik Martin
VP Business Development
Nfina Technologies, Inc.
+1.251.243.0043 x1003
Relentless Reliability

Mime
View raw message