cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "manasaveloori (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-5792) All the VMs are shown as running even after the host is put in maintenance.NPE in logs
Date Mon, 06 Jan 2014 10:33:50 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

manasaveloori updated CLOUDSTACK-5792:
--------------------------------------

    Attachment: management-server.rar

> All the VMs are shown as running even after the host is put in maintenance.NPE  in logs
> ---------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-5792
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5792
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Hypervisor Controller
>    Affects Versions: 4.3.0
>         Environment: upgraded from 2.2.16 to 4.3
>            Reporter: manasaveloori
>            Priority: Critical
>             Fix For: 4.3.0
>
>         Attachments: management-server.rar
>
>
> Steps:
>  Deploy CS 2.2 X.16 using Xen5.6 sp2 HV.
> 2. Add the External firewall SRX to CS.
> 3. Set the GC parameter firewall.rule.ui.enabled to "true."
> 4. Now acquire the IP and configure firewall and PF rules.
> 5. Upgrade the CS to 4.3.
> 6. Stop and start all the System VMs and router VMs so that the new template is upgraded.
> 7. Now perform Network restart on which the firwall and PF rules are configured.
> Here there is a issue  CLOUDSTACK-5747 
> 8.Enable maintenance mode on host.---CLOUDSTACK-5706
> 9. Now unmanage the cluster.
> Observation:
> Even after the host is in maintenance CS shows the VMs as running and continuously seeing
the following NPE in logs:
> 2014-01-06 21:15:40,767 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-80:ctx-531f0e5c)
Seq 1-1135935494: Executing request
> 2014-01-06 21:15:40,934 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped i-2-11-VM
> 2014-01-06 21:15:40,934 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped r-6-VM
> 2014-01-06 21:15:40,935 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped s-10-VM
> 2014-01-06 21:15:40,935 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped r-12-VM
> 2014-01-06 21:15:40,935 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped r-13-VM
> 2014-01-06 21:15:40,936 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped r-4-VM
> 2014-01-06 21:15:40,936 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-80:ctx-531f0e5c)
The VM is now missing marking it as Stopped v-7-VM
> 2014-01-06 21:15:40,936 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-80:ctx-531f0e5c)
Seq 1-1135935494: Response Received:
> 2014-01-06 21:15:40,938 DEBUG [c.c.a.t.Request] (DirectAgent-80:ctx-531f0e5c) Seq 1-1135935494:
Processing:  { Ans: , MgmtId: 7588401905746, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.ClusterSyncAnswer":{"_clusterId":1,"_newStates":{"i-2-11-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"r-6-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"s-10-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"r-12-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"r-13-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"r-4-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"},"v-7-VM":{"t":"aef4c88e-59df-46df-8441-4985a6481ee8","u":"Stopped"}},"_isExecuted":false,"result":true,"wait":0}}]
}
> 2014-01-06 21:15:40,938 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-80:ctx-531f0e5c) Seq
1-1135935494: Unable to find listener.
> 2014-01-06 21:15:40,965 DEBUG [c.c.v.VirtualMachineManagerImpl] (DirectAgent-80:ctx-531f0e5c)
VM r-4-VM: cs state = Running and realState = Stopped
> 2014-01-06 21:15:40,965 DEBUG [c.c.v.VirtualMachineManagerImpl] (DirectAgent-80:ctx-531f0e5c)
VM r-4-VM: cs state = Running and realState = Stopped
> 2014-01-06 21:15:40,965 DEBUG [c.c.h.HighAvailabilityManagerImpl] (DirectAgent-80:ctx-531f0e5c)
VM does not require investigation so I'm marking it as Stopped: VM[DomainRouter|r-4-VM]
> 2014-01-06 21:15:40,965 WARN  [c.c.a.m.DirectAgentAttache] (DirectAgent-80:ctx-531f0e5c)
Seq 1-1135935494: Exception caught
> java.lang.NullPointerException
>  at com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1268)
>  at com.cloud.ha.HighAvailabilityManagerImpl.scheduleRestart(HighAvailabilityManagerImpl.java:346)
>  at com.cloud.vm.VirtualMachineManagerImpl.compareState(VirtualMachineManagerImpl.java:2719)
>  at com.cloud.vm.VirtualMachineManagerImpl.deltaSync(VirtualMachineManagerImpl.java:2353)
>  at com.cloud.vm.VirtualMachineManagerImpl.processAnswers(VirtualMachineManagerImpl.java:2830)
>  at com.cloud.agent.manager.AgentManagerImpl.notifyAnswersToMonitors(AgentManagerImpl.java:301)
>  at com.cloud.agent.manager.AgentAttache.processAnswers(AgentAttache.java:306)
>  at com.cloud.agent.manager.ClusteredDirectAgentAttache.processAnswers(ClusteredDirectAgentAttache.java:65)
>  at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:242)
>  at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>  at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>  at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>  at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>  at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:636)
> 2014-01-06 21:15:45,667 INFO  [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-c19a3a0d)
Found the following agents behind on ping: [7, 10]
> 2014-01-06 21:15:45,670 DEBUG [c.c.h.Status] (AgentMonitor-1:ctx-c19a3a0d) Ping timeout
for host 7, do invstigation
> 2014-01-06 21:15:45,672 INFO  [c.c.a.m.AgentManagerImpl] (AgentTaskPool-14:ctx-e3a1c19a)
Investigating why host 7 has disconnected with event PingTimeout
> 2014-01-06 21:15:45,673 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-14:ctx-e3a1c19a)
checking if agent (7) is alive
> Attaching the MS logs:



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message