cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Somesh Naidu <Somesh.Na...@citrix.com>
Subject RE: Encountered unhandled exception during HA process
Date Tue, 17 Feb 2015 16:16:08 GMT
You'd probably need to delete the corresponding record from op_ha_work table. I guess there
is a HA task being scheduled for a VM that may no longer exists or something similar.

If you believe you haven't performed any manual DB updates prior to this then this NPE should
be treated as a defect and you should file a bug report for the same.

Regards,
Somesh


-----Original Message-----
From: Martin Emrich [mailto:martin.emrich@empolis.com] 
Sent: Tuesday, February 17, 2015 7:48 AM
To: users@cloudstack.apache.org
Subject: Encountered unhandled exception during HA process

Hello!

I just discovered that I periodically (every few minutes) a lot of these 
messages in the server log:

------------------------
2015-02-17 11:50:03,649 INFO  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-3:ctx-ee9d5d55 work-793) Processing 
HAWork[793-Migration-2-Stopped-Migrating]
2015-02-17 11:50:03,651 WARN  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-3:ctx-ee9d5d55 work-793) Encountered unhandled exception 
during HA process, reschedule retry
java.lang.NullPointerException
         at 
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
2015-02-17 11:50:03,651 INFO  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-4:ctx-029c212c work-794) Processing 
HAWork[794-Migration-2-Stopped-Migrating]
2015-02-17 11:50:03,651 INFO  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-3:ctx-ee9d5d55 work-793) Rescheduling 
HAWork[793-Migration-2-Stopped-Migrating] to try again at Tue Feb 17 
12:00:17 CET 2015
2015-02-17 11:50:03,651 ERROR [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-3:ctx-ee9d5d55 work-793) Caught this throwable,
java.lang.NullPointerException
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:925)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
2015-02-17 11:50:03,652 WARN  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-4:ctx-029c212c work-794) Encountered unhandled exception 
during HA process, reschedule retry
java.lang.NullPointerException
         at 
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
2015-02-17 11:50:03,652 INFO  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-4:ctx-029c212c work-794) Rescheduling 
HAWork[794-Migration-2-Stopped-Migrating] to try again at Tue Feb 17 
12:00:17 CET 2015
2015-02-17 11:50:03,653 INFO  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-1:ctx-30ba9813 work-795) Processing 
HAWork[795-Migration-2-Stopped-Migrating]
2015-02-17 11:50:03,653 ERROR [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-4:ctx-029c212c work-794) Caught this throwable,
java.lang.NullPointerException
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:925)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
2015-02-17 11:50:03,654 WARN  [c.c.h.HighAvailabilityManagerImpl] 
(HA-Worker-1:ctx-30ba9813 work-795) Encountered unhandled exception 
during HA process, reschedule retry
java.lang.NullPointerException
         at 
com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at 
com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
------------------

All VMs are running fine, so from the "outside" I cannot see anything wrong.

We run ACS 4.4.2 with 5x XenServer 6.2.

Can I fix this somehow?

Thanks

Martin
Mime
View raw message