cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Emrich <martin.emr...@empolis.com>
Subject Re: Encountered unhandled exception during HA process
Date Fri, 27 Feb 2015 11:21:50 GMT
Yes... I more and more learn the first rule with Cloudstack: If 
something does not work: Wait a day. If something is strange: Wait a 
week. ;)

Cheers

Martin

Am 26.02.2015 um 21:19 schrieb Somesh Naidu:
> Wonderful! Guess the HA task eventually hit the retry attempt and ended in Error state.
>
> Regards,
> Somesh
>
>
> -----Original Message-----
> From: Martin Emrich [mailto:martin.emrich@empolis.com]
> Sent: Thursday, February 26, 2015 5:44 AM
> To: users@cloudstack.apache.org
> Subject: AW: Encountered unhandled exception during HA process
>
> Hmm, without doing anything, the messages stopped by themselves ;)
>
> Thanks
>
> Martin
>
> -----Urspr√ľngliche Nachricht-----
> Von: Somesh Naidu [mailto:Somesh.Naidu@citrix.com]
> Gesendet: Dienstag, 17. Februar 2015 17:16
> An: users@cloudstack.apache.org
> Betreff: RE: Encountered unhandled exception during HA process
>
> You'd probably need to delete the corresponding record from op_ha_work table. I guess
there is a HA task being scheduled for a VM that may no longer exists or something similar.
>
> If you believe you haven't performed any manual DB updates prior to this then this NPE
should be treated as a defect and you should file a bug report for the same.
>
> Regards,
> Somesh
>
>
> -----Original Message-----
> From: Martin Emrich [mailto:martin.emrich@empolis.com]
> Sent: Tuesday, February 17, 2015 7:48 AM
> To: users@cloudstack.apache.org
> Subject: Encountered unhandled exception during HA process
>
> Hello!
>
> I just discovered that I periodically (every few minutes) a lot of these messages in
the server log:
>
> ------------------------
> 2015-02-17 11:50:03,649 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-3:ctx-ee9d5d55 work-793) Processing HAWork[793-Migration-2-Stopped-Migrating]
> 2015-02-17 11:50:03,651 WARN  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-3:ctx-ee9d5d55 work-793) Encountered unhandled exception during HA process,
reschedule retry java.lang.NullPointerException
>           at
> com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
> 2015-02-17 11:50:03,651 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-4:ctx-029c212c work-794) Processing HAWork[794-Migration-2-Stopped-Migrating]
> 2015-02-17 11:50:03,651 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-3:ctx-ee9d5d55 work-793) Rescheduling HAWork[793-Migration-2-Stopped-Migrating]
to try again at Tue Feb 17
> 12:00:17 CET 2015
> 2015-02-17 11:50:03,651 ERROR [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-3:ctx-ee9d5d55 work-793) Caught this throwable, java.lang.NullPointerException
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:925)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
> 2015-02-17 11:50:03,652 WARN  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-4:ctx-029c212c work-794) Encountered unhandled exception during HA process,
reschedule retry java.lang.NullPointerException
>           at
> com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
> 2015-02-17 11:50:03,652 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-4:ctx-029c212c work-794) Rescheduling HAWork[794-Migration-2-Stopped-Migrating]
to try again at Tue Feb 17
> 12:00:17 CET 2015
> 2015-02-17 11:50:03,653 INFO  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-1:ctx-30ba9813 work-795) Processing HAWork[795-Migration-2-Stopped-Migrating]
> 2015-02-17 11:50:03,653 ERROR [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-4:ctx-029c212c work-794) Caught this throwable, java.lang.NullPointerException
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:925)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
> 2015-02-17 11:50:03,654 WARN  [c.c.h.HighAvailabilityManagerImpl]
> (HA-Worker-1:ctx-30ba9813 work-795) Encountered unhandled exception during HA process,
reschedule retry java.lang.NullPointerException
>           at
> com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:631)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.runWithContext(HighAvailabilityManagerImpl.java:891)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.access$000(HighAvailabilityManagerImpl.java:848)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread$1.run(HighAvailabilityManagerImpl.java:860)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>           at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>           at
> com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:857)
> ------------------
>
> All VMs are running fine, so from the "outside" I cannot see anything wrong.
>
> We run ACS 4.4.2 with 5x XenServer 6.2.
>
> Can I fix this somehow?
>
> Thanks
>
> Martin
>

Mime
View raw message