Return-Path: X-Original-To: apmail-ambari-dev-archive@www.apache.org Delivered-To: apmail-ambari-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5416110F6F for ; Wed, 20 May 2015 20:05:04 +0000 (UTC) Received: (qmail 40885 invoked by uid 500); 20 May 2015 20:05:03 -0000 Delivered-To: apmail-ambari-dev-archive@ambari.apache.org Received: (qmail 40842 invoked by uid 500); 20 May 2015 20:05:03 -0000 Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ambari.apache.org Delivered-To: mailing list dev@ambari.apache.org Received: (qmail 40588 invoked by uid 99); 20 May 2015 20:05:03 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 May 2015 20:05:03 +0000 Date: Wed, 20 May 2015 20:05:02 +0000 (UTC) From: "Di Li (JIRA)" To: dev@ambari.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AMBARI-11285) Rolling upgrade: server throws "Work already begun on this thread. Looks like you have called UnitOfWork.begin() twice without a balancing call to end() in between" in Install Packages step MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/AMBARI-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Di Li updated AMBARI-11285: --------------------------- Description: I had a 3 node cluster. The ambari server node is also an ambari agent node. The server throws "Work already begun" error after all three agents had finished installing packages. Server always threw the error while dealing with agent heartbeat from the machine where it runs both the agent and the ambari server. SEVERE: Could not dispatch event: ActionFinalReportReceivedEvent{clusterId=2, hostname='bdavm076.svl.ibm.com', commandReportStatus=COMPLETED, commandReportRole=install_packages} to handler [wrapper public void org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(org.apache.ambari.server.events.ActionFinalReportReceivedEvent)] java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74) at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) at com.google.common.eventbus.EventBus.dispatch(EventBus.java:314) at com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34) at com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalStateException: Work already begun on this thread. Looks like you have called UnitOfWork.begin() twice without a balancing call to end() in between. at com.google.inject.internal.util.$Preconditions.checkState(Preconditions.java:142) at com.google.inject.persist.jpa.JpaPersistService.begin(JpaPersistService.java:66) at com.google.inject.persist.jpa.AmbariJpaPersistService.begin(AmbariJpaPersistService.java:29) at org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(DistributeRepositoriesActionListener.java:96) ... 12 more was: SEVERE: Could not dispatch event: ActionFinalReportReceivedEvent{clusterId=2, hostname='bdavm076.svl.ibm.com', commandReportStatus=COMPLETED, commandReportRole=install_packages} to handler [wrapper public void org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(org.apache.ambari.server.events.ActionFinalReportReceivedEvent)] java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74) at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) at com.google.common.eventbus.EventBus.dispatch(EventBus.java:314) at com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34) at com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalStateException: Work already begun on this thread. Looks like you have called UnitOfWork.begin() twice without a balancing call to end() in between. at com.google.inject.internal.util.$Preconditions.checkState(Preconditions.java:142) at com.google.inject.persist.jpa.JpaPersistService.begin(JpaPersistService.java:66) at com.google.inject.persist.jpa.AmbariJpaPersistService.begin(AmbariJpaPersistService.java:29) at org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(DistributeRepositoriesActionListener.java:96) ... 12 more > Rolling upgrade: server throws "Work already begun on this thread. Looks like you have called UnitOfWork.begin() twice without a balancing call to end() in between" in Install Packages step > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: AMBARI-11285 > URL: https://issues.apache.org/jira/browse/AMBARI-11285 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Affects Versions: 2.1.0 > Reporter: Di Li > Priority: Critical > Fix For: 2.1.0 > > > I had a 3 node cluster. The ambari server node is also an ambari agent node. The server throws "Work already begun" error after all three agents had finished installing packages. Server always threw the error while dealing with agent heartbeat from the machine where it runs both the agent and the ambari server. > SEVERE: Could not dispatch event: ActionFinalReportReceivedEvent{clusterId=2, hostname='bdavm076.svl.ibm.com', commandReportStatus=COMPLETED, commandReportRole=install_packages} to handler [wrapper public void org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(org.apache.ambari.server.events.ActionFinalReportReceivedEvent)] > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:74) > at com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) > at com.google.common.eventbus.EventBus.dispatch(EventBus.java:314) > at com.google.common.eventbus.AsyncEventBus.access$001(AsyncEventBus.java:34) > at com.google.common.eventbus.AsyncEventBus$1.run(AsyncEventBus.java:100) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.IllegalStateException: Work already begun on this thread. Looks like you have called UnitOfWork.begin() twice without a balancing call to end() in between. > at com.google.inject.internal.util.$Preconditions.checkState(Preconditions.java:142) > at com.google.inject.persist.jpa.JpaPersistService.begin(JpaPersistService.java:66) > at com.google.inject.persist.jpa.AmbariJpaPersistService.begin(AmbariJpaPersistService.java:29) > at org.apache.ambari.server.events.listeners.upgrade.DistributeRepositoriesActionListener.onActionFinished(DistributeRepositoriesActionListener.java:96) > ... 12 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)