Return-Path: X-Original-To: apmail-cloudstack-users-archive@www.apache.org Delivered-To: apmail-cloudstack-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2D00410D1E for ; Wed, 4 Feb 2015 12:02:39 +0000 (UTC) Received: (qmail 1899 invoked by uid 500); 4 Feb 2015 12:02:39 -0000 Delivered-To: apmail-cloudstack-users-archive@cloudstack.apache.org Received: (qmail 1842 invoked by uid 500); 4 Feb 2015 12:02:39 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 1830 invoked by uid 99); 4 Feb 2015 12:02:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Feb 2015 12:02:38 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of prashantkumar.mishra@citrix.com designates 185.25.65.24 as permitted sender) Received: from [185.25.65.24] (HELO SMTP.EU.CITRIX.COM) (185.25.65.24) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Feb 2015 12:02:14 +0000 X-IronPort-AV: E=Sophos;i="5.09,518,1418083200"; d="scan'208";a="31161348" From: Prashant Kumar Mishra To: "users@cloudstack.apache.org" Subject: Re: Unable to Cancel Maintenance mode after upgrading to from ACS 4.3.2 to 4.4.2 Thread-Topic: Unable to Cancel Maintenance mode after upgrading to from ACS 4.3.2 to 4.4.2 Thread-Index: PPBykd3Ev1tQZGh1ArRGUPqCtXJVX7+bOxyAJgmWXnD+z+HDgNm3Vd6rpkjXgwA= Date: Wed, 4 Feb 2015 12:01:04 +0000 Message-ID: References: <626428.972.1423007246280.JavaMail.andrei@tuchka> <15678843.987.1423007882082.JavaMail.andrei@tuchka> <13129632.1044.1423038996482.JavaMail.andrei@tuchka> <2137486.1155.1423046675302.JavaMail.andrei@tuchka> In-Reply-To: <2137486.1155.1423046675302.JavaMail.andrei@tuchka> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 Content-Type: text/plain; charset="us-ascii" Content-ID: <47F36548647C6442A5B10DC85C58672C@citrix.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-DLP: AMS1 X-Virus-Checked: Checked by ClamAV on apache.org Please open a ticket ~prashant On 2/4/15, 4:14 PM, "Andrei Mikhailovsky" wrote: >Prashant,=20 > >Yes, this is what I've done yesterday. I've change the resource_state to >Enable, just like the other hosts and done a force reconnect. However, >this was not enough. I also had to restart the agent on the host side for >things to be back to normal. After that I could successfully run vms on >the host and do live migration. > >After that I've tried to put the host in and out of Maintenance once >again. Same problem. Can't cancel the Maintenance mode. Seems like a bug. >Should I open a ticket? > >Andrei=20 > >----- Original Message ----- > >> From: "Prashant Kumar Mishra" >> To: users@cloudstack.apache.org >> Sent: Wednesday, 4 February, 2015 9:18:39 AM >> Subject: Re: Unable to Cancel Maintenance mode after upgrading to >> from ACS 4.3.2 to 4.4.2 > >> Seems a bug ,since host went in maintenance mode there is minimal >> chance >> for this but still good to check vm_instance table if some vm stuck >> on >> this host . > >> This should help, you can modify host table in cloud DB , update >> resource_State =3D Up and then do a force reconnect . > >> ~prashant > >> On 2/4/15, 2:06 PM, "Andrei Mikhailovsky" wrote: > >> >Prashant, >> > >> >the resource_state shows Maintenance >> > >> >Andrei >> > >> >----- Original Message ----- >> > >> >> From: "Prashant Kumar Mishra" >> >> To: users@cloudstack.apache.org >> >> Sent: Wednesday, 4 February, 2015 6:32:20 AM >> >> Subject: Re: Unable to Cancel Maintenance mode after upgrading to >> >> from ACS 4.3.2 to 4.4.2 >> > >> >> Can you check host table , for the host current state, is it in >> >> some >> >> transition state. >> > >> >> ~prashant >> > >> >> On 2/4/15, 5:28 AM, "Andrei Mikhailovsky" >> >> wrote: >> > >> >> >Hello guys, >> >> > >> >> >I have found another problem with ACS 4.4.2, which is pretty >> >> >serious >> >> >imho. I am no longer able to Cancel Maintenance mode. The GUI >> >> >shows >> >> >the >> >> >following error a few seconds after pressing the button: >> >> > >> >> >Command failed due to Internal Server Error >> >> > >> >> >The management server shows the following error: >> >> > >> >> >2015-02-03 23:42:15,621 DEBUG [c.c.a.ApiServlet] >> >> >(catalina-exec-23:ctx-04ea4b6d ctx-35701ff3) =3D=3D=3DEND=3D=3D=3D >> >> >192.168.169.91 -- >> >> >GET >> >> >>=20 >>>>>command=3DcancelHostMaintenance&id=3Dc092cb59-c770-4747-8d95-75aa49de5= d17& >>>>>re >> >>>sp >> >>=20 >>>onse=3Djson&sessionkey=3DfI2oaYTbgijs1h6HTOTMnJ%2FkChA%3D&_=3D1423006935= 464 >> >> >2015-02-03 23:42:15,622 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> >> >(API-Job-Executor-1:ctx-1fda9d17 job-11711) Add job-11711 into >> >> >job >> >> >monitoring >> >> >2015-02-03 23:42:15,623 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> >(API-Job-Executor-1:ctx-1fda9d17 job-11711) Executing AsyncJobVO >> >> >{id:11711, userId: 3, accountId: 2, instanceType: Host, >> >> >instanceId: >> >> >1, >> >> >cmd: >> >> >org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd, >> >> >cmdInfo: >> >> >>=20 >>>>>{"id":"c092cb59-c770-4747-8d95-75aa49de5d17","response":"json","sessio >>>>>nk >> >>>ey >> >> >>=20 >>>>>":"fI2oaYTbgijs1h6HTOTMnJ/kChA\u003d","ctxDetails":"{\"com.cloud.host. >>>>>Ho >> >>>st >> >> >>=20 >>>>>\":\"c092cb59-c770-4747-8d95-75aa49de5d17\"}","cmdEventType":"MAINT.CA >>>>>NC >> >>>EL >> >> >>=20 >>>>>","ctxUserId":"3","httpmethod":"GET","_":"1423006935464","uuid":"c092c >>>>>b5 >> >>>9- >> >> >>=20 >>>>>c770-4747-8d95-75aa49de5d17","ctxAccountId":"2","ctxStartEventId":"648 >>>>>57 >> >>>"} >> >> >, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, >> >> >resultCode: >> >> >0, >> >> >result: null, initMsid: 115129173025114, completeMsid: null, >> >> >lastUpdated: >> >> >null, lastPolled: null, created: null} >> >> >2015-02-03 23:42:15,646 ERROR [c.c.a.ApiAsyncJobDispatcher] >> >> >(API-Job-Executor-1:ctx-1fda9d17 job-11711) Unexpected exception >> >> >while >> >> >executing >> >> >org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd >> >> >java.lang.NullPointerException >> >> >at >> >> >>=20 >>>>>com.cloud.resource.ResourceManagerImpl.doCancelMaintenance(ResourceMan >>>>>ag >> >>>er >> >> >Impl.java:2083) >> >> >at >> >> >>=20 >>>>>com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManag >>>>>er >> >>>Im >> >> >pl.java:2140) >> >> >at >> >> >>=20 >>>>>com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManag >>>>>er >> >>>Im >> >> >pl.java:1127) >> >> >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> >at >> >> >>=20 >>>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j >>>>>av >> >>>a: >> >> >57) >> >> >at >> >> >>=20 >>>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess >>>>>or >> >>>Im >> >> >pl.java:43) >> >> >at java.lang.reflect.Method.invoke(Method.java:606) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflectio >>>>>n( >> >>>Ao >> >> >pUtils.java:317) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoi >>>>>np >> >>>oi >> >> >nt(ReflectiveMethodInvocation.java:183) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(R >>>>>ef >> >>>le >> >> >ctiveMethodInvocation.java:150) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke >>>>>(E >> >>>xp >> >> >oseInvocationInterceptor.java:91) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(R >>>>>ef >> >>>le >> >> >ctiveMethodInvocation.java:172) >> >> >at >> >> >>=20 >>>>>org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamic >>>>>Ao >> >>>pP >> >> >roxy.java:204) >> >> >at com.sun.proxy.$Proxy148.cancelMaintenance(Unknown Source) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd.exec >>>>>ut >> >>>e( >> >> >CancelMaintenanceCmd.java:102) >> >> >at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141) >> >> >at >> >> >>=20 >>>>>com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java: >>>>>10 >> >>>8) >> >> > >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInC >>>>>on >> >>>te >> >> >xt(AsyncJobManagerImpl.java:503) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(Man >>>>>ag >> >>>ed >> >> >ContextRunnable.java:49) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.cal >>>>>l( >> >>>De >> >> >faultManagedContext.java:56) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callW >>>>>it >> >>>hC >> >> >ontext(DefaultManagedContext.java:103) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWi >>>>>th >> >>>Co >> >> >ntext(DefaultManagedContext.java:53) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Manag >>>>>ed >> >>>Co >> >> >ntextRunnable.java:46) >> >> >at >> >> >>=20 >>>>>org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(As >>>>>yn >> >>>cJ >> >> >obManagerImpl.java:460) >> >> >at >> >>=20 >>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >> >> >at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> >> >at >> >> >>=20 >>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j >>>>>av >> >>>a: >> >> >1145) >> >> >at >> >> >>=20 >>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >>>>>ja >> >>>va >> >> >:615) >> >> >at java.lang.Thread.run(Thread.java:745) >> >> >2015-02-03 23:42:15,648 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> >(API-Job-Executor-1:ctx-1fda9d17 job-11711) Complete async >> >> >job-11711, >> >> >jobStatus: FAILED, resultCode: 530, result: >> >> >>=20 >>>>>org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": >>>>>[] >> >>>," >> >> >errorcode":530} >> >> >2015-02-03 23:42:15,664 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> >(API-Job-Executor-1:ctx-1fda9d17 job-11711) Done executing >> >> >org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd >> >> >for >> >> >job-11711 >> >> > >> >> > >> >> >There is absolutely nothing in the agent logs even if I use the >> >> >DEBUG >> >> >logging. >> >> > >> >> >I get the same error if I use CloudMonkey to cancel the >> >> >Maintenance >> >> >mode. >> >> > >> >> >I am running Ubuntu 14.04 on both the management server and the >> >> >agents. >> >> >I've recently upgraded from ACS 4.3.2 to version 4.4.2 where this >> >> >function was working. >> >> > >> >> >Does anyone know what is going on and how to fix it? >> >> > >> >> > >> >> > >> >> >Andrei >> >> >