Return-Path: X-Original-To: apmail-ambari-user-archive@www.apache.org Delivered-To: apmail-ambari-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 908FD18B9D for ; Tue, 8 Mar 2016 15:01:00 +0000 (UTC) Received: (qmail 29988 invoked by uid 500); 8 Mar 2016 15:01:00 -0000 Delivered-To: apmail-ambari-user-archive@ambari.apache.org Received: (qmail 29955 invoked by uid 500); 8 Mar 2016 15:01:00 -0000 Mailing-List: contact user-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ambari.apache.org Delivered-To: mailing list user@ambari.apache.org Received: (qmail 29940 invoked by uid 99); 8 Mar 2016 15:01:00 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Mar 2016 15:01:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id CB5DDC002C for ; Tue, 8 Mar 2016 15:00:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.798 X-Spam-Level: ** X-Spam-Status: No, score=2.798 tagged_above=-999 required=6.31 tests=[FSL_HELO_BARE_IP_2=1.499, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id R67w1xbsHkXK for ; Tue, 8 Mar 2016 15:00:54 +0000 (UTC) Received: from relayvx11b.securemail.intermedia.net (relayvx11b.securemail.intermedia.net [64.78.52.184]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 093605FE2F for ; Tue, 8 Mar 2016 15:00:53 +0000 (UTC) Received: from securemail.intermedia.net (localhost [127.0.0.1]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id 4E6C053EEA for ; Tue, 8 Mar 2016 07:00:51 -0800 (PST) Subject: Re: Ambari - error 500 - getAllRequests after upgrade MIME-Version: 1.0 x-echoworx-msg-id: 27eff355-27ef-42a3-a8e3-3445f9c46de1 x-echoworx-emg-received: Tue, 8 Mar 2016 07:00:51.250 -0800 x-echoworx-message-code-hashed: b71588e2cd7bdcd2c2d47155aee4c306d83d9b41b8ed10bfda4ceb31a3f59c1b x-echoworx-action: delivered Received: from 10.254.155.14 ([10.254.155.14]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 757 for ; Tue, 8 Mar 2016 07:00:51 -0800 (PST) Received: from MBX080-W4-CO-2.exch080.serverpod.net (unknown [10.224.117.102]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by emg-ca-1-1.localdomain (Postfix) with ESMTPS id EFD5053F2E for ; Tue, 8 Mar 2016 07:00:50 -0800 (PST) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) with Microsoft SMTP Server (TLS) id 15.0.1130.7; Tue, 8 Mar 2016 07:00:50 -0800 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1130.005; Tue, 8 Mar 2016 07:00:50 -0800 From: Jonathan Hurley To: "user@ambari.apache.org" Thread-Topic: Ambari - error 500 - getAllRequests after upgrade Thread-Index: AQHReSkSDnRYO4abZUi/kP/WYE78I59QKoYA Date: Tue, 8 Mar 2016 15:00:49 +0000 Message-ID: <8E9991E4-01EA-4604-AFF4-936A97D99AE5@hortonworks.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [72.82.246.82] x-source-routing-agent: Processed Content-Type: multipart/alternative; boundary="_000_8E9991E401EA4604AFF4936A97D99AE5hortonworkscom_" --_000_8E9991E401EA4604AFF4936A97D99AE5hortonworkscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable That's very odd, especially since the upgrade doesn't touch the topology ta= bles. Are you using MySQL by any chance? If so, can you check to make sure = that your database engine is Innodb and not MyISAM. You have an integrity v= iolation here which doesn't seem possible unless you're using a database wh= ich doesn't support foreign key constraints. There's probably some SQL which you can run to insert an entry into the top= ology_logical_request table, but it's probably best to understand why this = happened first. On Mar 8, 2016, at 5:55 AM, cs user > wrote: Hi All, I've upgraded Ambari from version 2.1.2-377 to version 2.2.1.0-161. After performing the upgrade on the server, agents, upgrading the database = and starting everything up, I keep seeing the following error in the logs o= n the server: 08 Mar 2016 10:07:05,087 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,088 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 3 08 Mar 2016 10:07:05,120 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,120 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 5 08 Mar 2016 10:07:05,134 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,134 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 8 08 Mar 2016 10:07:05,147 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,148 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 7 08 Mar 2016 10:07:05,158 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,158 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 6 08 Mar 2016 10:07:05,170 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,170 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 2 08 Mar 2016 10:07:05,184 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,185 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 1 08 Mar 2016 10:07:05,194 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: Host Assignment Pend= ing 08 Mar 2016 10:07:05,194 INFO [qtp-ambari-agent-55] LogicalRequest:420 - L= ogicalRequest.createHostRequests: created new outstanding host request ID = =3D 4 08 Mar 2016 10:07:05,290 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-21.node.example 08 Mar 2016 10:07:05,328 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-51.node.example 08 Mar 2016 10:07:05,384 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-11.node.example 08 Mar 2016 10:07:05,428 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-41.node.example 08 Mar 2016 10:07:05,507 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-31.node.example 08 Mar 2016 10:07:05,575 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-53.node.example 08 Mar 2016 10:07:05,627 INFO [qtp-ambari-agent-55] HostRequest:125 - Host= Request: Successfully recovered host request for host: ambdevtestdc2host-gr= oup-52.node.example 08 Mar 2016 10:07:05,644 WARN [qtp-ambari-agent-55] ServletHandler:563 - /= agent/v1/register/ambdevtestdc2host-group-51.node.example java.lang.NullPointerException at org.apache.ambari.server.topology.PersistedStateImpl.getAllReque= sts(PersistedStateImpl.java:157) at org.apache.ambari.server.topology.TopologyManager.ensureInitiali= zed(TopologyManager.java:131) at org.apache.ambari.server.topology.TopologyManager.onHostRegister= ed(TopologyManager.java:315) at org.apache.ambari.server.state.host.HostImpl$HostRegistrationRec= eived.transition(HostImpl.java:301) at org.apache.ambari.server.state.host.HostImpl$HostRegistrationRec= eived.transition(HostImpl.java:266) at org.apache.ambari.server.state.fsm.StateMachineFactory$SingleInt= ernalArc.doTransition(StateMachineFactory.java:354) at org.apache.ambari.server.state.fsm.StateMachineFactory.doTransit= ion(StateMachineFactory.java:294) at org.apache.ambari.server.state.fsm.StateMachineFactory.access$30= 0(StateMachineFactory.java:39) at org.apache.ambari.server.state.fsm.StateMachineFactory$InternalS= tateMachine.doTransition(StateMachineFactory.java:440) at org.apache.ambari.server.state.host.HostImpl.handleEvent(HostImp= l.java:570) at org.apache.ambari.server.agent.HeartBeatHandler.handleRegistrati= on(HeartBeatHandler.java:966) at org.apache.ambari.server.agent.rest.AgentResource.register(Agent= Resource.java:95) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor= Impl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod= AccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(J= avaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourc= eMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDisp= atchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMet= hodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpM= ethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(Ri= ghtHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(Re= sourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(Ri= ghtHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.acc= ept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handl= eRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handl= eRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handle= Request(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handle= Request(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebCom= ponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(Se= rvletContainer.java:540) at com.sun.jersey.spi.container.servlet.ServletContainer.service(Se= rvletContainer.java:715) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.jav= a:684) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Se= rvletHandler.java:1496) at org.apache.ambari.server.security.SecurityFilter.doFilter(Securi= tyFilter.java:67) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Se= rvletHandler.java:1467) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(Ambari= PersistFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Se= rvletHandler.java:1467) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFil= ter.java:82) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:2= 94) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Se= rvletHandler.java:1467) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler= .java:501) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(Context= Handler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.= java:429) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextH= andler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHand= ler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWr= apper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(Ab= stractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(Abstract= HttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.c= ontent(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java= :240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpCon= nection.java:82) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java= :196) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChan= nelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChann= elEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThre= adPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThrea= dPool.java:543) at java.lang.Thread.run(Thread.java:745) This is not specific to host group ambdevtestdc2host-group-51.node.example,= it is happening for all host groups. On the agents I see the following: Error 500 Server Error

HTTP ERROR: 500

Problem accessing /agent/v1/register/ambdevtestdc2host-group-51.node.exa= mple Reason: 

    Server Error

Powered by Jetty:// Is there a work around for this? It's just a test cluster, but it would be = good to know how to work around this, as I've seen it a number of times now= . Is there anything that can be modified in the database to resolve it? Thanks! --_000_8E9991E401EA4604AFF4936A97D99AE5hortonworkscom_ Content-Type: text/html; charset="us-ascii" Content-ID: <9380681C1A3BB94F9962B06CAC193216@exch080.serverpod.net> Content-Transfer-Encoding: quoted-printable That's very odd, espe= cially since the upgrade doesn't touch the topology tables. Are you using MySQL by any chance? If so, can you che= ck to make sure that your database engine is Innodb and not MyISAM. You hav= e an integrity violation here which doesn't seem possible unless you're usi= ng a database which doesn't support foreign key constraints.

There's pr= obably some SQL which you can run to insert an entry into the topology_logical_request table, but it's probably best to u= nderstand why this happened first.

On Mar 8, 2016, at 5:55 AM, cs user <acldstkusr@gmail.com> wrote:

Hi All, 

I've upgraded Ambari from version 2.1.2-377 to version=  2.2.1.0-161. 

After performing the upgrade on the server, agents, upgradi= ng the database and starting everything up, I keep seeing the following err= or in the logs on the server:

08 Mar 2016 10:07:05,087  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,088  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 3
08 Mar 2016 10:07:05,120  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,120  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 5
08 Mar 2016 10:07:05,134  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,134  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 8
08 Mar 2016 10:07:05,147  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,148  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 7
08 Mar 2016 10:07:05,158  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,158  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 6
08 Mar 2016 10:07:05,170  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,170  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 2
08 Mar 2016 10:07:05,184  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,185  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 1
08 Mar 2016 10:07:05,194  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= Host Assignment Pending
08 Mar 2016 10:07:05,194  INFO [qtp-ambari-agent-55] L= ogicalRequest:420 - LogicalRequest.createHostRequests: created new outstand= ing host request ID =3D 4
08 Mar 2016 10:07:05,290  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-21.node.example
08 Mar 2016 10:07:05,328  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-51.node.example
08 Mar 2016 10:07:05,384  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-11.node.example
08 Mar 2016 10:07:05,428  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-41.node.example
08 Mar 2016 10:07:05,507  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-31.node.example
08 Mar 2016 10:07:05,575  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-53.node.example
08 Mar 2016 10:07:05,627  INFO [qtp-ambari-agent-55] H= ostRequest:125 - HostRequest: Successfully recovered host request for host:= ambdevtestdc2host-group-52.node.example
08 Mar 2016 10:07:05,644  WARN [qtp-ambari-agent-55] S= ervletHandler:563 - /agent/v1/register/ambdevtestdc2host-group-51.node.exam= ple
java.lang.NullPointerException
        at org.apache.ambari.server.top= ology.PersistedStateImpl.getAllRequests(PersistedStateImpl.java:157)
        at org.apache.ambari.server.top= ology.TopologyManager.ensureInitialized(TopologyManager.java:131)
        at org.apache.ambari.server.top= ology.TopologyManager.onHostRegistered(TopologyManager.java:315)
        at org.apache.ambari.server.sta= te.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:301)
        at org.apache.ambari.server.sta= te.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:266)
        at org.apache.ambari.server.sta= te.fsm.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFacto= ry.java:354)
        at org.apache.ambari.server.sta= te.fsm.StateMachineFactory.doTransition(StateMachineFactory.java:294)
        at org.apache.ambari.server.sta= te.fsm.StateMachineFactory.access$300(StateMachineFactory.java:39)
        at org.apache.ambari.server.sta= te.fsm.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFa= ctory.java:440)
        at org.apache.ambari.server.sta= te.host.HostImpl.handleEvent(HostImpl.java:570)
        at org.apache.ambari.server.age= nt.HeartBeatHandler.handleRegistration(HeartBeatHandler.java:966)
        at org.apache.ambari.server.age= nt.rest.AgentResource.register(AgentResource.java:95)
        at sun.reflect.NativeMethodAcce= ssorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAcce= ssorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethod= AccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.inv= oke(Method.java:497)
        at com.sun.jersey.spi.container= .JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
        at com.sun.jersey.server.impl.m= odel.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker.= _dispatch(AbstractResourceMethodDispatchProvider.java:185)
        at com.sun.jersey.server.impl.m= odel.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMeth= odDispatcher.java:75)
        at com.sun.jersey.server.impl.u= ri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
        at com.sun.jersey.server.impl.u= ri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.u= ri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
        at com.sun.jersey.server.impl.u= ri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
        at com.sun.jersey.server.impl.u= ri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
        at com.sun.jersey.server.impl.a= pplication.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)<= /div>
        at com.sun.jersey.server.impl.a= pplication.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)<= /div>
        at com.sun.jersey.server.impl.a= pplication.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
        at com.sun.jersey.server.impl.a= pplication.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
        at com.sun.jersey.spi.container= .servlet.WebComponent.service(WebComponent.java:409)
        at com.sun.jersey.spi.container= .servlet.ServletContainer.service(ServletContainer.java:540)
        at com.sun.jersey.spi.container= .servlet.ServletContainer.service(ServletContainer.java:715)
        at javax.servlet.http.HttpServl= et.service(HttpServlet.java:848)
        at org.eclipse.jetty.servlet.Se= rvletHolder.handle(ServletHolder.java:684)
        at org.eclipse.jetty.servlet.Se= rvletHandler$CachedChain.doFilter(ServletHandler.java:1496)
        at org.apache.ambari.server.sec= urity.SecurityFilter.doFilter(SecurityFilter.java:67)
        at org.eclipse.jetty.servlet.Se= rvletHandler$CachedChain.doFilter(ServletHandler.java:1467)
        at org.apache.ambari.server.api= .AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)
        at org.eclipse.jetty.servlet.Se= rvletHandler$CachedChain.doFilter(ServletHandler.java:1467)
        at org.eclipse.jetty.servlets.U= serAgentFilter.doFilter(UserAgentFilter.java:82)
        at org.eclipse.jetty.servlets.G= zipFilter.doFilter(GzipFilter.java:294)
        at org.eclipse.jetty.servlet.Se= rvletHandler$CachedChain.doFilter(ServletHandler.java:1467)
        at org.eclipse.jetty.servlet.Se= rvletHandler.doHandle(ServletHandler.java:501)
        at org.eclipse.jetty.server.han= dler.ContextHandler.doHandle(ContextHandler.java:1086)
        at org.eclipse.jetty.servlet.Se= rvletHandler.doScope(ServletHandler.java:429)
        at org.eclipse.jetty.server.han= dler.ContextHandler.doScope(ContextHandler.java:1020)
        at org.eclipse.jetty.server.han= dler.ScopedHandler.handle(ScopedHandler.java:135)
        at org.eclipse.jetty.server.han= dler.HandlerWrapper.handle(HandlerWrapper.java:116)
        at org.eclipse.jetty.server.Ser= ver.handle(Server.java:370)
        at org.eclipse.jetty.server.Abs= tractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
        at org.eclipse.jetty.server.Abs= tractHttpConnection.content(AbstractHttpConnection.java:982)
        at org.eclipse.jetty.server.Abs= tractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043= )
        at org.eclipse.jetty.http.HttpP= arser.parseNext(HttpParser.java:865)
        at org.eclipse.jetty.http.HttpP= arser.parseAvailable(HttpParser.java:240)
        at org.eclipse.jetty.server.Asy= ncHttpConnection.handle(AsyncHttpConnection.java:82)
        at org.eclipse.jetty.io.nio.Ssl= Connection.handle(SslConnection.java:196)
        at org.eclipse.jetty.io.nio.Sel= ectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
        at org.eclipse.jetty.io.nio.Sel= ectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
        at org.eclipse.jetty.util.threa= d.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.threa= d.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.= java:745)


This is not specific to host group ambdevtestdc2host-group-= 51.node.example, it is happening for all host groups. 

On the agents I see the following:

<head>
<meta http-equiv=3D"Content-Type" content=3D&q= uot;text/html;charset=3DISO-8859-1"/>
<title>Error 500 Server Error</title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /agent/v1/register/ambdevtestdc2= host-group-51.node.example Reason:
<pre>    Server Error</pre></p>=
<hr /><i><small>Powered by Jetty://</s= mall></i>

Is there a work around for this? It's just a test cluster, = but it would be good to know how to work around this, as I've seen it a num= ber of times now. Is there anything that can be modified in the database to= resolve it?

Thanks!




--_000_8E9991E401EA4604AFF4936A97D99AE5hortonworkscom_--