ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Sposetti <j...@hortonworks.com>
Subject Re: Ambari UI not recovering after Machine power failure
Date Wed, 09 Jul 2014 13:45:01 GMT
Can you confirm ambari server is running? And stop/start as necessary?

ambari-server status

Also, confirm postgres is running after you stop/start ambari-server?

ps -ef | grep postgres
service postgresql status



On Wed, Jul 9, 2014 at 8:49 AM, Suraj Nayak <snayakm@gmail.com> wrote:

> Hi,
>
> I have a 3 node cluster with HA enabled via Ambari. Accidentally the power
> supply to all the nodes was disrupted. After the power was restored, the 3
> servers were turned on. Now if I try to login to Ambari Web UI, an error
> pops up  *Error in retrieving web client state from ambari server.*(After
> entering username and password)
>
> It doesn't show the cluster dashboard which was showing before. Instead it
> shows *"Cluster Install Wizard" *to setup a new cluster. How can I
> recover from the error? I do not want to restart the installation process
>
> Below is the error I found in */var/log/ambari-server/ambari-server.log *log
> file.
>
> I found an email archive at
> http://mail-archives.apache.org/mod_mbox/ambari-user/201402.mbox/%3CCF1BC3ED.2E43F%25are9004@nyp.org%3E.
> But I need suggestions on recovering but not starting the installation from
> scratch.
>
> *Error :*
>
> Local Exception Stack:
> Exception [EclipseLink-4002] (Eclipse Persistence Services -
> 2.4.0.v20120608-r11652):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: missing
> chunk number 0 for toast value 50294 in pg_toast_16548
> Error Code: 0
> Call: SELECT "key", "value" FROM key_value_store WHERE ("key" = ?)
>     bind => [1 parameter bound]
> Query: ReadObjectQuery(name="readObject" referenceClass=KeyValueEntity
> sql="SELECT "key", "value" FROM key_value_store WHERE ("key" = ?)")
>     at
> org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
>     at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:646)
>     at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:537)
>     at
> org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1800)
>     at
> org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:566)
>     at
> org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:207)
>     at
> org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:193)
>     at
> org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:668)
>     at
> org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2744)
>     at
> org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2697)
>     at
> org.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:452)
>     at
> org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1149)
>     at
> org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:852)
>     at
> org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1108)
>     at
> org.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:420)
>     at
> org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1196)
>     at
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2875)
>     at
> org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1602)
>     at
> org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1584)
>     at
> org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1535)
>     at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:838)
>     at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:778)
>     at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:671)
>     at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:543)
>     at
> org.apache.ambari.server.orm.dao.KeyValueDAO.findByKey(KeyValueDAO.java:42)
>     at
> org.apache.ambari.server.orm.AmbariLocalSessionInterceptor.invoke(AmbariLocalSessionInterceptor.java:53)
>     at
> org.apache.ambari.server.api.services.PersistKeyValueImpl.getValue(PersistKeyValueImpl.java:50)
>     at
> org.apache.ambari.server.api.services.PersistKeyValueService.getKey(PersistKeyValueService.java:83)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>     at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
>     at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
>     at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
>     at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>     at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>     at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>     at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>     at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
>     at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
>     at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
>     at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
>     at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
>     at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
>     at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:708)
>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>     at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:652)
>     at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1329)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
>     at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
>     at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.apache.ambari.server.security.authorization.internal.InternalTokenAuthenticationFilter.doFilter(InternalTokenAuthenticationFilter.java:53)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
>     at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
>     at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
>     at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
>     at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
>     at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
>     at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300)
>     at
> org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:48)
>     at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1300)
>     at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:445)
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>     at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:559)
>     at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1038)
>     at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:374)
>     at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189)
>     at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:972)
>     at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>     at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
>     at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>     at org.eclipse.jetty.server.Server.handle(Server.java:363)
>     at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
>     at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920)
>     at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982)
>     at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
>     at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>     at
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>     at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:627)
>     at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:51)
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>     at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>     at java.lang.Thread.run(Thread.java:745)
> Caused by: org.postgresql.util.PSQLException: ERROR: missing chunk number
> 0 for toast value 50294 in pg_toast_16548
>     at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
>     at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
>     at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
>     at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:559)
>     at
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
>     at
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:302)
>     at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:938)
>     at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:609)
>     ... 98 more
>
>
> --
> Thanks
> Suraj Nayak
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message