nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Lim <Keith....@ds-iq.com>
Subject Re: Starting Nifi encounters "unexpected error"
Date Mon, 16 May 2016 22:53:19 GMT
Thanks Andy,


There was an out-of-memory error which I didn't mention in the log.  I decided to clear out
the 4 repository directories, thinking that I may have to redo the workflow again.  I was
pleasantly surprised to see when it started up, all my  workflow intact.

Good to know where the actual workflow is stored.


Thanks,
Keith


________________________________
From: Andy LoPresto <alopresto@apache.org>
Sent: Monday, May 16, 2016 3:35 PM
To: users@nifi.apache.org
Subject: Re: Starting Nifi encounters "unexpected error"

Keith,

Can you check the <install dir>/logs/nifi-bootstrap.log and <install dir>/logs/nifi-app.log
files to see if there is an error explaining why the app does not start up?

The workflow itself is saved as <install dir>/conf/flow.xml.gz. I understand this terminology
can be confusing — a “flowfile” is the atomic unit of data in NiFi, and references the
associated content and provenance records as it passes through the data flow. The data flow
file (what you called your workflow) is not a “flowfile” and is stored at the location
above. It is a (compressed) XML representation of the data flow.

Because you deleted the content_repository, you will not be able to resume operation on the
data that was queued or being processed at the time the server went down. You may still have
flowfiles marked for processing in the flowfile_repository, and you may have to clear those
out. In the future, don’t delete the repositories when recovering from a server failure
unless further debugging demonstrates it is absolutely required. NiFi is fairly resilient
about resuming processing after the machine is restored.

Let us know if this does not help.


Andy LoPresto
alopresto@apache.org<mailto:alopresto@apache.org>
alopresto.apache@gmail.com<mailto:alopresto.apache@gmail.com>
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On May 16, 2016, at 2:31 PM, Keith Lim <Keith.Lim@ds-iq.com<mailto:Keith.Lim@ds-iq.com>>
wrote:

My Windows server was restarted and when I started Nifi again, I can longer browse to it :

I have done lots of work on a workflow, is there anyway I can retrieve that?
I have deleted the  content_repository and database_repository, but I have a feeling that
my work is still in the flowfile_repository.
Should I delete the provenance_repository?   How can I extract my workflow from the flowfile_repository?

Thanks,
Keith

<pastedImage.png>





________________________________
From: Keith Lim <Keith.Lim@ds-iq.com<mailto:Keith.Lim@ds-iq.com>>
Sent: Wednesday, May 04, 2016 2:16 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: RE: Starting Nifi encounters "unexpected error"

Thanks Matt, that works.



From: Matt Gilman [mailto:matt.c.gilman@gmail.com]
Sent: Wednesday, May 4, 2016 1:59 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: Starting Nifi encounters "unexpected error"



Keith,



If you restart NiFi again it doesn't help, correct?



Assuming this is the case, can you trying deleting the users database from database_repository
directory and restart again. The database will be created during startup so if it is in a
bad state it should be re-created. The database is simply a cache of users for the system.



Matt



On Wed, May 4, 2016 at 4:51 PM, Keith Lim <Keith.Lim@ds-iq.com<mailto:Keith.Lim@ds-iq.com>>
wrote:
Thanks Matt, the below is when I browsed to loca nifi url.  In the last working session of
nifi, I was configuring an ExecuteSql processor before it blue-d screen.  Could it have got
into a bad state?   How do I get out of it?



Thanks,
Keith





This is from nifi-app.log
2016-05-04 13:43:13,198 INFO [NiFi Web Server-69] c.s.j.s.i.application.WebApplicationImpl
Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
2016-05-04 13:43:13,839 WARN [NiFi Web Server-22] org.eclipse.jetty.servlet.ServletHandler
/nifi-api/controller/authorities
java.lang.NullPointerException: null
                at org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
~[nifi-web-security-0.6.0.jar:0.6.0]
                at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
~[nifi-web-security-0.6.0.jar:0.6.0]
                at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
                at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38)
~[classes/:na]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52) ~[classes/:na]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
[jetty-security-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
[jetty-io-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
                at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
2016-05-04 13:43:13,839 WARN [NiFi Web Server-21] org.eclipse.jetty.servlet.ServletHandler
/nifi-api/controller/identity
java.lang.NullPointerException: null
                at org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
~[nifi-web-security-0.6.0.jar:0.6.0]
                at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
~[nifi-web-security-0.6.0.jar:0.6.0]
                at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
                at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
                at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38)
~[classes/:na]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52) ~[classes/:na]
                at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
[jetty-security-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
[jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
[jetty-io-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
                at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
[jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
                at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]



This is from nifi-user.log
2016-05-04 13:43:13,774 INFO [NiFi Web Server-69] org.apache.nifi.web.filter.RequestLogger
Attempting request for (<no user found>) POST http://localhost:8080/nifi-api/access/kerberos
(source ip: 127.0.0.1)
2016-05-04 13:43:13,819 INFO [NiFi Web Server-69] o.a.n.w.a.c.IllegalStateExceptionMapper
java.lang.IllegalStateException: Access tokens are only issued over HTTPS.. Returning Conflict
response.





From: Matt Gilman [mailto:matt.c.gilman@gmail.com<mailto:matt.c.gilman@gmail.com>]
Sent: Wednesday, May 4, 2016 1:22 PM
To: users@nifi.apache.org<mailto:users@nifi.apache.org>
Subject: Re: Starting Nifi encounters "unexpected error"



Keith,



Were there any errors in any of the log's (nifi-app.log or nifi-user.log) before shutdown
or during start up. I'm not 100% sure what version your running but it appears it's having
trouble loading the Anonymous user. This user is created during startup, stored in the users
database, and should always be available. Any additional errors or details would be helpful.
Thanks.



Matt



On Wed, May 4, 2016 at 4:05 PM, Keith Lim <Keith.Lim@ds-iq.com<mailto:Keith.Lim@ds-iq.com>>
wrote:
My windows machine blue-d screen and after reboot I restarted nifi only to get the following
error when I browse to nifi (http://localhost:8080/nifi/)
I can do the long route or cleaning up the nifi folder and rebuilding it, but want to know
if there is a surgical way to remove this issue.



Thanks,
Keith



An unexpected error has occurred
<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 500 Server Error</title> </head> <body><h2>HTTP
ERROR 500</h2> <p>Problem accessing /nifi-api/controller/authorities. Reason:
<pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NullPointerException
at org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176) at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745)
</pre> <hr><i><small>Powered by Jetty://</small></i><hr/>
</body> </html>


Mime
View raw message