nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Caldwell <ddc02...@yahoo.com.INVALID>
Subject invokeHttp routing of exceptions like ConnectException and IOException to failure instead of retry
Date Thu, 31 Oct 2019 18:28:51 GMT
Hi,
While testing invokeHttp retry logic when the destination endpoint is offline, I learned that
invokeHttp processor routes exceptions caused by the offline endpoint to the failure relationship
instead of the retry relationship.
That surprised me since those types of errors are exactly what I would normally like to retry. 
What is the the reasoning for routing them to failure instead of retry?
Furthermore, when I routed the failure relationship back into invokeHttp to retry, I then
found that nifi cpu usage stays around 150-160% until the remote endpoint comes back online.
Additional digging showed that invokeHttp penalizes retry, no_retry & failure scenarios,
but yields only for retry and no_retry.  IOW, the failure scenario doesn't yield.

I don't really have a problem with failure not yielding.  That's just what I suspect may
be causing the excessive cpu utilization.  My real problem is that I think recoverable communications
exceptions should be routed to retry instead of failure.  Not only would that avoid developer
surprise, but it would include yield which I hope would prevent the high cpu utilization.
Reading the topic https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/html/developer-guide.html#penalization-vs-yielding,
the following points reinforced my thinking:   
   - Yield when processor won't be able to perform any useful function for some period of
time
   
   - This tells framework, don't waste resources triggering the processor to run, because
there's nothing it can do for a while
   - The topic actually uses a processor communicating with remote resource as the example
for yield
Thoughts?
David

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message