nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <>
Subject [jira] [Commented] (NIFI-238) Add processors to write datasets using Kite
Date Tue, 13 Jan 2015 21:46:35 GMT


Ryan Blue commented on NIFI-238:

bq. Can you explain what you mean by "I'm not clear on why I would use some method calls?"

This was just to lead into the next few questions, like what yield is doing. You're addressing
it with your other comments. :)

bq. context.yield() is a way to indicate that there's nothing useful that a Processor can

So I might use this if, for example, I were reading asynchronously and got 0 bytes. Yield
the time to another process and pick up where I left off reading the stream? That makes sense.

Why do I use it in the final {{catch (Throwable e)}} block in the Kite processor? It makes
sense that I want to avoid running continuously on the same file when it causes problems.
But what is the right way to do that? It seems like I could:
1. Penalize the file (if I think might be a file problem)
2. Yield to ensure I wait at least 1 second
3. Do nothing and count on the framework detecting a rollback (or several in a row) and taking
action. It sounds like this happens when the exception isn't a {{ProcessingException}}, so
I don't really need to do this in my code?

Thanks for the info on backpressure. I'll read that and probably ask more questions.

> Add processors to write datasets using Kite
> -------------------------------------------
>                 Key: NIFI-238
>                 URL:
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Ryan Blue
> I think it would be great to have a set of processors that parse incoming flow files
and add the data to Kite datasets.

This message was sent by Atlassian JIRA

View raw message