nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ddewaele <ddewa...@gmail.com>
Subject ListS3 processor question (duplicate files / maintaining state)
Date Sun, 26 Jun 2016 11:30:57 GMT
Hi,

I had a question on the ListS3 processor.
I'm using it to monitor the content of an S3 bucket.
The idea is that when new files come in, they need to be processed and sent
through the dataflow, using a FetchS3Object to process the file. This all
works but I had 2 questions :

1. Where does the S3 processor keep its state ? How does it know what files
it has already processed and is there a way to clear this state ?
2. Sometimes, when syncing files to my S3 buckets, I notice that the ListS3
processor is picking up the same file twice. Is there a way to avoid that ?





--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ListS3-processor-question-duplicate-files-maintaining-state-tp12278.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message