Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Sun, 26 Jun 2016 04:30:57 -0700 (MST)
From: ddewaele <ddewaele@gmail.com>
To: dev@nifi.apache.org
Message-ID: <1466940657854-12278.post@n7.nabble.com>
Subject: ListS3 processor question (duplicate files / maintaining state)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
archived-at: Sun, 26 Jun 2016 12:40:58 -0000

Hi,

I had a question on the ListS3 processor.
I'm using it to monitor the content of an S3 bucket.
The idea is that when new files come in, they need to be processed and sent
through the dataflow, using a FetchS3Object to process the file. This all
works but I had 2 questions :

1. Where does the S3 processor keep its state ? How does it know what files
it has already processed and is there a way to clear this state ?
2. Sometimes, when syncing files to my S3 buckets, I notice that the ListS3
processor is picking up the same file twice. Is there a way to avoid that ?


--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ListS3-processor-question-duplicate-files-maintaining-state-tp12278.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.