nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [nifi] davidvoit commented on pull request #4175: NIFI-7307 - Add User Metadata support to the Azure Blob Processors + LookupService
Date Thu, 30 Apr 2020 17:25:43 GMT

davidvoit commented on pull request #4175:
URL: https://github.com/apache/nifi/pull/4175#issuecomment-621993222


   We are using this code to deduplicate files which aren't arriving in the correct order.
   
   Old: Blob A Metadata MeasumentTime N --> New: Blob A Metadata MearumentTime N+1 (LookupKey
MearumentTime New > Old => overwrite blob)
   Old: Blob A Metadata MeasumentTime N+1 --> New: Blob A Metadata MearumentTime N (LookupKey
MearumentTime New < Old => drop flowfile)
   
   Thats our usecase, but I think there can be more of them. The most important thing is here
we don't want to fetch the file to get metadata and route here, because the blob file could
be huge and timeconsuming.
   
   Does this makes sense?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message