camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Crickenberger (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CAMEL-250) Allow FileConsumer to check if a file is unchanged before processing it
Date Fri, 30 Nov 2007 04:26:26 GMT

     [ https://issues.apache.org/activemq/browse/CAMEL-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aaron Crickenberger updated CAMEL-250:
--------------------------------------

    Attachment:     (was: camel-core.unchangedFileConsumer.patch)

> Allow FileConsumer to check if a file is unchanged before processing it
> -----------------------------------------------------------------------
>
>                 Key: CAMEL-250
>                 URL: https://issues.apache.org/activemq/browse/CAMEL-250
>             Project: Apache Camel
>          Issue Type: Improvement
>            Reporter: Aaron Crickenberger
>
> The use case is a FileConsumer polling a directory that's populated by an FTP server.
 Currently, the FileConsumer is almost guaranteed to grab files that aren't complete.  Setting
the consumer delay to something large just reduces the likelihood, and requiring external
lock files or renaming (though potentially more atomic) seems like too many moving parts.
> In an attempt to combat this, I added an isUnchanged() method to FileConsumer that is
used similar to isMatched(), along with two properties, unchangedDelay and unchangedSize.
 Setting unchangedDelay to eg: 5000 will cause FileConsumer to avoid processing a file until
its last modified time is at least 5 seconds in the past.  For large files being copied across
a local filesystem, this doesn't always work, because the modified date may be preserved even
though the file's not done copying.  Thus, setting unchangedSize to true will cause FileConsumer
to avoid processing a file until its filesize remains unchanged across two successive polls.
> There's probably a more camel-like way to do this, maybe involving a polling strategy,
so that the technique could shared with RemoteFileConsumers, but that made my brain hurt.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message