camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: File2: readLock/ready file alternatives
Date Tue, 23 Jun 2015 13:32:49 GMT
Hi

Maybe try with longer timeout values.

You can enable DEBUG/TRACE logging on
org.apache.camel.component.file.strategy to see what it logs

The code is
https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/component/file/strategy/FileChangedExclusiveReadLockStrategy.java

On Tue, Jun 23, 2015 at 3:06 PM, jamesburn <James.Burn@oup.com> wrote:
> Hi
>
> Thanks Claus for your prompt response!
>
> I did try readLock=changed first:
> <from uri="file:/mnt/DEVELOPMENT?delete=true&amp;localWorkDirectory=/tmp&amp;include=.*&amp;readLock=changed&amp;readLockTimeout=20000&amp;readLockCheckInterval=7000"/>
>
> but am still getting incomplete files delivered (mostly).
>
> I can see a zero byte .camelLock file written whilst the file is being collected.  Can
I monitor what is actually happening with readLock whilst it is working?
>
> Cheers
>
> James
>
> From: Claus Ibsen-2 [via Camel] [mailto:ml-node+s465427n5768500h34@n5.nabble.com]
> Sent: 23 June 2015 13:46
> To: BURN, James
> Subject: Re: File2: readLock/ready file alternatives
>
> Hi
>
> There is a readLock=change that check for file size / date
> modifications over a time window
>
> On Tue, Jun 23, 2015 at 2:25 PM, BURN, James <[hidden email]</user/SendEmail.jtp?type=node&node=5768500&i=0>>
wrote:
>
>> Hi
>>
>> I'm using Camel 2.13.2 on a Linux VM to collect/process files on a Windows server.
>>
>> This is done with a Linux filemount onto a folder on the Windows server.
>>
>> The issue we're getting is large (37Mb) files are often collected incomplete.
>>
>> I tried using the readLock, as per:
>> <from uri="file:/mnt/DEVELOPMENT?delete=true&amp;localWorkDirectory=/tmp&amp;include=.*&amp;readLock=rename&amp;readLockTimeout=20000&amp;readLockCheckInterval=7000"/>
>>
>> however, this didn't work and I can understand that perhaps the Windows filelock
methods are not supported (as per readLock notes in http://camel.apache.org/file2.html)
>>
>> I also tried fileLock=rename, but again ended up with an incomplete file.
>>
>> Getting a "ready" file written after the data file isn't an option as we have no
access to the script which writes the data files.
>>
>> Are there any other tricks we can use here. I'm wondering (and this is what I thought
readLock did when I first found it) whether my Camel route can poll any new files for xx seconds
to see if a file has stopped increasing in size. If so then it is collected. Is this what
exclusiveReadLockStrategy would allow? If so, how to I set this up?
>>
>> Thoughts most welcome.
>>
>> Thanks
>>
>> James
>>
>>
>> Oxford University Press (UK) Disclaimer
>>
>> This message is confidential. You should not copy it or disclose its contents to
anyone. You may use and apply the information for the intended purpose only. OUP does not
accept legal responsibility for the contents of this message. Any views or opinions presented
are those of the author only and not of OUP. If this email has come to you in error, please
delete it, along with any attachments. Please note that OUP may intercept incoming and outgoing
email communications.
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: [hidden email]</user/SendEmail.jtp?type=node&node=5768500&i=1>
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> hawtio: http://hawt.io/
> fabric8: http://fabric8.io/
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://camel.465427.n5.nabble.com/File2-readLock-ready-file-alternatives-tp5768499p5768500.html
> To start a new topic under Camel - Users, email ml-node+s465427n465428h2@n5.nabble.com<mailto:ml-node+s465427n465428h2@n5.nabble.com>
> To unsubscribe from Camel - Users, click here<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=SmFtZXMuQnVybkBvdXAuY29tfDQ2NTQyOHw5Njc4MTEwNDY=>.
> NAML<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> Oxford University Press (UK) Disclaimer
>
> This message is confidential. You should not copy it or disclose its contents to anyone.
You may use and apply the information for the intended purpose only. OUP does not accept legal
responsibility for the contents of this message. Any views or opinions presented are those
of the author only and not of OUP. If this email has come to you in error, please delete it,
along with any attachments. Please note that OUP may intercept incoming and outgoing email
communications.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File2-readLock-ready-file-alternatives-tp5768499p5768502.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Mime
View raw message