camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vjintegrates <vijaypawnar...@gmail.com>
Subject Re: Identifying and processing changed file when noop=true
Date Thu, 04 Feb 2010 12:22:52 GMT

Tracing functionality was helpful.   "readLock=changed" moves the file to
.camel/ dir, in our case we were trying keep the file in same location.  To
avoid the move I used noop=true, and idempotent=false/true. 

I saw output from following log statement even after setting the idempotent
flag. That tells me setting idempotent did not work. Shouldn't the
idempotent flag be respected ?

org/apache/camel/component/file/FileEndpoint.java

        // if noop=true then idempotent should also be configured
        if (isNoop() && !isIdempotentSet()) {
            log.info("Endpoint is configured with noop=true so forcing
endpoint to be idempotent as well");
            setIdempotent(true);
        }

Use case is
1) process the file without moving it -
2) process the file only if it is modified -
3) it is okay to process same file again -




Claus Ibsen-2 wrote:
> 
> On Sat, Jan 30, 2010 at 6:40 AM, vjintegrates <vijaypawnarkar@gmail.com>
> wrote:
>>
>> I had tried this option it does not result in file being processed based
>> on
>> timestamp.
>>
>> readLock=changed
>>
>> I believe this attribute is used to to check if the file can be locked or
>> not depending on last modified timestamp.
>>
>> Changed file were processed automatically in Camel 1.x, once I upgraded
>> to
>> 2.x it stopped working, therefore I decided to provide the implementation
>> of
>> IdempotentRepository
>>
> 
> readLock=change works in similar way as Camel 1.x file component.
> It uses both file length and timestamp to detect whether or not the
> file "changes" over time.
> 
> You can enable TRACE logging at
> org.apache.camel.component.file.strategy to see how it detects this.
> 
> The idea is that in 2.x to move or delete the processed file
> AFTERWARDS it has been processed to avoid reading it again.
> In 1.x this was not the case as it has this internal idempotent repo
> as well. If you really want to use that you can do it in 2.x as well.
> Beware that if you restart the server how are you going to know that
> you have already processed the file before?
> 
> 
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> Hi
>>>
>>> Have you tried the readLock=changed option?
>>> http://camel.apache.org/file2.html
>>>
>>>
>>> On Thu, Jan 28, 2010 at 4:01 PM, vjintegrates <vijaypawnarkar@gmail.com>
>>> wrote:
>>>>
>>>> I have added following implementation of IdempotentRepository to
>>>> process
>>>> a
>>>> specific file from directory  when the file is modified. This is using
>>>> Camel
>>>> 2.x, Is there an alternative approach ?
>>>>
>>>> <from
>>>> uri="file:src/data?noop=true&amp;idempotentRepository=#fileChanged&amp;delay=300000&amp;fileName=myfile.txt"/>
>>>>
>>>> <bean id="fileChanged" class="mypkg.FileChangedRepository">
>>>>        <property name="fileDir" value="src/data" />
>>>> </bean>
>>>>
>>>> public class FileChangedRepository implements
>>>> IdempotentRepository<String>{
>>>>
>>>>        private String fileDir;
>>>>        private long lastModified =0;
>>>>
>>>>        @Override
>>>>        public boolean add(String arg0) {
>>>>                return false;
>>>>        }
>>>>
>>>>        @Override
>>>>        public boolean confirm(String arg0) {
>>>>                return true;
>>>>        }
>>>>
>>>>        @Override
>>>>        public boolean contains(String arg0) {
>>>>                synchronized(this) {
>>>>                File file = new File(fileDir + File.separator + arg0);
>>>>
>>>>                if (file.lastModified() > lastModified) {
>>>>                        lastModified = file.lastModified();
>>>>                        return false;
>>>>                }
>>>>                return true;
>>>>               }
>>>>        }
>>>>
>>>>        @Override
>>>>        public boolean remove(String arg0) {
>>>>                return false;
>>>>        }
>>>>
>>>>        public void setFileDir(String fileDir) {
>>>>                this.fileDir = fileDir;
>>>>        }
>>>>
>>>>        public String getFileDir() {
>>>>                return fileDir;
>>>>        }
>>>> }
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27357357.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27381562.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> Claus Ibsen
> Apache Camel Committer
> 
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
> 
> 

-- 
View this message in context: http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27451896.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message