camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Identifying and processing changed file when noop=true
Date Thu, 04 Feb 2010 12:28:32 GMT
On Thu, Feb 4, 2010 at 1:22 PM, vjintegrates <vijaypawnarkar@gmail.com> wrote:
>
> Tracing functionality was helpful.   "readLock=changed" moves the file to
> .camel/ dir, in our case we were trying keep the file in same location.  To
> avoid the move I used noop=true, and idempotent=false/true.
>
> I saw output from following log statement even after setting the idempotent
> flag. That tells me setting idempotent did not work. Shouldn't the
> idempotent flag be respected ?
>
> org/apache/camel/component/file/FileEndpoint.java
>
>        // if noop=true then idempotent should also be configured
>        if (isNoop() && !isIdempotentSet()) {
>            log.info("Endpoint is configured with noop=true so forcing
> endpoint to be idempotent as well");
>            setIdempotent(true);
>        }
>
> Use case is
> 1) process the file without moving it -
> 2) process the file only if it is modified -
> 3) it is okay to process same file again -
>
>

Can you post
- Camel version used
- Full route, especially endpoint URI

And try creating the route in Java DSL. I suspect XML may play trick
on idempotentSet


>
>
> Claus Ibsen-2 wrote:
>>
>> On Sat, Jan 30, 2010 at 6:40 AM, vjintegrates <vijaypawnarkar@gmail.com>
>> wrote:
>>>
>>> I had tried this option it does not result in file being processed based
>>> on
>>> timestamp.
>>>
>>> readLock=changed
>>>
>>> I believe this attribute is used to to check if the file can be locked or
>>> not depending on last modified timestamp.
>>>
>>> Changed file were processed automatically in Camel 1.x, once I upgraded
>>> to
>>> 2.x it stopped working, therefore I decided to provide the implementation
>>> of
>>> IdempotentRepository
>>>
>>
>> readLock=change works in similar way as Camel 1.x file component.
>> It uses both file length and timestamp to detect whether or not the
>> file "changes" over time.
>>
>> You can enable TRACE logging at
>> org.apache.camel.component.file.strategy to see how it detects this.
>>
>> The idea is that in 2.x to move or delete the processed file
>> AFTERWARDS it has been processed to avoid reading it again.
>> In 1.x this was not the case as it has this internal idempotent repo
>> as well. If you really want to use that you can do it in 2.x as well.
>> Beware that if you restart the server how are you going to know that
>> you have already processed the file before?
>>
>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> Have you tried the readLock=changed option?
>>>> http://camel.apache.org/file2.html
>>>>
>>>>
>>>> On Thu, Jan 28, 2010 at 4:01 PM, vjintegrates <vijaypawnarkar@gmail.com>
>>>> wrote:
>>>>>
>>>>> I have added following implementation of IdempotentRepository to
>>>>> process
>>>>> a
>>>>> specific file from directory  when the file is modified. This is using
>>>>> Camel
>>>>> 2.x, Is there an alternative approach ?
>>>>>
>>>>> <from
>>>>> uri="file:src/data?noop=true&amp;idempotentRepository=#fileChanged&amp;delay=300000&amp;fileName=myfile.txt"/>
>>>>>
>>>>> <bean id="fileChanged" class="mypkg.FileChangedRepository">
>>>>>        <property name="fileDir" value="src/data" />
>>>>> </bean>
>>>>>
>>>>> public class FileChangedRepository implements
>>>>> IdempotentRepository<String>{
>>>>>
>>>>>        private String fileDir;
>>>>>        private long lastModified =0;
>>>>>
>>>>>        @Override
>>>>>        public boolean add(String arg0) {
>>>>>                return false;
>>>>>        }
>>>>>
>>>>>        @Override
>>>>>        public boolean confirm(String arg0) {
>>>>>                return true;
>>>>>        }
>>>>>
>>>>>        @Override
>>>>>        public boolean contains(String arg0) {
>>>>>                synchronized(this) {
>>>>>                File file = new File(fileDir + File.separator
+ arg0);
>>>>>
>>>>>                if (file.lastModified() > lastModified) {
>>>>>                        lastModified = file.lastModified();
>>>>>                        return false;
>>>>>                }
>>>>>                return true;
>>>>>               }
>>>>>        }
>>>>>
>>>>>        @Override
>>>>>        public boolean remove(String arg0) {
>>>>>                return false;
>>>>>        }
>>>>>
>>>>>        public void setFileDir(String fileDir) {
>>>>>                this.fileDir = fileDir;
>>>>>        }
>>>>>
>>>>>        public String getFileDir() {
>>>>>                return fileDir;
>>>>>        }
>>>>> }
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27357357.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27381562.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27451896.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message