camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vjintegrates <vijaypawnar...@gmail.com>
Subject Re: Identifying and processing changed file when noop=true
Date Thu, 11 Feb 2010 18:37:52 GMT

I tried both XML and JavaDSL, results were consistent with XML and JavaDSL
XML 

    <route>    	
      <from
uri="file:src/data?noop=true&amp;idempotent=false&amp;readLock=changed&amp;delay=60000&amp;fileName=Myfile.txt"/>
      <to uri="myPOJO"/>
    </route>
    
Java DSL

       CamelContext context = new DefaultCamelContext();
       context.addRoutes(new RouteBuilder() {
           public void configure() {
	         
from("file:src/data?noop=true&idempotent=false&readLock=changed&delay=60000&fileName=Myfile.txt")
	          	.process(new Processor(){
				@Override
			      public void process(Exchange arg0)
                                   	throws Exception {
					// TODO Auto-generated method stub
					logger.debug("Got the new file" + arg0.getIn().toString());
				}								
	               	});
	           }
        });
        context.start();

With Camel 2.0.0 
----------------------------------------------------
Result was the file was processed only first time and was never processed
whether changed or not. 

Upgraded and built it from Camel source 2.2-snapshot
----------------------------------------------------
Result was the File was processed every time whether changed or not. This
seems to be correct behavior when idempotent=false, I would say it was a bug
in Camel 2.0.0

I went through the source code to discover that "readLock" property
determines how file should be LOCKED, it does not determine whether file
should be processed or not. That decision to process file is done by what
are the values of "noop", "idempotent" flags along with
FileIdempotentRepository. Since FileIdempotentRepository considers only
filename/path to determine where the file was porcessed before or not.

In Use case is

 1) process the file without moving it -
 2) process the file only if it is modified -
 3) it is okay to process same file again during system restarts.

Providing an alternate implementation of  IdempotentRepository seem to
provide solution.

 <from

uri="file:src/data?noop=true&idempotentRepository=#fileChanged&delay=300000&fileName=myfile.txt"/>

 <bean id="fileChanged" class="mypkg.FileChangedRepository">
        <property name="fileDir" value="src/data" />
 </bean>

 public class FileChangedRepository implements
 IdempotentRepository<String>{

        private String fileDir;
        private long lastModified =0;

        @Override
        public boolean add(String arg0) {
                return false;
        }

        @Override
        public boolean confirm(String arg0) {
                return true;
        }

        @Override
        public boolean contains(String arg0) {
                synchronized(this) {
                File file = new File(fileDir + File.separator + arg0);

                if (file.lastModified() > lastModified) {
                        lastModified = file.lastModified();
                        return false;
                }
                return true;
               }
        }

        @Override
        public boolean remove(String arg0) {
                return false;
        }

        public void setFileDir(String fileDir) {
                this.fileDir = fileDir;
        }

        public String getFileDir() {
                return fileDir;
        }
 } 


Claus Ibsen-2 wrote:
> 
> On Thu, Feb 4, 2010 at 1:22 PM, vjintegrates <vijaypawnarkar@gmail.com>
> wrote:
>>
>> Tracing functionality was helpful.   "readLock=changed" moves the file to
>> .camel/ dir, in our case we were trying keep the file in same location.
>>  To
>> avoid the move I used noop=true, and idempotent=false/true.
>>
>> I saw output from following log statement even after setting the
>> idempotent
>> flag. That tells me setting idempotent did not work. Shouldn't the
>> idempotent flag be respected ?
>>
>> org/apache/camel/component/file/FileEndpoint.java
>>
>>        // if noop=true then idempotent should also be configured
>>        if (isNoop() && !isIdempotentSet()) {
>>            log.info("Endpoint is configured with noop=true so forcing
>> endpoint to be idempotent as well");
>>            setIdempotent(true);
>>        }
>>
>> Use case is
>> 1) process the file without moving it -
>> 2) process the file only if it is modified -
>> 3) it is okay to process same file again -
>>
>>
> 
> Can you post
> - Camel version used
> - Full route, especially endpoint URI
> 
> And try creating the route in Java DSL. I suspect XML may play trick
> on idempotentSet
> 
> 
>>
>>
> 

-- 
View this message in context: http://old.nabble.com/Identifying-and-processing-changed-file-when-noop%3Dtrue-tp27357357p27552227.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message