camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Monichev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CAMEL-9970) CamelFileLength header is wrong for long write file
Date Mon, 16 May 2016 13:33:13 GMT

     [ https://issues.apache.org/jira/browse/CAMEL-9970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Monichev updated CAMEL-9970:
-----------------------------------
    Description: 
Given a long write file slowfile.dat. Camel polls the file and stores its parameters (length,
lastModified...). Camel tries to get exclusive read lock and fails as file is writing by someone,
next time Camel gets lock. But file was changed and its parameters are wrong.
The solution is pretty simple - check the file length and lastModified when exclusive read
lock is granted and update them.

Here is a patch:
Index: camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
(revision 13b79316bc0590bc99e6886ca0ff95fe3b5fea86)
+++ camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
(revision )
@@ -50,6 +50,20 @@
             if (!lock) {
                 // do not begin since we could not get the exclusive read lock
                 return false;
+            } else {
+                // check current file length and lastModify
+                if (file.getFile() instanceof File) { // todo if not File what else?
+                    final long length = ((File) file.getFile()).length();
+                    final long lastModified = ((File) file.getFile()).lastModified();
+                    if (file.getFileLength() != length || file.getLastModified() != lastModified)
{
+                        // updating generic file length and lastModified
+                        file.setFileLength(length);
+                        file.setLastModified(lastModified);
+                        // updating headers
+                        exchange.getIn().setHeader(Exchange.FILE_LENGTH, length);
+                        exchange.getIn().setHeader(Exchange.FILE_LAST_MODIFIED, lastModified);
+                    }
+                }
             }
         }
 
Index: camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
(revision 13b79316bc0590bc99e6886ca0ff95fe3b5fea86)
+++ camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
(revision )
@@ -20,6 +20,7 @@
 import java.io.FileOutputStream;
 
 import org.apache.camel.ContextTestSupport;
+import org.apache.camel.Exchange;
 import org.apache.camel.builder.RouteBuilder;
 import org.apache.camel.component.mock.MockEndpoint;
 import org.slf4j.Logger;
@@ -39,10 +40,19 @@
         super.setUp();
     }
 
+    long expectedFileLength() {
+        long length = 0;
+        for (int i = 0; i < 20; i++) {
+            length += ("Line " + i + LS).getBytes().length;
+        }
+        return length;
+    }
+
     public void testChangedReadLock() throws Exception {
         MockEndpoint mock = getMockEndpoint("mock:result");
         mock.expectedMessageCount(1);
         mock.expectedFileExists("target/changed/out/slowfile.dat");
+        mock.expectedHeaderReceived(Exchange.FILE_LENGTH, expectedFileLength());
 
         writeSlowFile();
 

  was:
Given a long write file slowfile.dat. Camel polls the file and stores its parameters (length,
lastModified...). Camel tries to get exclusive read lock and fails as file is writing by someone,
next time Camel gets lock. But file was changed and its parameters are wrong.
The solution is pretty simple - check the file length and lastModified when exclusive read
lock is granted and update them.


> CamelFileLength header is wrong for long write file
> ---------------------------------------------------
>
>                 Key: CAMEL-9970
>                 URL: https://issues.apache.org/jira/browse/CAMEL-9970
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 2.16.3
>            Reporter: Sergey Monichev
>              Labels: patch
>
> Given a long write file slowfile.dat. Camel polls the file and stores its parameters
(length, lastModified...). Camel tries to get exclusive read lock and fails as file is writing
by someone, next time Camel gets lock. But file was changed and its parameters are wrong.
> The solution is pretty simple - check the file length and lastModified when exclusive
read lock is granted and update them.
> Here is a patch:
> Index: camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
> IDEA additional info:
> Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
> <+>UTF-8
> ===================================================================
> --- camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
(revision 13b79316bc0590bc99e6886ca0ff95fe3b5fea86)
> +++ camel-core/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
(revision )
> @@ -50,6 +50,20 @@
>              if (!lock) {
>                  // do not begin since we could not get the exclusive read lock
>                  return false;
> +            } else {
> +                // check current file length and lastModify
> +                if (file.getFile() instanceof File) { // todo if not File what else?
> +                    final long length = ((File) file.getFile()).length();
> +                    final long lastModified = ((File) file.getFile()).lastModified();
> +                    if (file.getFileLength() != length || file.getLastModified() !=
lastModified) {
> +                        // updating generic file length and lastModified
> +                        file.setFileLength(length);
> +                        file.setLastModified(lastModified);
> +                        // updating headers
> +                        exchange.getIn().setHeader(Exchange.FILE_LENGTH, length);
> +                        exchange.getIn().setHeader(Exchange.FILE_LAST_MODIFIED, lastModified);
> +                    }
> +                }
>              }
>          }
>  
> Index: camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
> IDEA additional info:
> Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
> <+>UTF-8
> ===================================================================
> --- camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
(revision 13b79316bc0590bc99e6886ca0ff95fe3b5fea86)
> +++ camel-core/src/test/java/org/apache/camel/component/file/strategy/FileChangedReadLockTest.java
(revision )
> @@ -20,6 +20,7 @@
>  import java.io.FileOutputStream;
>  
>  import org.apache.camel.ContextTestSupport;
> +import org.apache.camel.Exchange;
>  import org.apache.camel.builder.RouteBuilder;
>  import org.apache.camel.component.mock.MockEndpoint;
>  import org.slf4j.Logger;
> @@ -39,10 +40,19 @@
>          super.setUp();
>      }
>  
> +    long expectedFileLength() {
> +        long length = 0;
> +        for (int i = 0; i < 20; i++) {
> +            length += ("Line " + i + LS).getBytes().length;
> +        }
> +        return length;
> +    }
> +
>      public void testChangedReadLock() throws Exception {
>          MockEndpoint mock = getMockEndpoint("mock:result");
>          mock.expectedMessageCount(1);
>          mock.expectedFileExists("target/changed/out/slowfile.dat");
> +        mock.expectedHeaderReceived(Exchange.FILE_LENGTH, expectedFileLength());
>  
>          writeSlowFile();
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message