camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Magnus Palmér <magnus.palmer.w...@gmail.com>
Subject Re: From http4 via md5checksum to FTP: file cache or streaming ?
Date Fri, 12 Aug 2011 15:10:07 GMT
I would make your md5bean return a file. 
Not sure if the current Jira issue of not cleaning (deleting) the file until JVM stops will
apply for you then or not.

Don't have the link to it right now.

-- 
Magnus Palmér
+46 736 845680

12 aug 2011 kl. 16:51 skrev wing-tung Leung <wingtung.leung@gmail.com>:

> Hello,
> 
> one of our routes pulls binary data from from a HTTP service using
> http4, and then uploads the binary to a remote FTP directory. It also
> uses temporary files on the local filesystem behind the scenes , which
> makes sense because the binary image data can grow up till 10MB.
> 
> Now I want to add one extra step: MD5 checksum verification just after
> the download. Because of the size, I prefer not to load all the data
> into a byte array to calculate a simple checksum, and use a
> InputStream instead. At first sight this seems to work. In the
> debugger, I can see this input stream is a wrapper around the locally
> cached file, and at the end of the function I return the original
> input stream.
> 
> This is how the route currently looks like:
>    <to uri="http4:/" />
>    <to uri="bean:md5sum" />
>    <to uri="ftp://{{attachment.ftp.location}}/?username={{attachment.ftp.user}}&amp;password={{attachment.ftp.password}}&amp;binary=true"/>
> 
> The processing bean method's signature:
>    InputStream process(InputStream buffer,
> @Header("mgws_file_md5sum") String expectedChecksum)
> 
> 
> But now the route seems to hang. I assume returning the "used" input
> stream is wrong, since the FTP component can't do anything useful with
> this anymore. I basically see options to fix this:
> 1 - use fancy stream interception with a custom HttpBinder for http4,
> integrating MD5 checksum calculation on the fly
> 2 - redirect to temporary file explicitly, start a new route for the
> checksum, reuse same file for FTP upload and cleanup manually
> 
> 
> 
> Any other recommendations? I think this is a quite common use case, so
> I guess more experienced Camel user may want to give some useful
> advice to a novice like me ..
> 
> Some pages I have been looking at:
> http://camel.apache.org/http4.html
> http://camel.apache.org/file2.html
> http://camel.apache.org/stream-caching.html
> 
> (using Camel 2.6)
> 
> Thanks!
> 
> Tung

Mime
View raw message