camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wing-tung Leung <>
Subject From http4 via md5checksum to FTP: file cache or streaming ?
Date Fri, 12 Aug 2011 14:51:57 GMT

one of our routes pulls binary data from from a HTTP service using
http4, and then uploads the binary to a remote FTP directory. It also
uses temporary files on the local filesystem behind the scenes , which
makes sense because the binary image data can grow up till 10MB.

Now I want to add one extra step: MD5 checksum verification just after
the download. Because of the size, I prefer not to load all the data
into a byte array to calculate a simple checksum, and use a
InputStream instead. At first sight this seems to work. In the
debugger, I can see this input stream is a wrapper around the locally
cached file, and at the end of the function I return the original
input stream.

This is how the route currently looks like:
    <to uri="http4:/" />
    <to uri="bean:md5sum" />
    <to uri="ftp://{{attachment.ftp.location}}/?username={{attachment.ftp.user}}&amp;password={{attachment.ftp.password}}&amp;binary=true"/>

The processing bean method's signature:
    InputStream process(InputStream buffer,
@Header("mgws_file_md5sum") String expectedChecksum)

But now the route seems to hang. I assume returning the "used" input
stream is wrong, since the FTP component can't do anything useful with
this anymore. I basically see options to fix this:
1 - use fancy stream interception with a custom HttpBinder for http4,
integrating MD5 checksum calculation on the fly
2 - redirect to temporary file explicitly, start a new route for the
checksum, reuse same file for FTP upload and cleanup manually

Any other recommendations? I think this is a quite common use case, so
I guess more experienced Camel user may want to give some useful
advice to a novice like me ..

Some pages I have been looking at:

(using Camel 2.6)



View raw message