camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen" <>
Subject Re: Mega mega (VFS) file handler
Date Thu, 08 Jan 2009 06:21:14 GMT

On Thu, Jan 8, 2009 at 7:01 AM, Ramon Buckland <> wrote:
> Hi All,
> (*back from christmas / NY  - trust you all had a great break).*
> I have been thninking over the christmas break about file polling
> component(s) and the use of VFS as it is in Servicemix.
> My core client requirement (which I realised now as a GAP) is that a dated
> ZIP and a dated TGZ file are delivered to a remove SFTP host daily.
> These files need to be collected (removed), and copied locally (to
> AMQ+camel) in upacked form.
> To handle the ZIp and the Tgz .. TrueZip does this well. Of course, we are
> aware of commons-vfs.
Yeah that is a missing piece in Camel. A lot of integration is file
based using FTP with all kind of zipped or not zipped files.
Would be nice to be able to zip/unzip directly from Camel instead of
must use script files.

> Now a simple (eg: camel-file-archive) component could work as follows ...
>    from uri=sftp://user@host
> :/some/folder/where/zip/and/tgz/exist?consumer.delay=100000&...
>      to uri=archive:?unpack=true
>      to uri=file:///some/folder/destination/endpoint
> Nice .. reversing it .. we could have the archive component act much like a
> "aggregation endpoint" putting files into a ZIP or TGZ.
>    from uri=sftp://user@host
> :/some/folder/where/files/exist?consumer.delay=100000&...
>      to
> uri=archive:?pack=true&filename=myarchive&correlationIdentifier=foo&...
>      to uri=file:///some/folder/destination/endpoint
> Okay, so seems logical .. now what if I we used commons-vfs instead, as the
> poller and "collector"?
> So commons-vfs allows us to access a ZIP (and jar) etc etc .. using
> commons-vfs (quickly in my head) makes a "file" polling ZIP unpacker look
> like our trusty filepoller. But it does more than that in VFS ..
> Some examples of accessing a ZIP and a Jar with VFS looks like ...
>   - jar:../lib/classes.jar!/META-INF/
>   - zip:http://somehost/downloads/
>   -!/nested.jar!/somedir
>   -!/nested.jar!/some%21dirS
> So using VFS opens up a whole (good) can of worms. Has anybody thought about
> a camel-commons-vfs component ?
> If so what did you envisage ? (My requirement at the beginning of the email
> can be achieved with commons-vfs if it has TrueZip Support added in.
Now that SMX has commons-vfs support it's also a candidate for a new
component in Camel as well.
It does have some nice support for reading all sorts of file. But
lacks the zip that is what I think is the most important feature.

> *
> I am ready to code this thing but of course there are so many paths to
> choose that my head spun with many ways to solve it (my requirement top of
> the email).
I have raised my voice on this ticket as well. Hope they will react,
so it will get into commons-vfs.
However it concerns me that they dont react to the requests from other
Apache projects (Synapse) and there is even code donated to get
started with.

We could also consider adding our own component for TrueZip support to
be able to read/write archives, if commons-vfs doesnt pick it up.

> cheers
> ramon


/Claus Ibsen
Apache Camel Committer

View raw message