camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Straube <claus.stra...@catify.com>
Subject Re: Writing big files : stream or file ?
Date Fri, 02 May 2014 04:54:28 GMT
Hi,

have a look at 
http://www.catify.com/2012/07/09/parsing-large-files-with-apache-camel/ 
- perhaps this helps (it's also about writing big files).

Best regards - Claus

On 01.05.2014 17:17, Ephemeris Lappis wrote:
> Hello.
>
> We have to produce some rather big volumes of data and generate output
> files, in several steps, some of these steps using splitting to process
> inputs. I've been looking for two ways...
>
> The first way is writing a file, setting an "append" mode, that could be a
> nice solution to write batches of lines, but I suppose that the file if
> closed and reopened for each exchange, and thus may be a quite bad solution
> when writing millions of line one by one from a splitting loop.
>
> The second way is to use a stream, using "stream:file", but I've not found
> any way to control actually how the file is closed. FYI, we use Camel with
> Service Mix with an emdedded 2.10, and the "closeOnDone" option is not
> available. The "autoCloseCount" seemed to be the beginning of a solution,
> but as the value can't be set dynamically, using a property for example,
> this doesn't give a full control on the file closing, and outputs might stay
> opened, and make fail the following tasks.
>
> Any idea to write big files and control how to close them ?
>
> Thanks in advance.
>
> Regards.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Writing-big-files-stream-or-file-tp5750742.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Mime
View raw message