Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: pass (athena.apache.org: domain of claus.ibsen@gmail.com
 designates 209.85.223.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1362053025196-5728292.post@n5.nabble.com>
References: <1361481036524-5727977.post@n5.nabble.com>
 <5127A03B.7060900@gmail.com>
 <1361567095301-5728045.post@n5.nabble.com> <5127E163.5030609@gmail.com>
 <1362053025196-5728292.post@n5.nabble.com>
From: Claus Ibsen <claus.ibsen@gmail.com>
Date: Thu, 28 Feb 2013 14:17:54 +0100
Message-ID: 
 <CAGB5yNkaysdOh-J7aRLnVz7J1QssVBax2V-Pn4NrTxXLi0AhNQ@mail.gmail.com>
Subject: Re: Large file processing with Apache Camel
To: users@camel.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Feb 28, 2013 at 1:03 PM, cristisor <cristisor_ac@yahoo.com> wrote:
> After digging more into my problem I found that the slow db access was the
> main issue, maybe you heard before of setting
> sendStringParametersAsUnicode=false in the jdbc driver to dramatically
> increase the performance.
>
> Since the last time I posted here I learned a lot about apache camel and I
> implemented some nice changes, but I'm confronting with a synchronization
> problem right now. I split each line that comes from the file and I enable
> parallel processing:
> from().split(body(String.class).tokenize("\n")).parallelProcessing().process(processor).to()
>
> Inside this processor I take the line, send it to the xml mapper as an input
> stream and in the end I set the xml as the body of the out message. The
> processor's process method is not synchronized and I run into situations
> where the input line has a code "1234" and the output doesn't contain the
> code "1234" but "1235", but this code belongs to another line 100% sure. So
> I should control the synchronization process by myself, right?
>
>

If you run in parallel processing mode, then the lines can be
processed out of order and concurrently.
Also your custom processor will be invoked concurrently by multiple
threads as they process the lines in parallel. So you have to write
your code in a thread safe manner, if you want to access shared
resources.


>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Large-file-processing-with-Apache-Camel-tp5727977p5728292.html
> Sent from the Camel - Users mailing list archive at Nabble.com.


-- 
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen