camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <cyrille.eloun...@barclays.com>
Subject RE: Memory issue while using the spliter EIP
Date Mon, 18 Nov 2013 17:48:45 GMT
Thank you for your help Claus!
Apache camel is a wonderful framework.

Best Wishest!
Cyrille Eloundou
Barclays-Global Technology
Java Solution architect
work:201 499 2921
cell: 646 522 1567
cell private: 347 978 5022



-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com] 
Sent: Monday, November 18, 2013 11:38 AM
To: users@camel.apache.org
Subject: Re: Memory issue while using the spliter EIP

Hi

I am not sure but maybe the fetchXXX options is only hints to the JDBC driver. As it may return
the full data instead.

So if you have millions of rows in the database table you get them all.

You may write some SQL that is SQL server specific to tell it to only return first X rows.
Each RDBMS usually have some custom SQL for that.

Oracle has rownum etc.


On Mon, Nov 18, 2013 at 5:02 PM,  <cyrille.eloundou@barclays.com> wrote:
> Hi,
> I have a memory issue with the splitter.
> I have a big table with millions of records in a sql server database 2008.
>
> What I'm trying to do is :
>
>                                 <setBody>
>                                                 <simple>select * from  myBigTable</simple>
>                                 </setBody>
>                                 <to uri="jdbc:myDataSource />
>                                 <split streaming="true" stopOnException="true">
>                                                 <simple>${body}</simple>
>                                                 <log message="Split line : ${body}"
/>
>                                 </split>
>
>
> I'm getting an out of memory issue.
> I tried to use the property statement.fetchSize=10&amp;statement.fetchDirection=1000
 in the DataSource Bean configuration. However it's still not working. It seems that the fetchsize
is not set by the  apache jdbc component.
>
> Best Wishest!
> Cyrille Eloundou
> Barclays-Global Technology
> Java Solution architect
> work:201 499 2921
> cell: 646 522 1567
> cell private: 347 978 5022
>
>
>
> _______________________________________________
>
> This message is for information purposes only, it is not a recommendation, advice, offer
or solicitation to buy or sell a product or service nor an official confirmation of any transaction.
It is directed at persons who are professionals and is not intended for retail customer use.
Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
>
> For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding
market commentary from Barclays Sales and/or Trading, who are active market participants;
and in respect of Barclays Research, including disclosures relating to specific issuers, please
see http://publicresearch.barclays.com.
>
> _______________________________________________



--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
_______________________________________________

This message is for information purposes only, it is not a recommendation, advice, offer or
solicitation to buy or sell a product or service nor an official confirmation of any transaction.
It is directed at persons who are professionals and is not intended for retail customer use.
Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.

For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding
market commentary from Barclays Sales and/or Trading, who are active market participants;
and in respect of Barclays Research, including disclosures relating to specific issuers, please
see http://publicresearch.barclays.com.

_______________________________________________

Mime
View raw message