camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: handling large files
Date Wed, 14 Oct 2009 13:07:41 GMT
Hi

How do you want to split the file?
Is there a special character that denotes a new "record"

Using java.util.Scanner is great as it can do streaming. And also what
Camel can do if you for example want to split by new line etc.




On Wed, Oct 14, 2009 at 3:01 PM, mcarson <mcarson@amsa.com> wrote:
>
> I have to solve this exact same problem!  I am also trying to:
>   -- read an extremely large file from a file system
>   -- doing a split on that file
>   -- send each split record to a JMS queue
>
> However I am relatively new to Camel.. I do not understand how the stream
> component can help specify how much data to read before the split occurs...
> and when I tried to stream the file I got exceptions in my split because
> when streaming the document it is no longer well formed XML?
>
> Can someone help explain in more detail how using the stream component
> solves this problem?
>
>
> This is the camel context I was using successfully to perform these tasks
> for SMALL files:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <beans xmlns="http://www.springframework.org/schema/beans"
>       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>       xsi:schemaLocation="
>       http://www.springframework.org/schema/beans
>       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
>       http://camel.apache.org/schema/spring
>       http://camel.apache.org/schema/spring/camel-spring.xsd">
>
>  <bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
>    <property name="connectionFactory">
>        <bean class="org.apache.activemq.ActiveMQConnectionFactory">
>                <property name="brokerURL" value="tcp://localhost:61616"/>
>        </bean>
>    </property>
>  </bean>
>
>  <camelContext xmlns="http://camel.apache.org/schema/spring"
>        xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
>    <package>camel.myprototype</package>
>
>    <route>
>      <from uri="file:src/inbox?delete=true"/>
>      <split>
>        <xpath>/s:Envelope/s:Body/person</xpath>
>        <to uri="jms:queue:Q_People"/>
>      </split>
>    </route>
>  </camelContext>
> --
> View this message in context: http://www.nabble.com/handling-large-files-tp25826380p25890696.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message