camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mcarson <mcar...@amsa.com>
Subject Re: handling large files
Date Fri, 16 Oct 2009 12:41:02 GMT

Using the scanner seems to work for parsing down the huge file based upon a
delimiter.  However it appears that either the JmsTemplate I'm using to send
messages or ActiveMQ cannot keep pace.

Somewhere between 250K - 500K sends, I get this stack trace:

Exception in thread "main"
org.springframework.jms.UncategorizedJmsException: Uncategorized exception
occured during JMS processing; nested exception is javax.jms.JMSException:
java.io.EOFException
	at
org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:308)
	at
org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474)
	at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:548)
	at asa.camel.TestScanner.parseRecord(TestScanner.java:62)
	at asa.camel.TestScanner.readFile(TestScanner.java:34)
	at asa.camel.TestScanner.main(TestScanner.java:82)
Caused by: javax.jms.JMSException: java.io.EOFException
	at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
	at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1244)
	at
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1339)
	at
org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:298)
	at
org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:462)
	... 4 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:375)
	at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269)
	at
org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:210)
	at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:202)
	at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
	at java.lang.Thread.run(Thread.java:619)

Any ideas what could cause this?


Claus Ibsen-2 wrote:
> 
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> How do you want to split the file?
>>>>>>> Is there a special character that denotes a new "record"
>>>>>>>
>>>>>>> Using java.util.Scanner is great as it can do streaming. And
also
>>>>>>> what
>>>>>>> Camel can do if you for example want to split by new line etc.
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
> 
> 

-- 
View this message in context: http://www.nabble.com/handling-large-files-tp25826380p25924781.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message