cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Kulp <dk...@apache.org>
Subject Large messages and memory thoughts.....
Date Wed, 10 Oct 2007 14:08:18 GMT

I did quite a bit of work yesterday to fix some areas where large 
messages were causing problems.   They all pretty much centered around 
mis-use of the CachedOutputStream (assuming it's always a 
ByteArrayOutputStream which isn't true once it flips to a file).   
However, while digging through those, I've noticed a few other potential 
problems that I'd like to discuss.

1) Logging interceptors - right now, we log the entire message to the 
Logger.   However, the logger requires a String, so the entire message 
is converted to a String (keep in mind, a String is utf-16 so twice the 
memory is consumed) for logging.   Plus, do you really think logging a 
10MB message to the Logger is the best option?  I'm going to add a 
configurable "limit" that will Log just the first "limit" bytes of the 
message.   Should the default be -1 (no limit) or something reasonable 
like 4K?      Also, we could have a configuration to allow it to 
optionally log to a separate file instead of the Logger.  We can then 
directly copy from stream to stream with very little memory usage.  
That's going to be a lot trickier though as we'll have to deal with 
concurrent access to the file from multiple threads and such. 

2) JMS transport - the JMS transport always deals with the entire message 
as either a String (for TextMessage) or byte[].   That seems to be a 
limitation for standard JMS API's.   Some vendors (such as ActiveMQ) do 
have proprietary extensions to allow some proprietary stuff to help.  
Examples:
http://activemq.apache.org/blob-messages.html
http://activemq.apache.org/jms-streams.html
It might be good to allow some extra hooks into the JMS transport to 
allow for easy subclassing to enable some of the hooks.   (That said, 
I'd like to see the JMS transport also changed over to using the Spring 
JMS configuration.)

3) WS-RM - The RMMessage object holds the message as a byte[].   Thus, 
it's entirely in memory until it's stored.  It would probably be a good 
idea for this to be passed as the original CachedOutputStream and stored 
directly to/from that stream.

Thoughts?

-- 
J. Daniel Kulp
Principal Engineer
IONA
P: 781-902-8727    C: 508-380-7194
daniel.kulp@iona.com
http://www.dankulp.com/blog

Mime
View raw message