cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Kulp <dk...@apache.org>
Subject Re: Large messages and memory thoughts.....
Date Wed, 10 Oct 2007 21:43:07 GMT
On Wednesday 10 October 2007, Dan Diephouse wrote:
> Daniel Kulp wrote:
> > 1) Logging interceptors - right now, we log the entire message to
> > the Logger.   However, the logger requires a String, so the entire
> > message is converted to a String (keep in mind, a String is utf-16
> > so twice the memory is consumed) for logging.   Plus, do you really
> > think logging a 10MB message to the Logger is the best option?  I'm
> > going to add a configurable "limit" that will Log just the first
> > "limit" bytes of the message.   Should the default be -1 (no limit)
> > or something reasonable like 4K?      Also, we could have a
> > configuration to allow it to optionally log to a separate file
> > instead of the Logger.  We can then directly copy from stream to
> > stream with very little memory usage. That's going to be a lot
> > trickier though as we'll have to deal with concurrent access to the
> > file from multiple threads and such.
>
> Maybe a more reasonable limit would be 100 K and a warning message
> otherwise.

I've gone ahead and implemented that.   The default is 100K, but spring 
config can configure it to something else.

> > 2) JMS transport - the JMS transport always deals with the entire
> > message as either a String (for TextMessage) or byte[].   That seems
> > to be a limitation for standard JMS API's.   Some vendors (such as
> > ActiveMQ) do have proprietary extensions to allow some proprietary
> > stuff to help. Examples:
> > http://activemq.apache.org/blob-messages.html
> > http://activemq.apache.org/jms-streams.html
> > It might be good to allow some extra hooks into the JMS transport to
> > allow for easy subclassing to enable some of the hooks.   (That
> > said, I'd like to see the JMS transport also changed over to using
> > the Spring JMS configuration.)
>
> +1 on both accounts

I've created JIRA's for both:
https://issues.apache.org/jira/browse/CXF-1101
https://issues.apache.org/jira/browse/CXF-1102


> > 3) WS-RM - The RMMessage object holds the message as a byte[].  
> > Thus, it's entirely in memory until it's stored.  It would probably
> > be a good idea for this to be passed as the original
> > CachedOutputStream and stored directly to/from that stream
>
> Yeah, I suppose we could stream it to its storage location and then
> stream it back out later on. Never thought about that, but that makes
> perfect sense now that you mention it!
>
> - Dan

Created a JIRA for that as well:

https://issues.apache.org/jira/browse/CXF-1100


-- 
J. Daniel Kulp
Principal Engineer
IONA
P: 781-902-8727    C: 508-380-7194
daniel.kulp@iona.com
http://www.dankulp.com/blog

Mime
View raw message