axis-c-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manjula Peiris <manj...@wso2.com>
Subject Sending large attachments with Axis2/C with httpd
Date Wed, 23 Jan 2008 08:58:06 GMT
Hi devs,

While I am working on ,https://issues.apache.org/jira/browse/AXIS2C-862
i found with httpd it takes around 800MB of memory to process a 40MB
attachment. The reason is when processing the request with an optimized
binary data the entire SOAP message is parsed by mime_parse_parse()
method to separates the soap envelope and mime parts. Since the incoming
stream is read block by block, after processing a block the parser
reallocates the whole (previous + new) for the precessing of second
block. With simple axis server this is not a problem , because it uses
free () method to release the memory. But with httpd since we are
allocating memory from pool and releasing the pool is done by httpd the
memory usage grows, hence after 3 or 4 requests the system crashes due
to limited memory.

Actually the main problem here is httpd does not release the request
pool even after processing the request. So I create my own pool using my
own allocator and destroy both after processing the request. This cause
problems with Sandesha2/c becasue freeng allocator (here allocator means
apr_allocator not axutil_allocator)cause problems because Sandesha2/C
creates its own threads and those threads use the allocator.

Even the above solution does not prevent using larger amount of memory
to process large attachments. 

So I think we need to change the mime_parser to process each block
separately and concatenate them. The problem with this approach is the
search string(eg : MimeBoundary) may not contain in one buffer. I am
trying to come up with an algorithm for this problem.

Any thoughts?

Thanks,
-Manjula.

-- 
Manjula Peiris: http://manjula-peiris.blogspot.com/


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-c-dev-help@ws.apache.org


Mime
View raw message