axis-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piergiuliano Bossi <P.Bo...@quinary.com>
Subject Re: unexpected attachment performance
Date Thu, 02 Dec 2004 16:52:14 GMT
After some more thoughts it looks that we are not doing buffered I/O 
while reading the attachment from disk. As suggested in many places (for 
example 
http://java.sun.com/docs/books/performance/1st_edition/html/JPIOPerformance.fm.html) 
I/O operations in java are by default unbuffered.

Shame on us! :(

Preliminary measures taken on development workstation show an order of 
magnitude of improvement.
Hopefully tomorrow we will get better results on servers and under load 
as well.

Cheers,
Giuliano

Piergiuliano Bossi wrote:

> Hello all,
>
> one year ago we have developed a web service that has a few 
> parameters, one of which is a byte[]. At that time this parameter was 
> used to get a binary file (for example midi or gif) with size up to 
> 150 KB (max). We have had excellent results, both in terms of 
> reliability and speed.
> Customer's needs have grown since then and today we are working with 
> files that can be 4 MB in size. Of course, we have looked at 
> attachments for this (check thread 
> http://marc.theaimsgroup.com/?l=axis-user&m=109707110706202&w=2).
> Now our web service looks for the binary file in the byte array 
> parameter first, and then into attachments. The idea is that if the 
> file is small (ie: 100 KB) then using the parameter is ok, otherwise 
> the attachment got to be used.
>
> The problem is that we observe unexpected performance in production. 
> We have built some jmeter load tests, trying to send 4 MB content with 
> or without using attachments, 5 or 10 concurrent client threads (no 
> ramp-up period) and we get the following results (please watch it at 
> fixed width):
> thread  Average Min   Max   Error Attachments
> 5       14889   13373 16205 0.00%     no
> 5       32170   31282 33438 0.00%     yes
> 10      31673   31547 31857 0.00%     no
> 10      45556   41563 51975 0.00%     yes
>
> These measures are "round-trip", taken from client-side.
>
> As you can see it seems that without attachments performances are 
> better! This contradicts everything we have read so far and even logic 
> too.
>
> Please, note that production architecture is structured in this way:
> *) there are 2 servers, each with Red Hat Linux rel 8.0 (kernel 
> 2.4.18-14smp), 4 microproc. Intel Xeon 550 MHz, 1 GB RAM
> *) each server is using a 1.4.2_06 jvm, tomcat 4.1.30 and axis 1.1
> *) in front of the servers there is an Alteon balancer
>
> Load tests were performed using a fast client machine, a Sun Solaris 
> 220, double-processor with 2 GB RAM, therefore we expect that the 
> problem doesn't lie on client side.
>
> Before implementing attachments we have profiled a single request with 
> a 4 MB file (passed as a byte[]) to our web service (with OptimizeIT) 
> and we have discovered that our business logic was consuming 19.2% of 
> cpu time: the remining part was consumed by axis in various ways, for 
> example retrieving the Envelope 
> (org.apache.axis.Message.getSOAPEnvelope) and getting parameters in 
> java objects (org.apache.axis.message.RPCElement.getParams)
>
> We have not profiled current implementation with attachments yet, but 
> we will do it soon. Let's see if there will be any surprise.
>
> Real problem is that we should be able to have a round-trip max period 
> well under 10 seconds under more than 10 concurrent threads on client 
> side.
>
> What do you think about it? How can we explain those numbers? Is there 
> any way to improve significantly these performance results?
>
> We are considering passing the file over the filesystem, getting path 
> only into the web service, but this is a last chance that we prefer to 
> avoid, because it assumes that client and server share a filesystem 
> somewhere.
>
> TIA
> Giuliano
>
>
> PS: a couple of snippets of code (client side is used in load tests)
>
> ********************CUT HERE - CLIENT SIDE********************
> /**This class should store all attachment data in memory */
> static class MemoryOnlyDataSource extends 
> org.apache.axis.attachments.ManagedMemoryDataSource{
>
>   MemoryOnlyDataSource( byte [] in, String contentType) throws 
> java.io.IOException{
>     super( new java.io.ByteArrayInputStream(in) , Integer.MAX_VALUE 
> -2, contentType, true);
>   }
>   MemoryOnlyDataSource( String in, String contentType)throws 
> java.io.IOException{
>     this( in.getBytes() ,  contentType);
>   }
> }
>
> service._setProperty(Call.ATTACHMENT_ENCAPSULATION_FORMAT,
>  Call.ATTACHMENT_ENCAPSULATION_FORMAT_DIME);
> service.setTimeout(60*60*1000);
>
> DataHandler attachment = new DataHandler(new 
> MemoryOnlyDataSource(aContentAsAttachment, mimeType));
> service.addAttachment(attachment);
>
> byte[] contentData = null;   // if null, then attachments are used
> service.insertContent(param1, param2, param3, param4, param5, 
> contentData, param6, param7, param8);
> ********************CUT HERE - CLIENT SIDE********************
>
> ********************CUT HERE - SERVER SIDE********************
> public int insertContent(String param1, String param2, String param3, 
> String param4, String param5, byte[] contentData, int param6, boolean 
> param7, StringHolder param8)
> {
>    if (contentData == null)
>    {
>        try
>        {
>            AttachmentPart messageAttachment = getmessageAttachment();
>            if (messageAttachment != null)
>                contentData = toByteArray(messageAttachment);
>        }
>        catch (Exception e)
>        {
>            // ...
>        }
>    }
>    // ... business logic here ...
> }
>
> private AttachmentPart getmessageAttachment() throws AxisFault
> {
>    MessageContext msgContext = MessageContext.getCurrentContext();
>    Message reqMsg = msgContext.getRequestmessage();
>    Attachments allAttachments = reqMsg.getAttachmentsImpl();
>
>    AttachmentPart attachment = null;
>    if (allAttachments == null)
>        // ...
>    else if (allAttachments.getAttachmentCount() > 0)
>        attachment = 
> (AttachmentPart)allAttachments.getAttachments().iterator().next();
>         return attachment;
> } 
> private byte[] toByteArray(AttachmentPart attachment) throws 
> SOAPException, IOException
> {
>    ByteArrayOutputStream outputStream = new 
> ByteArrayOutputStream(attachment.getSize());
>    InputStream inputStream = (InputStream) attachment.getContent();
>    int currentByte = -1;
>    while ((currentByte = inputStream.read()) != -1)
>        outputStream.write(currentByte);
>
>    inputStream.close();
>    outputStream.close();
>
>    return outputStream.toByteArray();
> }  ********************CUT HERE - SERVER SIDE********************
>
>


Mime
View raw message