axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Veithen <andreas.veit...@gmail.com>
Subject Re: Introducing a new feature to enable MTOM attachment streaming
Date Mon, 25 Jul 2011 18:33:27 GMT
Some time ago I was thinking about this issue. It is highly non
trivial to solve. Consider the following variant of your example:

     public void useBinaryData(DataHandler dh1, DataHandler dh2) {
         try {
             OutputStream out = new FileOutputStream(new File("output1.zip"));
             dh1.writeTo(out);
             out = new FileOutputStream(new File("output2.zip"));
             dh2.writeTo(out);
         } catch (IOException e) {
             throw new RuntimeException(e);
         }
     }

Taking into account that the client is not required to send the MIME
parts in any particular order and that the service is not required to
consume them in any particular order, you will always end up with
scenarios where you have no other choice than to buffer some of the
MIME parts. Whether streaming is possible or not depends on how the
MIME parts are accessed, but you always need to support buffering if
necessary.

What this example also shows is that you need to defer the calls to
Attachments#getDataHandler(String) until the very last moment. When
working with an Axiom tree, this is already the case: the call to
Attachments#getDataHandler(String) occurs when OMText#getDataHandler()
is called (and not when the OMText node is built). However, when using
a data binding, one would have to create some sort of DataHandler
proxy that defers the call to Attachments#getDataHandler(String).

Andreas

On Mon, Jul 25, 2011 at 09:21, Sadeep Jayasumana <gayansadeep@gmail.com> wrote:
> Hi Devs,
> Here is the benefit of this feature from Axis2's perspective.
>
> Currently, when I use Axis2 to deploy a service class as follows,
> public class MTOMService {
>     public void useBinaryData(String username, DataHandler dataHandler) {
>         try {
>             System.out.println("Name : " + username); // line1
>             OutputStream out = new FileOutputStream(new File("output.zip"));
>             dataHandler.writeTo(out); // line2
>             System.out.println("Saving done!");
>         } catch (IOException e) {
>             throw new RuntimeException(e);
>         }
>     }
> }
> the entire attachment is loaded before useBinaryData() method is called.
> Therefore, when a large attachment is used execution of line 1 will
> be significantly delayed.
> However, when the suggested feature is implemented, stream will be read only
> when it is absolutely needed (i.e., in line 2). Therefore, execution of
> line1 will happen right after receiving the client request.
> Enabling attachment streaming will be similar to enabling file caching
> of attachments [1], an axis2.xml parameter or MessageContext property could
> be used.
> [1] http://axis.apache.org/axis2/java/core/docs/mtom-guide.html#a41
> Thanks,
> Sadeep
>
> On Mon, Jul 25, 2011 at 11:46 AM, Sadeep Jayasumana <gayansadeep@gmail.com>
> wrote:
>>
>> Hi Devs,
>> In Apache Syanpse, we have a requirement to proxy an MTOM enabled web
>> service with minimum overhead. Large files (even in GB range) should be able
>> to go through Synapse without running it OOM.
>> To satisfy this requirement, Synapse should be able to forward an incoming
>> SOAP message with MTOM attachments to the backend service without building
>> the attachments. Synapse might read/modify the SOAP envelop but not the
>> attachments. Therefore, it should be possible to stream attachments directly
>> from the Synpase's client to the backend service.
>> However, in the current implementation of AXIOM and Axis2, MTOM
>> attachments are built (in memory or in a file) by SOAPMessageFormatter. This
>> caused Synapse to run OOM when in the above mentioned scenario.
>> I have come up with a fix for this. It is to introduce a
>> new org.apache.axiom.attachments.impl.AbstractPart implementation which
>> streams non-soap MIME parts without building them.
>> To introduce this new feature without breaking existing stuff, I'm
>> planning to introduce a new message context property which enables MTOM
>> streaming. org.apache.axis2.builder.BuilderUtils class will check this
>> property in the message context and
>> create org.apache.axiom.attachments.Attachments object accordingly. Does
>> this sound like the correct way of introducing this feature?
>> Appreciate your feedback.
>> Thanks,
>> --
>> Sadeep Jayasumana
>> Software Engineer,
>> WSO2 Inc.
>
> --
> Sadeep Jayasumana
> Software Engineer,
> WSO2 Inc.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org


Mime
View raw message