axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sadeep Jayasumana <gayansad...@gmail.com>
Subject Re: Introducing a new feature to enable MTOM attachment streaming
Date Wed, 27 Jul 2011 02:45:42 GMT
Hi,

> Whether streaming is possible or not depends on how the
> MIME parts are accessed, but you always need to support buffering if
> necessary.

Yes, existing functionality will not be broken. In fact, building the Part
in memory will be the default behavior. Streaming will kick in only if it is
explicitly enabled.

> The Attachments object is one of the first things that will be created
> when a new message is received. Therefore nobody will be able to set a
> property in the message context, i.e. the setting would necessarily be
> a global property in axis2.xml. This however is a problem if you have
> services with diverging requirements. The decision to buffer or not
> the content of the MIME part can't be taken at this stage in the
> processing. It can only be taken when the MIME part is actually
> accessed (which includes serializing the message to forward it)

I agree with your comment. This functionality could be provided by
introducing a new method, setAttachmentStreaming(), to Attachments class or
MessageContext class. However, I'm wondering whether it is an elegant way of
doling this. Other ways of doing the same involves significant modifications
to Axiom API.

Thanks,
Sadeep

On Tue, Jul 26, 2011 at 12:03 AM, Andreas Veithen <andreas.veithen@gmail.com
> wrote:

> Some time ago I was thinking about this issue. It is highly non
> trivial to solve. Consider the following variant of your example:
>
>      public void useBinaryData(DataHandler dh1, DataHandler dh2) {
>          try {
>              OutputStream out = new FileOutputStream(new
> File("output1.zip"));
>              dh1.writeTo(out);
>              out = new FileOutputStream(new File("output2.zip"));
>              dh2.writeTo(out);
>          } catch (IOException e) {
>              throw new RuntimeException(e);
>          }
>      }
>
> Taking into account that the client is not required to send the MIME
> parts in any particular order and that the service is not required to
> consume them in any particular order, you will always end up with
> scenarios where you have no other choice than to buffer some of the
> MIME parts. Whether streaming is possible or not depends on how the
> MIME parts are accessed, but you always need to support buffering if
> necessary.
>
> What this example also shows is that you need to defer the calls to
> Attachments#getDataHandler(String) until the very last moment. When
> working with an Axiom tree, this is already the case: the call to
> Attachments#getDataHandler(String) occurs when OMText#getDataHandler()
> is called (and not when the OMText node is built). However, when using
> a data binding, one would have to create some sort of DataHandler
> proxy that defers the call to Attachments#getDataHandler(String).
>
> Andreas
>
> On Mon, Jul 25, 2011 at 09:21, Sadeep Jayasumana <gayansadeep@gmail.com>
> wrote:
> > Hi Devs,
> > Here is the benefit of this feature from Axis2's perspective.
> >
> > Currently, when I use Axis2 to deploy a service class as follows,
> > public class MTOMService {
> >     public void useBinaryData(String username, DataHandler dataHandler) {
> >         try {
> >             System.out.println("Name : " + username); // line1
> >             OutputStream out = new FileOutputStream(new
> File("output.zip"));
> >             dataHandler.writeTo(out); // line2
> >             System.out.println("Saving done!");
> >         } catch (IOException e) {
> >             throw new RuntimeException(e);
> >         }
> >     }
> > }
> > the entire attachment is loaded before useBinaryData() method is called.
> > Therefore, when a large attachment is used execution of line 1 will
> > be significantly delayed.
> > However, when the suggested feature is implemented, stream will be read
> only
> > when it is absolutely needed (i.e., in line 2). Therefore, execution of
> > line1 will happen right after receiving the client request.
> > Enabling attachment streaming will be similar to enabling file caching
> > of attachments [1], an axis2.xml parameter or MessageContext property
> could
> > be used.
> > [1] http://axis.apache.org/axis2/java/core/docs/mtom-guide.html#a41
> > Thanks,
> > Sadeep
> >
> > On Mon, Jul 25, 2011 at 11:46 AM, Sadeep Jayasumana <
> gayansadeep@gmail.com>
> > wrote:
> >>
> >> Hi Devs,
> >> In Apache Syanpse, we have a requirement to proxy an MTOM enabled web
> >> service with minimum overhead. Large files (even in GB range) should be
> able
> >> to go through Synapse without running it OOM.
> >> To satisfy this requirement, Synapse should be able to forward an
> incoming
> >> SOAP message with MTOM attachments to the backend service without
> building
> >> the attachments. Synapse might read/modify the SOAP envelop but not the
> >> attachments. Therefore, it should be possible to stream attachments
> directly
> >> from the Synpase's client to the backend service.
> >> However, in the current implementation of AXIOM and Axis2, MTOM
> >> attachments are built (in memory or in a file) by SOAPMessageFormatter.
> This
> >> caused Synapse to run OOM when in the above mentioned scenario.
> >> I have come up with a fix for this. It is to introduce a
> >> new org.apache.axiom.attachments.impl.AbstractPart implementation which
> >> streams non-soap MIME parts without building them.
> >> To introduce this new feature without breaking existing stuff, I'm
> >> planning to introduce a new message context property which enables MTOM
> >> streaming. org.apache.axis2.builder.BuilderUtils class will check this
> >> property in the message context and
> >> create org.apache.axiom.attachments.Attachments object accordingly. Does
> >> this sound like the correct way of introducing this feature?
> >> Appreciate your feedback.
> >> Thanks,
> >> --
> >> Sadeep Jayasumana
> >> Software Engineer,
> >> WSO2 Inc.
> >
> > --
> > Sadeep Jayasumana
> > Software Engineer,
> > WSO2 Inc.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>


-- 
Sadeep

Mime
View raw message