axis-c-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thilina Gunarathne" <cset...@gmail.com>
Subject Re: MTOM Caching With a user provided callback
Date Wed, 11 Jun 2008 18:38:51 GMT
Hi,

> Yes while we are parsing the stream depending on the size write it to a
> file or keep it on memory. This parsing happens at very early stages of
> request processing at the transport level even before invoking the
> engine. The data_handler has a reference to the attachment if it is in
> memory or it knows if it is cached.

In java, we parse the attachments only when they are absolutely needed.

>
>
> This means calling the mime_parser happens at the service business
> logic. Till that the mime attachments are in the incoming data stream.

yes. It has some advantages. For an example business logic gets to know
about the message before attachments are processed, in which case it can
decides what to do with the attachment, whether it does not need it or
whether it's going to forward it (eg:esb).. Axis2/java also allows the users
to directly access the incoming attachments streams if they want to, which
will allow them to forward the stream directly without reading to memory or
file  at all..

>
> So inside the business logic of the service we need to have access to
> the incoming data stream in order to separate the attachment. Please
> correct me if I am wrong.
>
It's like deffered StAX parsing of XML and building the Axiom. We virtually
have a seperate object  model for deffered parsed attachments.

In Axiom/java, the builder (which is the wrapper for StaXparser) keeps a
reference to the MIME parser instance, which also contains the above MIME
objects.  All Axiom objects have a link to it's builder.  When a service
calls the textNode.getDataHandler(), the method requests the attachments
from the builder if it's not already in the text node, then the builder
calls the Mime Parser and requests it. Mime parser looks at it's objects
model and if the MIME parts if not parsed yet it parses it and returns the
data handler.

So IMHO there aren't any more complexity or exposing than what we have with
Stax parsing. Hope the internal are same in the Axiom/c too..

thanks,
Thilina


>
>
>
>  >  At that point we are aware of the service and even the operation. We
> > might be able to pass in that information to the MIME parser when we
> > are requesting the actual attachment.
> > There is the very rare case of SOAP part not being the first MIME part
> > scenerio which will break the above model as you need to parse the
> > earlier MIME parts before getting to the SOAP part.  But frankly
> > I have not seeing any web service engine or client does that, even
> > though the spec does not restricts it.
> >
> >  thanks,
> > Thilina
> >
> >
> >         -Manjula.
> >
> >
> >
> >         > Samisa...
> >         >
> >         > Supun Kamburugamuva wrote:
> >         > > Hi Manula,
> >         > >
> >         > > I understand your point. But as Samisa said this is a
> >         service specific
> >         > > configuration and I'm sure users will definitely have
> >         different
> >         > > callbacks for different services. So I think we need to
> >         figure out a
> >         > > way to do this configuration at the service level.
> >         > >
> >         > > Supun..
> >         > >
> >         > > On Mon, Jun 9, 2008 at 9:19 PM, Manjula Peiris
> >         <manjula@wso2.com
> >         > > <mailto:manjula@wso2.com>> wrote:
> >         > >
> >         > >
> >         > >     On Mon, 2008-06-09 at 23:52 +0530, Samisa Abeysinghe
> >         wrote:
> >         > >     > Manjula Peiris wrote:
> >         > >     > > On Mon, 2008-06-09 at 20:24 +0530, Samisa
> >         Abeysinghe wrote:
> >         > >     > >
> >         > >     > >> Manjula Peiris wrote:
> >         > >     > >>
> >         > >     > >>> Seems that every one is ok with this approach.
> >         But as I
> >         > >     mentioned this
> >         > >     > >>> approach has problems when there are multiple
> >         MTOM services.
> >         > >     > >>>
> >         > >     > >> You mean threading issues?
> >         > >     > >>
> >         > >     > >
> >         > >     > > No in the implementation I have specified the
> >         callback in the
> >         > >     axis2.xml.
> >         > >     > > But when there are many services which have
> >         different
> >         > >     callbacks then
> >         > >     > > there should be a way to specify them. We can't
> >         put this in the
> >         > >     > > services.xml becasue mime_parser does not have any
> >         information
> >         > >     about the
> >         > >     > > services.
> >         > >     > >
> >         > >     >
> >         > >     > I think the correct way to do this is to use the one
> >         given in
> >         > >     axis2.xml
> >         > >     > as the default and override that with one in
> >         services.xml, if
> >         > >     there is
> >         > >     > one provided. If the callback is set as a parameter,
> >         then this
> >         > >     will work
> >         > >     > seamlessly, as service level parameters override
> >         conf level params.
> >         > >
> >         > >     When we are invoking the mime_parser the service is
> >         not dispatched. So
> >         > >     the mime_parser don't know which callback to load if
> >         it is
> >         > >     specified in
> >         > >     the services.xml.
> >         > >
> >         > >     >
> >         > >     > Samisa...
> >         > >     >
> >         > >     > >
> >         > >     > >> Thanks,
> >         > >     > >> Samisa...
> >         > >     > >>
> >         > >     > >>
> >         > >     > >>> Any way I
> >         > >     > >>> will integrate the stuff when the sending
side
> >         is finished.
> >         > >     > >>>
> >         > >     > >>> Thanks,
> >         > >     > >>> -Manjula.
> >         > >     > >>>
> >         > >     > >>>
> >         > >     > >>> On Mon, 2008-06-02 at 12:12 +0530, Manjula
> >         Peiris wrote:
> >         > >     > >>>
> >         > >     > >>>
> >         > >     > >>>> Hi all,
> >         > >     > >>>>
> >         > >     > >>>> I have developed the MTOM Caching stuff
and it
> >         is almost
> >         > >     finished in the
> >         > >     > >>>> receiving side. The implementation is
here [1]
> >         as a branch.
> >         > >     I have
> >         > >     > >>>> tested with a 160MB attachment and the
results
> >         are
> >         > >     promising. The main
> >         > >     > >>>> advantage of this implementation is
it keeps a
> >         fixed user
> >         > >     defined size
> >         > >     > >>>> of the attachment at any moment during
the
> >         attachment
> >         > >     processing. The
> >         > >     > >>>> user can implement his own callback
to cache
> >         the
> >         > >     attachment. I have
> >         > >     > >>>> included a sample callback [2] which
stores the
> >         attachment
> >         > >     in to a
> >         > >     > >>>> file.
> >         > >     > >>>>
> >         > >     > >>>> Following are the user configurable
parameters
> >         in axis2.xml.
> >         > >     > >>>>
> >         > >     > >>>> <parameter name="MTOMBufferSize"
> >         locked="false">10</parameter>
> >         > >     > >>>>
> >         > >     > >>>> Using this parameter the user can govern
the
> >         caching
> >         > >     threshold. This is
> >         > >     > >>>> in KB and I will fix this to be in MB.For
> >         example this is
> >         > >     10 means any
> >         > >     > >>>> attachment greater than 10KB will be
cached.
> >         > >     > >>>>
> >         > >     > >>>> <parameter name="MTOMMaxBuffers"
> >         > >     locked="false">1000</parameter>
> >         > >     > >>>>
> >         > >     > >>>> This will tell maximum numbers of buffers
> >         Axis2/C can
> >         > >     handle when the
> >         > >     > >>>> attachment comes with a pretty large
SOAP
> >         envelope. Because
> >         > >     we are not
> >         > >     > >>>> caching the SOAP envelope.
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >     > >>>> <parameter name="MTOMCachingCallback"
> >         > >     > >>>>
> >         locked="false">/path/to/your/caching_callabck</parameter>
> >         > >     > >>>>
> >         > >     > >>>> User will specify the callback name
using this
> >         parameter so
> >         > >     that Axis2/C
> >         > >     > >>>> can load when it wants to cache the
attachment.
> >         If the user
> >         > >     does not
> >         > >     > >>>> specify this the attachment will be
in memory.
> >         > >     > >>>>
> >         > >     > >>>> I also added a flag called 'cached'
to the
> >         > >     axiom_data_handler. This flag
> >         > >     > >>>> will tell whether the attachment is
cached or
> >         not using the
> >         > >     user
> >         > >     > >>>> provided callback. If it is cached then
it is
> >         user's
> >         > >     responsibility in
> >         > >     > >>>> his client/service code to find the
attachment,
> >         because
> >         > >     Axis2/C cached
> >         > >     > >>>> it using his callback.
> >         > >     > >>>>
> >         > >     > >>>> I have a one major problem in this model.
That
> >         is if there
> >         > >     are more than
> >         > >     > >>>> one mtom services and if those are going
to use
> >         different
> >         > >     callbacks then
> >         > >     > >>>> we can't specify this in axis2.xml.
We can't
> >         specify this in
> >         > >     > >>>> services.xml either because attachment
> >         separation happens
> >         > >     before
> >         > >     > >>>> accessing the service.
> >         > >     > >>>>
> >         > >     > >>>> So really appreciate your ideas regarding
the
> >         above
> >         > >     parameters, callback
> >         > >     > >>>> and the problem I have mentioned above.
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >
> >         [1]
> https://svn.apache.org/repos/asf/webservices/axis2/branches/c/post_1_4_mtom/c
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >
> >         [2]
> https://svn.apache.org/repos/asf/webservices/axis2/branches/c/post_1_4_mtom/c/samples/mtom_caching_callback/mtom_caching_callback.c
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >     > >>>> Thanks,
> >         > >     > >>>> -Manjula
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >     > >>>>
> >         > >     > >>>
> >         > >
> >
> ---------------------------------------------------------------------
> >         > >     > >>> To unsubscribe, e-mail:
> >         axis-c-dev-unsubscribe@ws.apache.org
> >         > >     <mailto:axis-c-dev-unsubscribe@ws.apache.org>
> >         > >     > >>> For additional commands, e-mail:
> >         > >     axis-c-dev-help@ws.apache.org
> >         <mailto:axis-c-dev-help@ws.apache.org>
> >         > >     > >>>
> >         > >     > >>>
> >         > >
> >
> ------------------------------------------------------------------------
> >         > >     > >>>
> >         > >     > >>>
> >         > >     > >>> No virus found in this incoming message.
> >         > >     > >>> Checked by AVG.
> >         > >     > >>> Version: 8.0.100 / Virus Database: 270.0.0/1489
> >         - Release
> >         > >     Date: 6/7/2008 11:17 AM
> >         > >     > >>>
> >         > >     > >>>
> >         > >     > >>
> >         > >     > >
> >         > >     > >
> >         > >     > >
> >         > >
> >
> ---------------------------------------------------------------------
> >         > >     > > To unsubscribe, e-mail:
> >         axis-c-dev-unsubscribe@ws.apache.org
> >         > >     <mailto:axis-c-dev-unsubscribe@ws.apache.org>
> >         > >     > > For additional commands, e-mail:
> >         axis-c-dev-help@ws.apache.org
> >         > >     <mailto:axis-c-dev-help@ws.apache.org>
> >         > >     > >
> >         > >     > >
> >         > >
> >
> ------------------------------------------------------------------------
> >         > >     > >
> >         > >     > >
> >         > >     > > No virus found in this incoming message.
> >         > >     > > Checked by AVG.
> >         > >     > > Version: 8.0.100 / Virus Database: 270.0.0/1489
-
> >         Release
> >         > >     Date: 6/7/2008 11:17 AM
> >         > >     > >
> >         > >     >
> >         > >     >
> >         > >
> >         > >
> >         > >
> >
> ---------------------------------------------------------------------
> >         > >     To unsubscribe, e-mail:
> >         axis-c-dev-unsubscribe@ws.apache.org
> >         > >     <mailto:axis-c-dev-unsubscribe@ws.apache.org>
> >         > >     For additional commands, e-mail:
> >         axis-c-dev-help@ws.apache.org
> >         > >     <mailto:axis-c-dev-help@ws.apache.org>
> >         > >
> >         > >
> >         > >
> >
> ------------------------------------------------------------------------
> >         > >
> >         > >
> >         > > No virus found in this incoming message.
> >         > > Checked by AVG.
> >         > > Version: 8.0.100 / Virus Database: 270.0.0/1489 - Release
> >         Date: 6/7/2008 11:17 AM
> >         > >
> >         >
> >         >
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------
> >         To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
> >         For additional commands, e-mail: axis-c-dev-help@ws.apache.org
> >
> >
> >
> >
> >
> > --
> > Thilina Gunarathne - http://thilinag.blogspot.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-c-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-c-dev-help@ws.apache.org
>
>


-- 
Thilina Gunarathne - http://thilinag.blogspot.com

Mime
View raw message