synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Veithen <andreas.veit...@gmail.com>
Subject Re: Streaming processing in Synapse with AXIOM
Date Tue, 13 Aug 2013 13:22:35 GMT
On Tue, Aug 13, 2013 at 12:06 PM, Auke Schrijnen <aschrijnen@bkwi.nl> wrote:
> Hi all,
>
> We have a couple of custom mediators for Synapse to do various transformations,
> logging, validation, filtering and caching. Working with the AXIOM model is
> pretty straight forward, one can easily navigate through the model, alter
> elements, rename or remove them and so on.
>
> While we get deferred parsing for free with AXIOM, the whole tree has to be
> built at some point which is fine for smaller messages but could cause memory
> problems and introduce longer delays for larger messages.
>
> Most of the things we do could benefit some form of streaming processing but
> that is not so easy with Synapse/AXIOM. This is what i have come up with:
>
> The SOAP Envelope, Header including all header blocks and the Body element
> will always be built as OM model because the header modules need to process
> soap headers. We also have a custom module for processing our custom header.
> The headers are small compared to the body so building the headers at once
> isn't a problem and we could actually use this later on.
>
> One form of streamed processing could be accomplished by getting the
> XMLStreamReader and apply the processing to the stream. This will need some
> work, but events from the stream can be altered or removed.
>
> Applying a filter to an existing OMElement isn't straight forward because we
> can't replace the element with something else and as soon as the element gets
> detached it will consume it's input stream and build the whole model. The only
> way i found is creating a new soap envelope and replace the current envelope in
> the Synapse message context with this new envelope, move the headers from the
> old envelope to the new envelope (here we benefit from the fact that the
> headers are already built), create a new soap body and add an OMDataSource as
> first child. This OMDataSouce uses the XMLStreamReader from the original
> element and wraps it in a filtering XMLStreamReader. Because the SOAP Body
> element cannot be wrapped in an OMDataSource we can only create one child
> element. I'm not sure this is a problem because WS-I BP only allows one child
> in the Body element.
>
> Now we can apply all kinds of filtering to the events from the XMLStreamReader,
> create new events and use the events for logging, validation and caching. The
> OM model isn't even built for most of the payload.
>
> The downside to this approach is that it operates on the XMLStreamReader which
> is very low-level compared to the OM model from AXIOM and it can only be
> applied to the first element of the SOAP Body.
>
> Does anyone have solved this problem before? Or does someone has an idea for a
> different approach? Andreas do you have more magic AXIOM tricks up your sleeve?

Yes I do. As you have noted, one of the key limitations right now is that

"as soon as the element gets detached it will consume it's input
stream and build the whole model."

That is actually not an intrinsic limitation of the architecture
behind Axiom, but merely a limitation of the current design of one of
the components, namely the builder (StAXBuilder/StAXOMBuilder). I was
thinking about this the other day and it should actually be quite easy
to remove that limitation.

Andreas

> Thanks,
>
> Auke
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
> For additional commands, e-mail: dev-help@synapse.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
For additional commands, e-mail: dev-help@synapse.apache.org


Mime
View raw message