Return-Path: Delivered-To: apmail-ws-axis-dev-archive@www.apache.org Received: (qmail 15782 invoked from network); 19 Sep 2007 14:59:50 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 19 Sep 2007 14:59:50 -0000 Received: (qmail 83745 invoked by uid 500); 19 Sep 2007 14:59:38 -0000 Delivered-To: apmail-ws-axis-dev-archive@ws.apache.org Received: (qmail 83693 invoked by uid 500); 19 Sep 2007 14:59:38 -0000 Mailing-List: contact axis-dev-help@ws.apache.org; run by ezmlm Precedence: bulk Reply-To: axis-dev@ws.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list axis-dev@ws.apache.org Received: (qmail 83682 invoked by uid 99); 19 Sep 2007 14:59:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Sep 2007 07:59:38 -0700 X-ASF-Spam-Status: No, hits=-2.0 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of scheu@us.ibm.com designates 32.97.110.152 as permitted sender) Received: from [32.97.110.152] (HELO e34.co.us.ibm.com) (32.97.110.152) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Sep 2007 14:59:36 +0000 Received: from d03relay02.boulder.ibm.com (d03relay02.boulder.ibm.com [9.17.195.227]) by e34.co.us.ibm.com (8.13.8/8.13.8) with ESMTP id l8JExEJo030221 for ; Wed, 19 Sep 2007 10:59:14 -0400 Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by d03relay02.boulder.ibm.com (8.13.8/8.13.8/NCO v8.5) with ESMTP id l8JExEh0497352 for ; Wed, 19 Sep 2007 08:59:14 -0600 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id l8JExEnd002515 for ; Wed, 19 Sep 2007 08:59:14 -0600 Received: from d03nm118.boulder.ibm.com (d03nm118.boulder.ibm.com [9.17.195.144]) by d03av04.boulder.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id l8JExDnb002489 for ; Wed, 19 Sep 2007 08:59:13 -0600 In-Reply-To: <60708f4b0709190119n3f42b405t1f6e4efafbd9528@mail.gmail.com> Subject: Re: [axis2] Revisiting the Proposal for fixing out-of-memory error JIRA axis2-2968 To: axis-dev@ws.apache.org Cc: axis-dev@ws.apache.org X-Mailer: Lotus Notes Release 7.0 HF277 June 21, 2006 Message-ID: From: R J Scheuerle Jr Date: Wed, 19 Sep 2007 09:59:13 -0500 X-MIMETrack: Serialize by Router on D03NM118/03/M/IBM(Release 8.0|August 02, 2007) at 09/19/2007 08:59:13 MIME-Version: 1.0 Content-type: multipart/alternative; Boundary="0__=08BBF9C8DFC2BA488f9e8a93df938690918c08BBF9C8DFC2BA48" Content-Disposition: inline X-Virus-Checked: Checked by ClamAV on apache.org --0__=08BBF9C8DFC2BA488f9e8a93df938690918c08BBF9C8DFC2BA48 Content-type: text/plain; charset=US-ASCII Thanks Ann. Let me try to summarize to see if I understand this change: The new patch is the WSDL Wrapper "interface" without any implementation. There are no changes to the loading, unloading or logic. When WSDL4J completes their "reloading" implementation, then we can easily plug in the implementation under the WSDL Wrapper. And if someone wants an interim solution, they can provide their own implementation. Sounds like a good plan for me. Do you have an outlook when the WSDL4J implementation will be ready ? Thanks for the thorough explanation. Rich Scheuerle IBM Web Services Apache Axis2 (scheu@apache.org) 512-838-5115 (IBM TL 678-5115) "Amila Suriarachchi" wrote on 09/19/2007 03:19:09 AM: > > On 9/19/07, Ann Robinson wrote: > Hi, all, > This is a re-send of the note with the correct [axis2] prefix in the > subject line. > > I have updated the proposed fix for the out-of-memory error that occurs on > the server in some environments, and I would be interested in getting some > feedback on the updated proposal. > > Some background: > ---------------- > This out-of-memory (OOM) error happens on the server in some > environments where there is > a large number of applications. > > At deployment time, when the applications are loaded, the > AxisService objects are created. > For those applications where the loading ends up invoking the > WSDL11ToAxisServiceBuilder, > the wsdl is read in and saved as a Parameter on the AxisService > object. As the applications > are loaded, an OOM happens before requests start flowing. > > In analyzing the java heap dumps, one of the biggest consumers of memory > is with the wsdl4j WSDLDefinition objects. The heap dump indicates that > the WSDLDefinition uses the xerces dom for underlying support, particularly > for schemas. This makes the WSDLDefinition very heavy-weight. > > The heap dump also shows that the WSDLDefinition objects of concern are the > ones being saved in the AxisService object's ParameterInclude list. > Comments in > the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that, if > some component needs to utilize the WSDLDefinition, the component can access > it via a Parameter in the AxisService object. > > The initial proposed fix was to use a light-weight wrapper on the > WSDL4J's WSDL definition > object in order to release resources and reload WSDL if needed. > Prototypes of this fix > did resolve the OOM. This solution, however, had drawbacks with > repect to custom WSDL locators > and ?WSDL support. > > > Updated Proposed Fix: > --------------------- > > I have been working with the WSDL4J team on ways to reduce the > memory footprint > of the WSDL definition implementation. This means that a newer > version of WSDL4J > will need to be picked up by AXIS2 when the updated WSDL4J becomes available. > > In the meantime, for Axis2, I would like to propose using a wrapper on the > WSDL4J's WSDLDefinition object for the following reasons: > - to keep changes to the WSDL4J implementation from impacting users > (especially those components that expect to access a WSDLDefinition > from the parameter include list in the AxisService) > - to allow certain environments to take advantage of changesin WSDL4J, > (eg. new APIs, new steps, or catch exceptions) to reduce > memory footprint > - to have the option of doing an intermediate solution for > certain environments > while preserving current behavior of the Axis2 kernel for > the rest of the > environments > > The difference in this solution from the previously proposed solution is that > - no resources will be released > - no need to try to reload the original WSDL (so we don't > have to worry about > custom WSDL locators or ?WSDL support) > > This wrapper on the WSDLDefinition would do the following: > - implement the wsdl definition interface and delegate the > methods to the actual > WSDL4J wsdl definition implementation that is being wrapped > - provide tracing for debug situations > - perform new steps to reduce memory footprint only when > needed for certain environments > - - this could be controlled via a configuration > parameter so that the default > behavior is the same as current behavior > > what is the parameter you have used for that? As I saw in your patch > you have changed the existing classes with out checking for a parameter. > As I saw this change is effect to the ?wsdl and wsdl2java tool and > wsdl11Axisservice builder at deployment time. > does all these cases work properly with imports and includes? > if these working features are preserved I am ok with this change. > > - - this could include serializing/deserializing the > WSDL4J's WSDLDefinition > object if the WSDLDefinition can be serialized > > > Comments? > - Thanks, Ann > > > > -- > Amila Suriarachchi, > WSO2 Inc. --0__=08BBF9C8DFC2BA488f9e8a93df938690918c08BBF9C8DFC2BA48 Content-type: text/html; charset=US-ASCII Content-Disposition: inline

Thanks Ann.

Let me try to summarize to see if I understand this change:

The new patch is the WSDL Wrapper "interface" without any implementation.
There are no changes to the loading, unloading or logic.

When WSDL4J completes their "reloading" implementation, then we can easily plug in the
implementation under the WSDL Wrapper.

And if someone wants an interim solution, they can provide their own implementation.

Sounds like a good plan for me. Do you have an outlook when the WSDL4J implementation will be ready ?

Thanks for the thorough explanation.

Rich Scheuerle
IBM Web Services
Apache Axis2 (scheu@apache.org)
512-838-5115 (IBM TL 678-5115)

"Amila Suriarachchi" <amilasuriarachchi@gmail.com> wrote on 09/19/2007 03:19:09 AM:

>

> On 9/19/07, Ann Robinson <robinsona@us.ibm.com> wrote:
> Hi, all,
> This is a re-send of the note with the correct [axis2] prefix in the
> subject line.
>
> I have updated the proposed fix for the out-of-memory error that occurs on
> the server in some environments, and I would be interested in getting some
> feedback on the updated proposal.
>
> Some background:
> ----------------
> This out-of-memory (OOM) error happens on the server in some
> environments where there is
> a large number of applications.
>
> At deployment time, when the applications are loaded, the
> AxisService objects are created.
> For those applications where the loading ends up invoking the
> WSDL11ToAxisServiceBuilder,
> the wsdl is read in and saved as a Parameter on the AxisService
> object.  As the applications
> are loaded, an OOM happens before requests start flowing.
>
> In analyzing the java heap dumps, one of the biggest consumers of memory
> is with the wsdl4j WSDLDefinition objects.  The heap dump indicates that
> the WSDLDefinition uses the xerces dom for underlying support, particularly
> for schemas.  This makes the WSDLDefinition very heavy-weight.
>
> The heap dump also shows that the WSDLDefinition objects of concern are the
> ones being saved in the AxisService object's ParameterInclude list.
> Comments in
> the code (WSDL11ToAxisServiceBuilder) indicate that this is done so that, if
> some component needs to utilize the WSDLDefinition, the component can access
> it via a Parameter in the AxisService object.
>
> The initial proposed fix was to use a light-weight wrapper on the
> WSDL4J's WSDL definition
> object in order to release resources and reload WSDL if needed.
> Prototypes of this fix
> did resolve the OOM. This solution, however, had drawbacks with
> repect to custom WSDL locators
> and ?WSDL support.
>
>
> Updated Proposed Fix:
> ---------------------
>
> I have been working with the WSDL4J team on ways to reduce the
> memory footprint
> of the WSDL definition implementation.  This means that a newer
> version of WSDL4J
> will need to be picked up by AXIS2 when the updated WSDL4J becomes available.
>
> In the meantime, for Axis2, I would like to propose using a wrapper on the
> WSDL4J's WSDLDefinition object for the following reasons:
>         - to keep changes to the WSDL4J implementation from impacting users
>           (especially those components that expect to access a WSDLDefinition
>            from the parameter include list in the AxisService)
>         - to allow certain environments to take advantage of changesin WSDL4J,
>           (eg. new APIs, new steps, or catch exceptions) to reduce
> memory footprint
>         - to have the option of doing an intermediate solution for
> certain environments
>            while preserving current behavior of the Axis2 kernel for
> the rest of the
>            environments
>
> The difference in this solution from the previously proposed solution is that
>         - no resources will be released
>         - no need to try to reload the original WSDL (so we don't
> have to worry about
>            custom WSDL locators or ?WSDL support)
>
> This wrapper on the WSDLDefinition would do the following:
>         - implement the wsdl definition interface and delegate the
> methods to the actual
>           WSDL4J wsdl definition implementation that is being wrapped
>         - provide tracing for debug situations
>         - perform new steps to reduce memory footprint only when
> needed for certain environments
>                - - this could be controlled via a configuration
> parameter so that the default
>                     behavior is the same as current behavior
>
> what is the parameter you have used for that? As I saw in your patch
> you have changed the existing classes with out checking for a parameter.
> As I saw this change is effect to the ?wsdl and wsdl2java tool and
> wsdl11Axisservice builder at deployment time.
> does all these cases work properly with imports and includes?
> if these working features are preserved I am ok with this change.

>
>                 - - this could include serializing/deserializing the
> WSDL4J's WSDLDefinition
>                      object if the WSDLDefinition can be serialized
>
>
> Comments?
> - Thanks, Ann
>
>
>
> --
> Amila Suriarachchi,
> WSO2 Inc. --0__=08BBF9C8DFC2BA488f9e8a93df938690918c08BBF9C8DFC2BA48--