Return-Path: Delivered-To: apmail-ws-axis-user-archive@www.apache.org Received: (qmail 98950 invoked from network); 12 Aug 2008 07:10:00 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 12 Aug 2008 07:10:00 -0000 Received: (qmail 4227 invoked by uid 500); 12 Aug 2008 07:09:50 -0000 Delivered-To: apmail-ws-axis-user-archive@ws.apache.org Received: (qmail 3931 invoked by uid 500); 12 Aug 2008 07:09:49 -0000 Mailing-List: contact axis-user-help@ws.apache.org; run by ezmlm Precedence: bulk Reply-To: axis-user@ws.apache.org list-help: list-unsubscribe: List-Post: List-Id: Delivered-To: mailing list axis-user@ws.apache.org Received: (qmail 3920 invoked by uid 99); 12 Aug 2008 07:09:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Aug 2008 00:09:49 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [203.22.161.11] (HELO itreefw.itree.com.au) (203.22.161.11) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Aug 2008 07:08:49 +0000 Received: from imail.itree.com.au (mclaren.itree.com.au [149.135.204.50]) by itreefw.itree.com.au (8.11.6+Sun/8.10.2) with ESMTP id m7C71hr20324 for ; Tue, 12 Aug 2008 17:01:44 +1000 (EST) Received: from MCLAREN.itree.com.au ([149.135.204.50]) by MCLAREN.itree.com.au ([149.135.204.50]) with mapi; Tue, 12 Aug 2008 17:00:46 +1000 From: Shehan Simen To: "axis-user@ws.apache.org" Date: Tue, 12 Aug 2008 17:00:44 +1000 Subject: RE: Out Of Memory issue in axis2 1.4 Thread-Topic: Out Of Memory issue in axis2 1.4 Thread-Index: Acj8RYJPyxrlDEdvTByB3Z73ZgPcbwAAyITQ Message-ID: References: In-Reply-To: Accept-Language: en-US, en-AU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US, en-AU Content-Type: multipart/alternative; boundary="_000_BE6DB3905E87A7448CD6EE0AC22A4D86123BCD4608MCLARENitreec_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_BE6DB3905E87A7448CD6EE0AC22A4D86123BCD4608MCLARENitreec_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Thilina, Thank you for the explanation. But if I loaded the OmElement from the method, OmElement =3D request.getOM= Element, does not it mean that the whole request (with the attachment) load= ed to memory? If it is the case, then there is no point of doing streaming to the already= loaded OmElement. Please let me know. Thank you. Regards, Simen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Tuesday, 12 August 2008 4:35 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Nope... It's not due to Axis2.. It might be vulnerable due to your implementation.. For an example, you sai= d you are writing the whole OMElement to a DataBase. In such a case if you = just did OMElement.toString(), it'll cause the attachment to get loaded in = to memory. This will eventually cause a OOM. In such a scenerio you can try= streamingly writing it to the database (I don't know much about databases)= by serialising the OMElement to a stream. Or you can seperately store the = cached binary file in the database.. thanks, Thilina On Mon, Aug 11, 2008 at 8:27 PM, Shehan Simen > wrote: Hi Thilina, I will report the issue in JIRA. Why did you say that my server side is vulnerable for large attachments? Ar= e there known bugs in mtom? Please advise me. Thank you. Regards, simen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Tuesday, 12 August 2008 12:49 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Glad to know it's working.. You can go to https://issues.apache.org/jira/br= owse/AXIS2 and create a new issues... You might need to create account... Also keep in mind that u'r service side logic may be vulnerable to fail whe= n presented with a large attachment. thanks, Thilina On Mon, Aug 11, 2008 at 4:58 PM, Shehan Simen > wrote: Hi Thilina, Yes it is working if I use xsd:base64Binary rather than a simple type. So t= his is an issue to fix. Sorry, I am not sure how to log it in JIRA. If it is easy, could you please log it or let me know how to do it. Thank you for your help so far. Otherwise I will be stuck here forever. Regards, simen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Monday, 11 August 2008 5:56 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Hi, I'm not sure whether ADB will work for simple type restriction on ADB. Can = you change it to simply a xsd:base64Binary element and see whether it'll wo= rk. You can have a look at the wsdl given in Axis2 MTOM guide to get an id= ea. If that works, please log a JIRA for the simple type restriction issue. thanks, Thilina On Sun, Aug 10, 2008 at 10:52 PM, Shehan Simen > wrote: Hi Thilina, I am using ADB. The client and server are generated using wsdl2java eclipse= plugin. Both using ADB. This is the part related to my wsdl. - < - < = < Can you figure out the issue? Thanks. Regards, ssimen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Monday, 11 August 2008 3:35 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Hi, Can u explain a bit about ur client side... Which data binding framework r = u using?.. Does ur WSDL have any xsd:base64Binary elements. thanks, ~Thilina On Sun, Aug 10, 2008 at 10:22 PM, Shehan Simen > wrote: Hi Thilina, I looked the tcpmon and confirm that it is inside the body. But still I Can= 't understand why it is sending the attachment inside the soap body. I followed the http://wso2.org/library/264 to send attachments using MTOM Reading to the OMElement can be overcome by increasing the heap memory size= of the tomcat server. But the fundamental question still remains, that is = why the attachment is going inside the soap body? I tried almost everything= to send it as an mtom attachment. Much appreciate your help. Thank you. Regards, ssimen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Monday, 11 August 2008 3:14 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Hi, Yes, it is sending inside the soap body rather than an attachment. Why it i= s going inside the soap body? I changed all the required changes to send it= as separate attachment. (stub._getServiceClient().getOptions().setProperty( Constants.Configuration.ENABLE_MTOM, Constant= s.VALUE_TRUE);) Next question is, eventhough it is sending inside the soap bosy, what the p= roblem of it? Is there any size limitation to the soap message? It's the JVM heap size... The message get's loaded in to the memory with ad= ded base64 overhead. Did you check the message using TCPMON or are you ref= erring to the message you see in the database.. I am reading the whole request to the OMElement to save it in the database. Is that the problem? How can I fix it? Please help me. You can fix it by sending the binary data as an attachment.. But even the c= ached attachment will get loaded to the memory if you are reading the whole= request to an OMElement and writing it to the database. You will need to s= pecially handle the cached binary data file.. thanks, Thilina Thank you. Regards, Simen From: Thilina Gunarathne [mailto:csethil@gmail.com] Sent: Monday, 11 August 2008 2:01 PM To: axis-user@ws.apache.org Subject: Re: Out Of Memory issue in axis2 1.4 Can anyone please help me? Whats wrong with axis2? It can be two things... 1. Attachment may be going inline with base64 encoding, rather than as an a= ttachment. Check the message using tcpmon.(http://ws.apache.org/commons/tcp= mon) 2. Your service implementation might be loading the whole cached binary dat= a to the memory.. thanks, Thilina Thank you. Regards -- Thilina Gunarathne - http://thilinag.blogspot.com -- Thilina Gunarathne - http://thilinag.blogspot.com -- Thilina Gunarathne - http://thilinag.blogspot.com -- Thilina Gunarathne - http://thilinag.blogspot.com -- Thilina Gunarathne - http://thilinag.blogspot.com -- Thilina Gunarathne - http://thilinag.blogspot.com --_000_BE6DB3905E87A7448CD6EE0AC22A4D86123BCD4608MCLARENitreec_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Thilina,

Thank you for the explanation.

But if I loaded the OmElement from the method, OmElement &nb= sp;=3D request.getOMElement, does not it mean that the whole request (with the attachment) loaded to mem= ory?

If it is the case, then there is no point of doing streaming= to the already loaded OmElement.

Please let me know.

 

Thank you.

 

Regards,

Simen

 

From: Thilina Gunar= athne [mailto:csethil@gmail.com]
Sent: Tuesday, 12 August 2008 4:35 PM
To: axis-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Nope... It's not due to= Axis2..

It might be vulnerable due to your implementation.. For an example, you sai= d you are writing the whole OMElement to a DataBase. In such a case if you ju= st did OMElement.toString(), it'll cause the attachment to get loaded in to memory. This will eventually cause a OOM. In such a scenerio you can try streamingly writing it to the database (I don't know much about databases) = by serialising the OMElement to a stream. Or you can seperately store the cach= ed binary file in the database..

thanks,
Thilina

On Mon, Aug 11, 2008 at 8:27 PM, Shehan Simen <ssimen@itree.com.au> wrote:=

Hi Thilina,

I will report the issue i= n JIRA.

Why did you say that my s= erver side is vulnerable for large attachments? Are there known bugs in mtom?

Please advise me.<= o:p>

 <= /p>

Thank you.

 <= /p>

Regards,

simen <= /p>

 <= /p>

From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Tuesday, 12 August 2008 12:49 PM


To: axi= s-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Glad to know it's working.. You can go to= http= s://issues.apache.org/jira/browse/AXIS2 and create a new issues... You might need to create account...

Also keep in mind that u'r service side logic may be vulnerable to fail whe= n presented with a large attachment.

thanks,
Thilina

On Mon, Aug 11, 2008 at 4:58 PM, Shehan Simen <ssimen@itree.com.au> wrote:

Hi Thilina,

Yes it is working if I us= e xsd:base64Binary rather than a simple type. So this is an issue to fix. Sor= ry, I am not sure how to log it in JIRA.

If it is easy, could you = please log it or let me know how to do it.

 <= /p>

Thank you for your help s= o far. Otherwise I will be stuck here forever.

 <= /p>

Regards,

simen

 <= /p>

From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Monday, 11 August 2008 5:56 PM


To: axi= s-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Hi,
I'm not sure whether ADB will work for simple type restriction on ADB. Can = you change it to simply a xsd:base64Binary element and see whether it'll work. = You can have a look at the wsdl given in  Axis2 MTOM guide to get an idea.=

If that works, please log a JIRA for the simple type restriction issue.

thanks,
Thilina

On Sun, Aug 10, 2008 at 10:52 PM, Shehan Simen <ssimen@itree.com.au> wrote:

Hi Thilina,

I am using ADB. The clien= t and server are generated using wsdl2java eclipse plugin. Both using ADB.=

This is the part related = to my wsdl.

 <= /p>

- <<xs:simple= Type name=3D"FileRawDataType">

- <    <xs:restriction base=3D"xs:base64Binary">

  <   &nb= sp;    <xs:maxLength value=3D"5242880" />

    </xs:restriction>= ;

 </xs:simpleType>

 

  Can you figure out the issue?

 

   Thanks.

   Regards,

    ssimen

 <= /p>

 <= /p>

From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Monday, 11 August 2008 3:35 PM


To: axi= s-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Hi,
Can u explain a bit about ur client side... Which data binding framework r = u using?.. Does ur WSDL have any xsd:base64Binary elements.

thanks,
~Thilina

On Sun, Aug 10, 2008 at 10:22 PM, Shehan Simen <ssimen@itree.com.au> wrote:

Hi Thilina,

I looked the tcpmon and c= onfirm that it is inside the body. But still I Can't understand why it is sending = the attachment inside the soap body.

I followed the http://wso2.org/libr= ary/264 to send attachments using MTOM

 

Reading to the OMElement can be overcome by increasing the heap memory s= ize of the tomcat server. But the fundamental question still remains, that is w= hy the attachment is going inside the soap body? I tried almost everything to = send it as an mtom attachment.

Much appreciate your help.

 

Thank you.

 

Regards,

ssimen

 <= /p>

 <= /p>

From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Monday, 11 August 2008 3:14 PM


To: axi= s-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Hi,

Yes, it is sending inside= the soap body rather than an attachment. Why it is going inside the soap body? = I changed all the required changes to send it as separate attachment.<= o:p>

(stub._getServiceClient().getOptions().setProperty(

    &nb= sp;            =              Constants.Configuration.ENABLE_MTOM, Constants.VALUE_TRUE);)

 

Next question is, eventhoug= h it is sending inside the soap bosy, what the problem of it? Is there any size limitation to the soap message?

It's the JVM heap size... The message get's loaded in to the memory with added base64 overhead.  Did you check the message using TCPMON or are = you referring to the message you see in the database..

I am reading the whole requ= est to the OMElement to save it in the database.

Is that the problem?=

How can I fix it? Please he= lp me.

You can fix it by sending the binary data as an attachment.. But even th= e cached attachment will get loaded to the memory if you are reading the whol= e request to an OMElement and writing it to the database. You will need to specially handle the cached binary data file..

thanks,
Thilina

 

Thank you.

Regards,<= /p>

Simen

 <= /p>

From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Monday, 11 August 2008 2:01 PM
To: axi= s-user@ws.apache.org
Subject: Re: Out Of Memory issue in axis2 1.4

 

Can anyone please help me? Whats wrong with axis2?

It can be two things...
1. Attachment may be going inline with base64 encoding, rather than as an attachment. Check the message using tcpmon.(http://ws.ap= ache.org/commons/tcpmon)
2. Your service implementation might be loading the whole cached binary dat= a to the memory..

thanks,
Thilina

 

Thank you.

 

Regards

 




--
Thilina Gunarathne - http://thilinag.blogspot.com




--
Thilina Gunarathne - http://thilinag.blogspot.com




--
Thilina Gunarathne - http://thilinag.blogspot.com




--
Thilina Gunarathne - http://thilinag.blogspot.com




--
Thilina Gunarathne - http://thilinag.blogspot.com




--
Thilina Gunarathne - http://thilin= ag.blogspot.com

--_000_BE6DB3905E87A7448CD6EE0AC22A4D86123BCD4608MCLARENitreec_--