synapse-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajika Kumarasiri <raj...@wso2.com>
Subject Re: Please help: does Synapse leak memory?
Date Fri, 20 May 2011 05:21:54 GMT
Can you attach your Synapse configuration if possible? And tell which
endpoint has a typo ?

Rajika

On Thu, May 19, 2011 at 11:47 PM, Abid Khan-EXT <Abid.Khan@jeppesen.com>wrote:

> Thanks Hiranya,
>
> I have done some more research, and found out the it leaks memory only on
> the failures. Not on successful deliver.
>
> To give you the back ground, I had a typo in one of the endpoint URL. And I
> had only two endpoints, all the message that were attempted to be delivered
> to that "wrong" url; failed. That is what lead to this leaking issue.
>
> On the other hand the server I was testing had -Xms1024m, and -Xmx2048m. So
> its little better than what I was thinking.
>
> I am still digging into and trying to find where the leak is. I will update
> if I found anything but so far it's a leak on the failures only.
>
> If you can find out if this is correct that Synapse could leak those two
> object on failures. That will help.
>
> Thanks,
> Abid
>
>
>
>
> -----Original Message-----
> From: Hiranya Jayathilaka [mailto:hiranya911@gmail.com]
> Sent: Thursday, May 19, 2011 12:09 PM
> To: user@synapse.apache.org
> Subject: Re: Please help: does Synapse leak memory?
>
> Synapse creates an Axis2MessageContext for each message received. Each
> Axis2MessageContext instance encapsulates a MessageContext object. But
> Synapse does not keep them in memory for too long. As soon as messages are
> mediated and sent to the corresponding endpoints/clients, they are
> discarded. These objects could be fairly big since they contain message
> payload, properties and all other context information of messages. So if
> Synapse is keeping them in memory forever, we won't be able to run any
> scenario with Synapse for more than several hours. But as Paul mentioned we
> have seen some pretty heavy duty deployments of Synapse dealing with
> hundreds of transactions every second. One of the key strengths of Synapse
> is that it can handle very high volumes of messages with a very small
> memory
> footprint.
>
> How much memory have you allocated for JBoss server? Also what is your
> Synapse configuration like? What's the memory usage pattern when you run
> Synapse standalone (ie without JBoss)?
>
> Thanks,
> Hiranya
>
> On Thu, May 19, 2011 at 9:17 PM, Abid Khan-EXT <Abid.Khan@jeppesen.com
> >wrote:
>
> > Hello Paul,
> >
> > What you are telling me is a good news. The facts I have here, are
> > different. I ran the server for a load-test and ran only 26000 message
> (of
> > about 1K each message size), that made the GC threads taking over every
> > thing, the java process was so stuck that the jboss-shutdown command
> could
> > not bring it down.
> >
> > And my trace log (memory profiling) showed that one instance of both
> > MessageContext and Axis2MessageContext were leaked for every message that
> is
> > delivered/processed.
> >
> > I am using 1.2 GA of synapse. And I am running it in JBoss.
> >
> > And will be more than happy to send the thread dump, but that is very
> big.
> >
> > Once again thanks for getting back to me and I am looking for good
> pointers
> > from you very soon.
> >
> > Regards,
> > Abid
> >
> >
> >
> >
> > -----Original Message-----
> > From: Paul Fremantle [mailto:pzfreo@gmail.com]
> > Sent: Wednesday, May 18, 2011 8:39 PM
> > To: user@synapse.apache.org
> > Subject: Re: Please help: does Synapse leak memory?
> >
> > Abid
> >
> > Can you give us some idea of the flows? I know that Synapse is usually
> > memory safe - I have thread dumps from people who have pumped 134
> > terabytes of data, 1.4 billion messages and all this with a max heap
> > size of <2Gb and no leaks.
> >
> > Paul
> >
> > On Thu, May 19, 2011 at 1:49 AM, Abid Khan-EXT <Abid.Khan@jeppesen.com>
> > wrote:
> > > Hello all,
> > >
> > > I am troubleshooting an instance where I had used Synapse, the issue is
> > that after delivering few thousand message server runs out of memory.
> > >
> > > I investigated and found out that for each message that is delivered
> > Synapse is leaking following two objects:
> > >
> > > 1) org.apache.axis2.context.MessageContext
> > >
> > > 2) org.apache.synapse.core.axis2.Axis2MessageContext
> > >
> > > I thought that it might be related to threads and ThreadLocal, but when
> I
> > scanned the thread local of all 160+ threads those objects were not
> there.
> > >
> > > It seems that synapse is keeping those two object in its some internal
> > object, that is difficult to find.
> > >
> > > Because of this problem those objects; server leaks memory at very high
> > rate, and freezes after few hours of running?
> > >
> > > Any immediate response will be appreciated,
> > >
> > >
> > > Thanks,
> > > Abid
> > >
> >
> >
> >
> > --
> > Paul Fremantle
> > Co-Founder and CTO, WSO2
> > Apache Synapse PMC Chair
> > OASIS WS-RX TC Co-chair
> >
> > blog: http://pzf.fremantle.org
> > paul@wso2.com
> >
> > "Oxygenating the Web Service Platform", www.wso2.com
> >
>
>
>
> --
> Hiranya Jayathilaka
> Senior Software Engineer;
> WSO2 Inc.;  http://wso2.org
> E-mail: hiranya@wso2.com;  Mobile: +94 77 633 3491
> Blog: http://techfeast-hiranya.blogspot.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message