airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Supun Kamburugamuva <supu...@gmail.com>
Subject Re: Implementing a cache registry
Date Fri, 30 May 2014 19:50:14 GMT
You can use Eclipse memory analyzer for finding the leak.I find it much
easier than JProfiler.

http://www.eclipse.org/mat/

Thanks,
Supun..


On Fri, May 30, 2014 at 3:44 PM, Chathuri Wimalasena <kamalasini@gmail.com>
wrote:

> I did some performance testing with JConsole. Attaching two images with
> caching and without. I ran getAllUserExperiments 500 times with 180
> experiments in the database.  To run 500 iterations with caching is like 5
> mins but without caching it took more than a hour. In both cases memory and
> cpu usages are not much difference as I feel. But it seems we have a memory
> leak somewhere in our code since memory usage is increasing over the time.
> I will dig more into it.
>
>
> On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <marpierc@iu.edu> wrote:
>
>> +1 very nice improvement.
>> On 5/29/14 2:41 PM, Saminda Wijeratne wrote:
>> > wow.... thats great news... Could you also monitor memory and CPU usage
>> > impact? See also if there's any drawbacks that'll affect us for using
>> > openJPA caching.
>> >
>> >
>> > On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake <glahiru@gmail.com
>> >
>> > wrote:
>> >
>> >> Sounds great Chathuri, I think we can tweak the caching configuration
>> of
>> >> openJPA and find out the best caching configuration and publish it in
>> wiki.
>> >>
>> >> If you can make the caching parameters configurable in
>> >> airavata-server.properties that would be great.
>> >>
>> >> Regards
>> >> Lahiru
>> >>
>> >>
>> >> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena <
>> kamalasini@gmail.com
>> >>> wrote:
>> >>> I enabled openJPA caching and the results are pretty impressive. I
>> have
>> >>> 180 experiments in default derby database. I query
>> getAllUserExperiments
>> >>> 100 times and average time is around 7000 - 8000 milliseconds without
>> >>> enabling caching. After enabling caching that number reduced to
>> average of
>> >>> 400 - 500 milliseconds.
>> >>>
>> >>>
>> >>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena <
>> >>> kamalasini@gmail.com> wrote:
>> >>>
>> >>>> We can try openJPA caching [1] and see how it improves the
>> performance.
>> >>>> This will have minimal changes to the existing code.
>> >>>>
>> >>>> [1]
>> >>>>
>> http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html
>> >>>>
>> >>>>
>> >>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva <
>> supun06@gmail.com
>> >>>>> wrote:
>> >>>>> I'm not 100% percent sure on the architecture and the requirements.
>> May
>> >>>>> be you should look in to Redis or Memcache.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Supun..
>> >>>>>
>> >>>>>
>> >>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake <
>> glahiru@gmail.com
>> >>>>>> wrote:
>> >>>>>> I think we can implement this from the scratch and won't
be a hard
>> >>>>>> thing to do.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne <
>> >>>>>> samindaw@gmail.com> wrote:
>> >>>>>>
>> >>>>>>> +1.
>> >>>>>>>
>> >>>>>>> Do you know any good frameworks which supports this?
IMO the
>> tricky
>> >>>>>>> part is when to know that the data in cache has expired.
>> Jackrabbit had a
>> >>>>>>> feature where it does a callback whenever something
gets updated
>> in some
>> >>>>>>> tree path. If we restrict all registry access through
a single
>> >>>>>>> component/layer instance we would be able to do the
same by a
>> subscription
>> >>>>>>> pattern.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake
<
>> >>>>>>> glahiru@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Devs,
>> >>>>>>>>
>> >>>>>>>> In our current implementation we have large number
of Experiment
>> >>>>>>>> retrieval and experiment storing happen in between
experiment
>> creation and
>> >>>>>>>> experiment completion. We do not really parse these
data-model
>> objects
>> >>>>>>>> between component and we simply parse the ids of
these
>> experiment so every
>> >>>>>>>> component has to retrieve them everytime. I think
>> programatically this
>> >>>>>>>> approach looks much cleaner than parsing big objects.
But to
>> make this more
>> >>>>>>>> efficient we can use a cachedRegistry implementation
as another
>> >>>>>>>> implementation of registry and make sure we do not
get objects
>> all the way
>> >>>>>>>> from the database.
>> >>>>>>>>
>> >>>>>>>> Each component can init its own cache registry object
and it will
>> >>>>>>>> build a cache on that module and update the cache
if some other
>> component
>> >>>>>>>> had changed the data-model objects. IMHO if we implement
a good
>> caching
>> >>>>>>>> layer on our current data-model airavata registry
will be more
>> efficient.
>> >>>>>>>>
>> >>>>>>>> WDYT ?
>> >>>>>>>>
>> >>>>>>>> Regards
>> >>>>>>>> Lahiru
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> System Analyst Programmer
>> >>>>>>>> PTI Lab
>> >>>>>>>> Indiana University
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> System Analyst Programmer
>> >>>>>> PTI Lab
>> >>>>>> Indiana University
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Supun Kamburugamuva
>> >>>>> Member, Apache Software Foundation; http://www.apache.org
>> >>>>> E-mail: supun06@gmail.com;  Mobile: +1 812 369 6762
>> >>>>> Blog: http://supunk.blogspot.com
>> >>>>>
>> >>>>>
>> >>
>> >> --
>> >> System Analyst Programmer
>> >> PTI Lab
>> >> Indiana University
>> >>
>>
>>
>


-- 
Supun Kamburugamuva
Member, Apache Software Foundation; http://www.apache.org
E-mail: supun06@gmail.com;  Mobile: +1 812 369 6762
Blog: http://supunk.blogspot.com

Mime
View raw message