airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chathuri Wimalasena <kamalas...@gmail.com>
Subject Re: Implementing a cache registry
Date Fri, 30 May 2014 19:44:44 GMT
I did some performance testing with JConsole. Attaching two images with
caching and without. I ran getAllUserExperiments 500 times with 180
experiments in the database.  To run 500 iterations with caching is like 5
mins but without caching it took more than a hour. In both cases memory and
cpu usages are not much difference as I feel. But it seems we have a memory
leak somewhere in our code since memory usage is increasing over the time.
I will dig more into it.


On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <marpierc@iu.edu> wrote:

> +1 very nice improvement.
> On 5/29/14 2:41 PM, Saminda Wijeratne wrote:
> > wow.... thats great news... Could you also monitor memory and CPU usage
> > impact? See also if there's any drawbacks that'll affect us for using
> > openJPA caching.
> >
> >
> > On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake <glahiru@gmail.com>
> > wrote:
> >
> >> Sounds great Chathuri, I think we can tweak the caching configuration of
> >> openJPA and find out the best caching configuration and publish it in
> wiki.
> >>
> >> If you can make the caching parameters configurable in
> >> airavata-server.properties that would be great.
> >>
> >> Regards
> >> Lahiru
> >>
> >>
> >> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena <
> kamalasini@gmail.com
> >>> wrote:
> >>> I enabled openJPA caching and the results are pretty impressive. I have
> >>> 180 experiments in default derby database. I query
> getAllUserExperiments
> >>> 100 times and average time is around 7000 - 8000 milliseconds without
> >>> enabling caching. After enabling caching that number reduced to
> average of
> >>> 400 - 500 milliseconds.
> >>>
> >>>
> >>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena <
> >>> kamalasini@gmail.com> wrote:
> >>>
> >>>> We can try openJPA caching [1] and see how it improves the
> performance.
> >>>> This will have minimal changes to the existing code.
> >>>>
> >>>> [1]
> >>>>
> http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html
> >>>>
> >>>>
> >>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva <
> supun06@gmail.com
> >>>>> wrote:
> >>>>> I'm not 100% percent sure on the architecture and the requirements.
> May
> >>>>> be you should look in to Redis or Memcache.
> >>>>>
> >>>>> Thanks,
> >>>>> Supun..
> >>>>>
> >>>>>
> >>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake <
> glahiru@gmail.com
> >>>>>> wrote:
> >>>>>> I think we can implement this from the scratch and won't be
a hard
> >>>>>> thing to do.
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne <
> >>>>>> samindaw@gmail.com> wrote:
> >>>>>>
> >>>>>>> +1.
> >>>>>>>
> >>>>>>> Do you know any good frameworks which supports this? IMO
the tricky
> >>>>>>> part is when to know that the data in cache has expired.
> Jackrabbit had a
> >>>>>>> feature where it does a callback whenever something gets
updated
> in some
> >>>>>>> tree path. If we restrict all registry access through a
single
> >>>>>>> component/layer instance we would be able to do the same
by a
> subscription
> >>>>>>> pattern.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake <
> >>>>>>> glahiru@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi Devs,
> >>>>>>>>
> >>>>>>>> In our current implementation we have large number of
Experiment
> >>>>>>>> retrieval and experiment storing happen in between experiment
> creation and
> >>>>>>>> experiment completion. We do not really parse these
data-model
> objects
> >>>>>>>> between component and we simply parse the ids of these
experiment
> so every
> >>>>>>>> component has to retrieve them everytime. I think programatically
> this
> >>>>>>>> approach looks much cleaner than parsing big objects.
But to make
> this more
> >>>>>>>> efficient we can use a cachedRegistry implementation
as another
> >>>>>>>> implementation of registry and make sure we do not get
objects
> all the way
> >>>>>>>> from the database.
> >>>>>>>>
> >>>>>>>> Each component can init its own cache registry object
and it will
> >>>>>>>> build a cache on that module and update the cache if
some other
> component
> >>>>>>>> had changed the data-model objects. IMHO if we implement
a good
> caching
> >>>>>>>> layer on our current data-model airavata registry will
be more
> efficient.
> >>>>>>>>
> >>>>>>>> WDYT ?
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Lahiru
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> System Analyst Programmer
> >>>>>>>> PTI Lab
> >>>>>>>> Indiana University
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> System Analyst Programmer
> >>>>>> PTI Lab
> >>>>>> Indiana University
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Supun Kamburugamuva
> >>>>> Member, Apache Software Foundation; http://www.apache.org
> >>>>> E-mail: supun06@gmail.com;  Mobile: +1 812 369 6762
> >>>>> Blog: http://supunk.blogspot.com
> >>>>>
> >>>>>
> >>
> >> --
> >> System Analyst Programmer
> >> PTI Lab
> >> Indiana University
> >>
>
>

Mime
View raw message