fineract-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Binny Gopinath Sreevas <binny.gopin...@gmail.com>
Subject Re: [jira] [Updated] (FINERACT-135) Performance improvement using EHCache or any other cache managers
Date Thu, 07 Apr 2016 11:53:55 GMT
Keith, Markus, Adi,

Accounting is enabled. And it is synchronously tied to loan transactions. I
am looking at Catalina.out. Hence, I don't think it could be network
latency.

Fetch of a loan takes about 20+ seconds  and about another 8-9 seconds for
fetching the data tables associated to the loan.
(Fyi, each loan has 2-3 fees, and about 52 entries in repayment schedule -
and hence, potentially that many transactions and journal entries as the
client makes repayments)
The GET is being called with parameters
"associations=all&exclude=guarantors".

The collection sheet APIs -
Generate collection sheet takes about 18 seconds
The saving of the collection sheet takes about - . 60+ seconds

Examples:
GET of loan:
{"startTime":1460024693065,"totalTime":16912,"method":"GET",
"url":"<<XXXX>>/mifosng-provider/api/v1/loans/295087","parameters":{"associations":["all"],"exclude":["guarantors"]}}

GET of data tables associated to Loans:
{"startTime":1460024693441,"totalTime":9609,"method":"GET",
"url":"<<XXXX>>/mifosng-provider/api/v1/datatables/PPI%20Details/295087","parameters":{"genericResultSet":["true"]}}

POST of loans:
{"startTime":1460024697085,"totalTime":17197,"method":"POST",
"url":"<<XXXX>>/mifosng-provider/api/v1/loans","parameters":{}}

Generate collection sheet:
{"startTime":1460029687320,"totalTime":15564,"method":"POST",
"url":"<<XXXX>>/mifosng-provider/api/v1/centers/50689","parameters":{"command":["generateCollectionSheet"]}}

Save collection sheet:
{"startTime":1460024733759,"totalTime":69372,"method":"POST",
"url":"<<XXXX>>/mifosng-provider/api/v1/centers/64305","parameters":{"command":["saveCollectionSheet"]}}

Note: This is still MIFOS code deployed and not yet upgraded to FINERACT -
which will happen in couple of weeks. I am not expecting FINERACT
performance to be mcuh different.

The environment is: tomcat running on a Linux box with 16GB RAM and
database on a separate box with 32GB RAM. Both of them on AWS.

- Binny



On Thu, Apr 7, 2016 at 3:58 PM, Adi Raju <adi.raju@confluxtechnologies.com>
wrote:

> As the loan portfolio size increases, performance of most of loan
> transaction APIs is seen to be slow, more specifically collection sheet and
> repayments.
> In any loan related transaction, Loan entity object is created. Initial
> observation is that creation of this Loan object is resulting in around
> 40-50 select queries to DB.
> Also there are so many other tables like office/user/GLaccounts/code values
> etc which rarely change but again result in queries to DB.
>
> Hibernate 2nd Level cache and possibly JOIN FETCH strategies could help to
> start with.
>
> Regards,
> Adi
>
>
> -----Original Message-----
> From: Markus GeiƟ [mailto:markus.geiss@live.de]
> Sent: 07 April 2016 15:46
> To: dev@fineract.incubator.apache.org
> Subject: RE: [jira] [Updated] (FINERACT-135) Performance improvement using
> EHCache or any other cache managers
>
> Hey,
> as far as I know EHCache is used to enable 2nd levelcache for Hibernate, so
> a per node cache.
> If we want upfront distributed caching I'd vote forHazelcast, because we'd
> need to implement it anyways.
> I back Keith' question, it would be good to know whatthe issue is to
> understand the needed adjustments.
> Is the bottleneck on the service side, is it DB related,or maybe the
> network. Where we loose performance here?
>
> Best,
>
> Markus
>
> .::YAGNI likes a DRY KISS::.
>
> > Date: Thu, 7 Apr 2016 10:04:16 +0100
> > Subject: Re: [jira] [Updated] (FINERACT-135) Performance improvement
> > using EHCache or any other cache managers
> > From: keithwoodlock@gmail.com
> > To: dev@fineract.incubator.apache.org
> >
> > Binny,
> >
> > >> In a production environment wiith about 130,000 clients with loans
> > >> - the
> > performance of the loan APIs are poor.
> >
> > This statement is too vague. What functionality of loans is said to
> > have poor performance. Is it the fetch part of loans of the
> > transactions that happen over the loans like make payment etc which if
> > I remember interact closely with the Accounting area. Is accounting
> > turned on/configured when performance is poor? Is it 'synchronously'
> > coupled to the loans functionality as part of loan transactions.
> >
> > Caches will only help on the GET/Fetch side of things.
> >
> > Is distributed caching supported on Mifos using EHcache or is it just
> > single node cache? Some like Hazelcast or similiar might be a good
> > option if wanting to support distributed caching.
> >
> > regards,
> > Keith.
> >
> > On Thu, Apr 7, 2016 at 9:57 AM, Binny Gopinath Sreevas (JIRA) <
> > jira@apache.org> wrote:
> >
> > >
> > >      [
> > > https://issues.apache.org/jira/browse/FINERACT-135?page=com.atlassia
> > > n.jira.plugin.system.issuetabpanels:all-tabpanel
> > > ]
> > >
> > > Binny Gopinath Sreevas updated FINERACT-135:
> > > --------------------------------------------
> > >     Description:
> > > In a production environment wiith about 130,000 clients with loans -
> > > the performance of the loan APIs are poor. I have been told is that
> > > enabling EHCache and making it work will definitely improve the
> > > performance. Can this be done and tested for deployment scenarios -
> > > including multi-tenant setup and load balanced (multiple
> > > tomcats/FINERACT instances which talk to the same database).
> > >
> > > Reference to previous work on this:
> > >
> > > https://mifosforge.jira.com/wiki/display/projects/GSOC+2013+-+Distri
> > > buted+Caching+support
> > >
> > >   was:
> > > In a production environment wiith about 130,000 clients with loans -
> > > the performance of the loan APIs are poor. I have been told is that
> > > enabling EHCache and making it work will definitely improve the
> > > performance. Can this be done and tested for deployment scenarios -
> > > including multi-tenant setup and load balanced (multiple
> > > tomcats/FINERACT instances which talk to the same database).
> > >
> > >
> > >
> > >
> > > > Performance improvement using EHCache or any other cache managers
> > > > -----------------------------------------------------------------
> > > >
> > > >                 Key: FINERACT-135
> > > >                 URL:
> https://issues.apache.org/jira/browse/FINERACT-135
> > > >             Project: Apache Fineract
> > > >          Issue Type: Improvement
> > > >            Reporter: Binny Gopinath Sreevas
> > > >            Assignee: Markus Geiss
> > > >            Priority: Critical
> > > >
> > > > In a production environment wiith about 130,000 clients with loans
> > > > - the
> > > performance of the loan APIs are poor. I have been told is that
> > > enabling EHCache and making it work will definitely improve the
> > > performance. Can this be done and tested for deployment scenarios -
> > > including multi-tenant setup and load balanced (multiple
> > > tomcats/FINERACT instances which talk to the same database).
> > > > Reference to previous work on this:
> > > >
> > > https://mifosforge.jira.com/wiki/display/projects/GSOC+2013+-+Distri
> > > buted+Caching+support
> > >
> > >
> > >
> > > --
> > > This message was sent by Atlassian JIRA
> > > (v6.3.4#6332)
> > >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message