isis-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From César Camilo Lugo Marcos <cesar.l...@sisorg.com.mx>
Subject Re: Concurrency
Date Tue, 31 May 2016 19:25:22 GMT
Correction, we are using Apache ISIS 1.11.1

Cesar.

On Tue, 2016-05-31 at 18:54 +0000, César Camilo Lugo Marcos wrote:
> Hello Dan. Answers to your questions:
> 
> - which version of Isis are you on?
>     1.12.0
> - how have you implemented the stress test?  is it automated, eg JMeter, or just adhoc
manual testing?
>     Using JMeter. We recorded a quite simple script with about 6 read only operations
using th Wicket viewer. All samplers are the recorded HTTP.
> - is this stress testing the app running locally on Tomcat, or up in the cloud (on Amazon
Elastic Beanstalk)?
>     We have it installed in the AWS Elastic Beanstalk with PostgreSQL.
>     We also made a test on a local host with Tomcat and MySQL, obtaining very similar
results.
> > - how much DB traffic does a single request create?  Is it all read-only; or are
there also writes?
>     Quite low traffic, About 6 read only operations per user.
> - for reads, are you seeing any N+1 issues?  if so, have you tried to prevent them using
DataNucleus eager loading hint (the "fetch groups" concept)?
>     Not sure what N+1 is. We are letting Datanucleous handle all foreign keys and primary
keys. We have not configured any fetch groups or any other configurations than defaults.
> - for writes, are they expected?
>     Did not understand this question. Please explain.
> - have you configured any of the various addons that could be writing to the DB, eg auditing,
command or publishing?  are these perhaps enabled for safe/query-only actions?  If so for
any looping within the app itself, can you eliminate using the QueryResultsCache?
>     We are only using the logging add-on, to log login and logout operations. No auditing,
command or publishing add ons.
> - what data volumes is this on?  If large volumes of data, are there perhaps missing
indices?  What's the performance like with small volumes of data?
>     No Data Object or table has more that a few tens of records.
> - is this for in-memory HSQLDB or out-of-memory DB (eg Postgres)?  What's the performance
like of one versus the other
>     This is PostgreSQL. Also tried MySQL with very similar results. Have not tried in
memory HSQLDB.
> 
> 
> Cesar.
> 
> 
> On Tue, 2016-05-31 at 18:53 +0100, Dan Haywood wrote:
> 
> 
> Hi Cesar,
> 
> Those action semantics don't play a large part in the transaction
> management; they are mostly used to determine which type of HTTP method to
> expose the action under (GET, PUT or POST) and there is also the constraint
> that contributed/mixin properties/collections can only be supplied by SAFE
> actions.  The _ARE_YOU_SURE variants only impact the Wicket viewer.  The
> _REQUEST_CACHEABLE is only honoured if the action is invoked via the
> wrapper factory.
> 
> As Martin and Jeroen have said, using JProfiler or similar will likely
> yield valuable info.
> 
> If the issue is with database concurrency (which does seem like a good
> hypothesis), then for the most part that is under the management of
> DataNucleus itself.  There are a bunch of configuration properties that can
> be specified in persistor_datanucleus.properties (or isis.properties,
> actually); we strip off any "isis.persistor.datanucleus.impl" prefix and
> pass the rest of the key down to DN to configure.
> 
> Questions for you:
> - which version of Isis are you on?
> - how have you implemented the stress test?  is it automated, eg JMeter, or
> just adhoc manual testing?
> - is this stress testing the app running locally on Tomcat, or up in the
> cloud (on Amazon Elastic Beanstalk)?
> - how much DB traffic does a single request create?  Is it all read-only;
> or are there also writes?
> - for reads, are you seeing any N+1 issues?  if so, have you tried to
> prevent them using DataNucleus eager loading hint (the "fetch groups"
> concept)?
> - for writes, are they expected?
> - have you configured any of the various addons that could be writing to
> the DB, eg auditing, command or publishing?  are these perhaps enabled for
> safe/query-only actions?  If so
> - for any looping within the app itself, can you eliminate using the
> QueryResultsCache?
> - what data volumes is this on?  If large volumes of data, are there
> perhaps missing indices?  What's the performance like with small volumes of
> data?
> - is this for in-memory HSQLDB or out-of-memory DB (eg Postgres)?  What's
> the performance like of one versus the other
> 
> I think what I would look for is fast response times for a 1-user system
> running locally on Tomcat, with an inmemory database, and low data
> volumes.  From there, ramp up in different ways, but change one thing at a
> time.
> 
> HTH
> Dan
> 
> 
> 
> On 31 May 2016 at 15:12, César Camilo Lugo Marcos <cesar.lugo@sisorg.com.mx<mailto:cesar.lugo@sisorg.com.mx>>
> wrote:
> 
> > Thank you both for the hint. I am looking into JProfiler. Still,
> > anything about the concurrency control options?
> >
> > Cesar.
> >
> > On Tue, 2016-05-31 at 11:13 +0200, Jeroen van der Wal wrote:
> > > I agree with Martin that profiling is the only way to go. To illustrate:
> > we
> > > recently made some code 8 times faster by a few simple code changes on
> > > bottlenecks revealed by JProfiler. And those were in places that we've
> > > never guessed.
> > >
> > > On 31 May 2016 at 08:39, Martin Grigorov <mgrigorov@apache.org<mailto:mgrigorov@apache.org>>
wrote:
> > >
> > > > Hi,
> > > >
> > > > To find out where is the problem you will have to use a profiler like
> > > > JProfiler, Yourkit, JVisualVM, etc.
> > > > Even some thread dumps would help to see what is going on.
> > > >
> > > > Martin Grigorov
> > > > Wicket Training and Consulting
> > > > https://twitter.com/mtgrigorov
> > > >
> > > > On Mon, May 30, 2016 at 9:00 PM, César Camilo Lugo Marcos <
> > > > cesar.lugo@sisorg.com.mx<mailto:cesar.lugo@sisorg.com.mx>> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I would like to add to this topic the following:
> > > > >
> > > > > Most of the transactions we are testing in these stress tests are
not
> > > > > bound in ACTIONS, but just plain reads or default transactions using
> > > > > Apache ISIS wicket viewer defaults. I don't see any place where I
> > could
> > > > > define semantics for default read or write transactions not bound
> > into
> > > > > ACTION methods. Is there any place I should look into for that?
> > > > >
> > > > > Cesar.
> > > > >
> > > > >
> > > > > On Mon, 2016-05-30 at 18:45 +0000, César Camilo Lugo Marcos wrote:
> > > > > > Hello,
> > > > > >
> > > > > > We have sarted performing some stress tests to our Apache ISIS
> > > > > > application. We have found this behavior:
> > > > > >
> > > > > > - If we run 1 concurrent user, average response times to the
main
> > > > object
> > > > > > reads through the wicket viewer are from 1 to 1.5 seconds.
> > > > > > - If we run 2 concurrent users, same transactions go to average
> > > > response
> > > > > > times up to 16 to 27 seconds.
> > > > > > - If we run 10 concurrent users, the transactions start to slow
> > down
> > > > > > significantly until the app server freezes and we have to restart
> > it.
> > > > > >
> > > > > > As you can see, this is a very significant increase in response
> > time
> > > > for
> > > > > > such a slight change in user load (from 1 user to 2 users).
So we
> > are
> > > > > > guessing we should look into the concurrency control.
> > > > > >
> > > > > > In the documentation I have found that the only way to influence
> > the
> > > > way
> > > > > > Apache ISIS manages transactions and concurrency checking is
by
> > using
> > > > > > the semantics configuration of the ACTION annotation.
> > > > > >
> > > > > > semantics=SAFE_AND_REQUEST_CACHEABLE
> > > > > > semantics=SAFE
> > > > > > semantics=IDEMPOTENT
> > > > > > semantics=IDEMPOTENT_ARE_YOU_SURE
> > > > > > semantics=NON_IDEMPOTENT
> > > > > > semantics=NON_IDEMPOTENT_ARE_YOU_SURE
> > > > > >
> > > > > > I just wanted to confirm if this is the place to look into,
or are
> > > > there
> > > > > > any other places where we should be looking into too, to solve
the
> > > > > > performance issue.
> > > > > >
> > > > > > Cesar.
> > > > >
> > > > >
> > > >
> >
> >
> 
> 

Mime
View raw message