Mailing-List: contact dev-help@continuum.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@continuum.apache.org
Received-SPF: pass (athena.apache.org: domain of odeaching@gmail.com
 designates 74.125.92.148 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=EPtl0z+9BxupDoic/VCRi30Ca5teOnb0T3zF0mQ512kniM0F4z3pN0NYPgvYG6SQyj
         RlXxLW/s5q9nA1vuRjWZ7klFKjHwQQOpPAupw86gnQm1xdSPLjMNOnqGvmucSC0G/ZOk
         a+zxN1Hn4Wr+d416ynBkDeHiw7vMYG4ES+EuI=
MIME-Version: 1.0
Sender: odeaching@gmail.com
In-Reply-To: <adba96190906151351l60da30e8v3fe60dae66332b3f@mail.gmail.com>
References: <adba96190906150939n34d6e233h6ac2f55014ae3ba3@mail.gmail.com>
	 <f345f3480906151158nef6ea5bi220256e5e539722d@mail.gmail.com>
	 <adba96190906151351l60da30e8v3fe60dae66332b3f@mail.gmail.com>
Date: Tue, 16 Jun 2009 10:25:26 +0800
Message-ID: <8667b1bd0906151925y12735d50kaabc6b7bc47c6041@mail.gmail.com>
Subject: Re: Adding project groups takes longer and longer
From: Deng Ching <oching@apache.org>
To: dev@continuum.apache.org
Content-Type: multipart/alternative; boundary=0016360e3d183fd482046c6de34f

--0016360e3d183fd482046c6de34f
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Tue, Jun 16, 2009 at 4:51 AM, Wendy Smoak <wsmoak@gmail.com> wrote:

> The actual system with the performance problem is using MySQL, however
> these results and the graph were from a fresh 1.3.3 instance using the
> embedded Derby database.
>
> Another use case that is slow is simply logging in.  I've personally
> timed it at 3 minutes from clicking the button to log in, before I see
> the next page rendered.
>
> Brett suggested turning on the JPOX sql query logging, and here's what
> I see while logging in as admin:
>
> Continuum 1.3.3 with 20 project groups - 3,192 SELECT queries
> Continuum 1.3.3 with 400 project groups - 60,184 SELECT queries
>
> It's not especially slow to log in as admin (5-6 seconds) with 400
> project groups, but this is with an embedded Derby db, so the queries
> aren't going out on the network.
>
> In any case, why does it need to do *that* many queries simply to
> process a login?


One of these many queries I saw yesterday from the logs was due to the cache
being reset during each login. The cache is cleared first then all the
roles/permissions are queried and added to the cache. Since we're using
resource-based roles, it takes longer to process the login if there are a
lot of resources (e.g. projects)..

- Deng


>
>
> --
> Wendy
>
> On Mon, Jun 15, 2009 at 11:58 AM, Emmanuel
> Venisse<emmanuel.venisse@gmail.com> wrote:
> > It's very bad.
> >
> > Do you use  an embedded derby DB or an external? I don't think it is good
> to
> > use an embedded DB for a "big" DB.
> >
> > Emmanuel
> >
> > On Mon, Jun 15, 2009 at 6:39 PM, Wendy Smoak <wsmoak@gmail.com> wrote:
> >
> >> I've noticed Continuum getting slower and slower... we're trying to
> >> track down exactly why, but one theory relates to there being lots of
> >> user roles that have to be checked, which would mean there are lots of
> >> project groups.
> >>
> >> To try to reproduce that, I added 400 groups to a fresh Continuum
> >> 1.3.3 instance.  I notice that it takes longer and longer to add each
> >> group.  Here's a graph:
> >>
> >>
> http://people.apache.org/~wsmoak/continuum/continuum-1.3.3-add-400-groups.png<http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png>
> <
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> >
> >>
> >> On trunk it's similar.  There in the audit log I can see there are 3
> >> seconds between adding the second and third groups, and 19 seconds
> >> between groups 399 and 400.  All in all it takes over an hour to add
> >> 400 groups to an instance.
> >>
> >> The test code is in the sandbox:
> >>
> >>
> http://svn.apache.org/repos/asf/continuum/sandbox/continuum-webapp-test-many-roles/
> >>
> >> Any ideas?  Are we missing an index on some field?
> >>
> >> --
> >> Wendy
> >>
> >
>

--0016360e3d183fd482046c6de34f--