Mailing-List: contact dev-help@continuum.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@continuum.apache.org
Received-SPF: pass (nike.apache.org: domain of emmanuel.venisse@gmail.com
 designates 209.85.220.213 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=ezqW1x1vWNugxiYa2LmJUownFx/O0H+jPQUNFZHcGPQCXmKp+GQdRE0auVXpzjocIc
         vsYhMnA1ETOJieQrYZd0l14pLFu8jbQB+aMQW6gL4E+uhXskUCYIBsgnbZKHRNmRItQ9
         eTCImFcsUdsaaN63f5wm9xNJ5U7Ai/85ZEksI=
MIME-Version: 1.0
In-Reply-To: <8667b1bd0906160226w6b57a1aer5150859ddadf39b9@mail.gmail.com>
References: <adba96190906150939n34d6e233h6ac2f55014ae3ba3@mail.gmail.com>
	 <adba96190906151356t5d04501eqc8d0fa8f0c2747f7@mail.gmail.com>
	 <f345f3480906160150l540f5c96jd2bcfbce65a099f3@mail.gmail.com>
	 <8667b1bd0906160226w6b57a1aer5150859ddadf39b9@mail.gmail.com>
Date: Tue, 16 Jun 2009 11:45:29 +0200
Message-ID: <f345f3480906160245g3e54507ey3c95f765a9fa2ccb@mail.gmail.com>
Subject: Re: Adding project groups takes longer and longer
From: Emmanuel Venisse <emmanuel.venisse@gmail.com>
To: dev@continuum.apache.org
Content-Type: multipart/alternative; boundary=001636c59864f80ca8046c74084c

--001636c59864f80ca8046c74084c
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

On Tue, Jun 16, 2009 at 11:26 AM, Deng Ching <oching@apache.org> wrote:

> On Tue, Jun 16, 2009 at 4:50 PM, Emmanuel Venisse <
> emmanuel.venisse@gmail.com> wrote:
>
> > On Mon, Jun 15, 2009 at 10:56 PM, Wendy Smoak <wsmoak@gmail.com> wrote:
> >
> > > On Mon, Jun 15, 2009 at 9:39 AM, Wendy Smoak<wsmoak@gmail.com> wrote:
> > >
> > > > To try to reproduce that, I added 400 groups to a fresh Continuum
> > > > 1.3.3 instance.  I notice that it takes longer and longer to add each
> > > > group.  Here's a graph:
> > > >
> > >
> >
> http://people.apache.org/~wsmoak/continuum/continuum-1.3.3-add-400-groups.png<http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png>
> <
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> >
> > <
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > >
> > >
> > > Back to the "too many queries" idea, I found that
> > >  - adding group401 resulted in 61,144 queries
> > >  - adding group402 restulted in 61,278 queries
> >
> >
> > I'm very surprised by this numbers. We must work on the cache to not
> reload
> > all the cache content each time.
> > Actually the cache is reloaded every 4hours except for userPermissions
> and
> > userAssignments that are reloaded every 10 minutes.
>
>
> Aside from these caches configured in Redback, JPOX also has it's own Level
> 1 caching (which cannot be turned off) and that also contributes to the
> queries. This caching can be switched to Level 2 caching though and JPOX
> has
> support for Ehcache in that level. Since Redback already uses Ehcache, I'm
> trying to figure out if it's possible (and how) to configure JPOX to use
> the
> ehcache configured in Redback so that caching would only happen once.


It is possible to implement our own level2 cache for JPOX (
http://www.jpox.org/docs/1_2/extensions/level2_cache.html ) and use the
cache manager defined in redback, but I'm not sure it is a good idea. If we
want to use a Levle2 cache, I think it would be better to use the standard
level2 ehcache plugin for all jpox request and keep our own ehcache in
redback to keep in memory pre-compute data. I don't consider both caches at
the same level.


>
>
>
> >
> >
> > The cache must not be totally reloaded without an expiration, if a data
> > must
> > be reloaded, only this one must be removed/readded in the cache.
> >
> > For the groupSummary page, Continuum do some request to get the content:
> > - 1 request to get the project group list
> > - 1 request by project group to get the summary
> > - 1 request by project group to get the local repository informations
> >
> > So for 400 project groups, Continuum run 1 + 400 + 400 = 801 requests. I
> > think it would be good to run run less requests, maybe we can use ehcache
> > here to run 0 requests.
> >
> > For each web request, Redback do a select to see if an authorization key
> > (seesion id) exist and if not (it never exist) it insert a new entry in
> the
> > table. I don't think it is good.
>
>
> +1
>
>
> > Emmanuel
> >
>
> Thanks,
> Deng
>

--001636c59864f80ca8046c74084c--