continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deng Ching <och...@apache.org>
Subject Re: Adding project groups takes longer and longer
Date Tue, 16 Jun 2009 11:38:29 GMT
More additional info..

I tried using the JDO user manager implementation instead of the cached user
manager implementation (which is using the ehcache configured in Redback)
and I didn't see much improvement on the queries during login so it's
possible that the jpox caching is the main cause of all the queries. I'll
investigate this further tomorrow.

Thanks,
Deng

On Tue, Jun 16, 2009 at 5:45 PM, Emmanuel Venisse <
emmanuel.venisse@gmail.com> wrote:

> On Tue, Jun 16, 2009 at 11:26 AM, Deng Ching <oching@apache.org> wrote:
>
> > On Tue, Jun 16, 2009 at 4:50 PM, Emmanuel Venisse <
> > emmanuel.venisse@gmail.com> wrote:
> >
> > > On Mon, Jun 15, 2009 at 10:56 PM, Wendy Smoak <wsmoak@gmail.com>
> wrote:
> > >
> > > > On Mon, Jun 15, 2009 at 9:39 AM, Wendy Smoak<wsmoak@gmail.com>
> wrote:
> > > >
> > > > > To try to reproduce that, I added 400 groups to a fresh Continuum
> > > > > 1.3.3 instance.  I notice that it takes longer and longer to add
> each
> > > > > group.  Here's a graph:
> > > > >
> > > >
> > >
> >
> http://people.apache.org/~wsmoak/continuum/continuum-1.3.3-add-400-groups.png<http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png>
> <
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> >
> > <
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > >
> > > <
> > >
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > > >
> > > >
> > > > Back to the "too many queries" idea, I found that
> > > >  - adding group401 resulted in 61,144 queries
> > > >  - adding group402 restulted in 61,278 queries
> > >
> > >
> > > I'm very surprised by this numbers. We must work on the cache to not
> > reload
> > > all the cache content each time.
> > > Actually the cache is reloaded every 4hours except for userPermissions
> > and
> > > userAssignments that are reloaded every 10 minutes.
> >
> >
> > Aside from these caches configured in Redback, JPOX also has it's own
> Level
> > 1 caching (which cannot be turned off) and that also contributes to the
> > queries. This caching can be switched to Level 2 caching though and JPOX
> > has
> > support for Ehcache in that level. Since Redback already uses Ehcache,
> I'm
> > trying to figure out if it's possible (and how) to configure JPOX to use
> > the
> > ehcache configured in Redback so that caching would only happen once.
>
>
> It is possible to implement our own level2 cache for JPOX (
> http://www.jpox.org/docs/1_2/extensions/level2_cache.html ) and use the
> cache manager defined in redback, but I'm not sure it is a good idea. If we
> want to use a Levle2 cache, I think it would be better to use the standard
> level2 ehcache plugin for all jpox request and keep our own ehcache in
> redback to keep in memory pre-compute data. I don't consider both caches at
> the same level.
>
>
> >
> >
> >
> > >
> > >
> > > The cache must not be totally reloaded without an expiration, if a data
> > > must
> > > be reloaded, only this one must be removed/readded in the cache.
> > >
> > > For the groupSummary page, Continuum do some request to get the
> content:
> > > - 1 request to get the project group list
> > > - 1 request by project group to get the summary
> > > - 1 request by project group to get the local repository informations
> > >
> > > So for 400 project groups, Continuum run 1 + 400 + 400 = 801 requests.
> I
> > > think it would be good to run run less requests, maybe we can use
> ehcache
> > > here to run 0 requests.
> > >
> > > For each web request, Redback do a select to see if an authorization
> key
> > > (seesion id) exist and if not (it never exist) it insert a new entry in
> > the
> > > table. I don't think it is good.
> >
> >
> > +1
> >
> >
> > > Emmanuel
> > >
> >
> > Thanks,
> > Deng
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message