continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Venisse <emmanuel.veni...@gmail.com>
Subject Re: Adding project groups takes longer and longer
Date Tue, 16 Jun 2009 09:45:29 GMT
On Tue, Jun 16, 2009 at 11:26 AM, Deng Ching <oching@apache.org> wrote:

> On Tue, Jun 16, 2009 at 4:50 PM, Emmanuel Venisse <
> emmanuel.venisse@gmail.com> wrote:
>
> > On Mon, Jun 15, 2009 at 10:56 PM, Wendy Smoak <wsmoak@gmail.com> wrote:
> >
> > > On Mon, Jun 15, 2009 at 9:39 AM, Wendy Smoak<wsmoak@gmail.com> wrote:
> > >
> > > > To try to reproduce that, I added 400 groups to a fresh Continuum
> > > > 1.3.3 instance.  I notice that it takes longer and longer to add each
> > > > group.  Here's a graph:
> > > >
> > >
> >
> http://people.apache.org/~wsmoak/continuum/continuum-1.3.3-add-400-groups.png<http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png>
> <
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> >
> > <
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > >
> > >
> > > Back to the "too many queries" idea, I found that
> > >  - adding group401 resulted in 61,144 queries
> > >  - adding group402 restulted in 61,278 queries
> >
> >
> > I'm very surprised by this numbers. We must work on the cache to not
> reload
> > all the cache content each time.
> > Actually the cache is reloaded every 4hours except for userPermissions
> and
> > userAssignments that are reloaded every 10 minutes.
>
>
> Aside from these caches configured in Redback, JPOX also has it's own Level
> 1 caching (which cannot be turned off) and that also contributes to the
> queries. This caching can be switched to Level 2 caching though and JPOX
> has
> support for Ehcache in that level. Since Redback already uses Ehcache, I'm
> trying to figure out if it's possible (and how) to configure JPOX to use
> the
> ehcache configured in Redback so that caching would only happen once.


It is possible to implement our own level2 cache for JPOX (
http://www.jpox.org/docs/1_2/extensions/level2_cache.html ) and use the
cache manager defined in redback, but I'm not sure it is a good idea. If we
want to use a Levle2 cache, I think it would be better to use the standard
level2 ehcache plugin for all jpox request and keep our own ehcache in
redback to keep in memory pre-compute data. I don't consider both caches at
the same level.


>
>
>
> >
> >
> > The cache must not be totally reloaded without an expiration, if a data
> > must
> > be reloaded, only this one must be removed/readded in the cache.
> >
> > For the groupSummary page, Continuum do some request to get the content:
> > - 1 request to get the project group list
> > - 1 request by project group to get the summary
> > - 1 request by project group to get the local repository informations
> >
> > So for 400 project groups, Continuum run 1 + 400 + 400 = 801 requests. I
> > think it would be good to run run less requests, maybe we can use ehcache
> > here to run 0 requests.
> >
> > For each web request, Redback do a select to see if an authorization key
> > (seesion id) exist and if not (it never exist) it insert a new entry in
> the
> > table. I don't think it is good.
>
>
> +1
>
>
> > Emmanuel
> >
>
> Thanks,
> Deng
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message