continuum-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Venisse <emmanuel.veni...@gmail.com>
Subject Re: Adding project groups takes longer and longer
Date Tue, 16 Jun 2009 11:41:20 GMT
Thanks Deng

Emmanuel

On Tue, Jun 16, 2009 at 1:38 PM, Deng Ching <oching@apache.org> wrote:

> More additional info..
>
> I tried using the JDO user manager implementation instead of the cached
> user
> manager implementation (which is using the ehcache configured in Redback)
> and I didn't see much improvement on the queries during login so it's
> possible that the jpox caching is the main cause of all the queries. I'll
> investigate this further tomorrow.
>
> Thanks,
> Deng
>
> On Tue, Jun 16, 2009 at 5:45 PM, Emmanuel Venisse <
> emmanuel.venisse@gmail.com> wrote:
>
> > On Tue, Jun 16, 2009 at 11:26 AM, Deng Ching <oching@apache.org> wrote:
> >
> > > On Tue, Jun 16, 2009 at 4:50 PM, Emmanuel Venisse <
> > > emmanuel.venisse@gmail.com> wrote:
> > >
> > > > On Mon, Jun 15, 2009 at 10:56 PM, Wendy Smoak <wsmoak@gmail.com>
> > wrote:
> > > >
> > > > > On Mon, Jun 15, 2009 at 9:39 AM, Wendy Smoak<wsmoak@gmail.com>
> > wrote:
> > > > >
> > > > > > To try to reproduce that, I added 400 groups to a fresh Continuum
> > > > > > 1.3.3 instance.  I notice that it takes longer and longer to
add
> > each
> > > > > > group.  Here's a graph:
> > > > > >
> > > > >
> > > >
> > >
> >
> http://people.apache.org/~wsmoak/continuum/continuum-1.3.3-add-400-groups.png<http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png>
> <
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> >
> > <
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > >
> > > <
> > >
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > > >
> > > > <
> > > >
> > >
> >
> http://people.apache.org/%7Ewsmoak/continuum/continuum-1.3.3-add-400-groups.png
> > > > >
> > > > >
> > > > > Back to the "too many queries" idea, I found that
> > > > >  - adding group401 resulted in 61,144 queries
> > > > >  - adding group402 restulted in 61,278 queries
> > > >
> > > >
> > > > I'm very surprised by this numbers. We must work on the cache to not
> > > reload
> > > > all the cache content each time.
> > > > Actually the cache is reloaded every 4hours except for
> userPermissions
> > > and
> > > > userAssignments that are reloaded every 10 minutes.
> > >
> > >
> > > Aside from these caches configured in Redback, JPOX also has it's own
> > Level
> > > 1 caching (which cannot be turned off) and that also contributes to the
> > > queries. This caching can be switched to Level 2 caching though and
> JPOX
> > > has
> > > support for Ehcache in that level. Since Redback already uses Ehcache,
> > I'm
> > > trying to figure out if it's possible (and how) to configure JPOX to
> use
> > > the
> > > ehcache configured in Redback so that caching would only happen once.
> >
> >
> > It is possible to implement our own level2 cache for JPOX (
> > http://www.jpox.org/docs/1_2/extensions/level2_cache.html ) and use the
> > cache manager defined in redback, but I'm not sure it is a good idea. If
> we
> > want to use a Levle2 cache, I think it would be better to use the
> standard
> > level2 ehcache plugin for all jpox request and keep our own ehcache in
> > redback to keep in memory pre-compute data. I don't consider both caches
> at
> > the same level.
> >
> >
> > >
> > >
> > >
> > > >
> > > >
> > > > The cache must not be totally reloaded without an expiration, if a
> data
> > > > must
> > > > be reloaded, only this one must be removed/readded in the cache.
> > > >
> > > > For the groupSummary page, Continuum do some request to get the
> > content:
> > > > - 1 request to get the project group list
> > > > - 1 request by project group to get the summary
> > > > - 1 request by project group to get the local repository informations
> > > >
> > > > So for 400 project groups, Continuum run 1 + 400 + 400 = 801
> requests.
> > I
> > > > think it would be good to run run less requests, maybe we can use
> > ehcache
> > > > here to run 0 requests.
> > > >
> > > > For each web request, Redback do a select to see if an authorization
> > key
> > > > (seesion id) exist and if not (it never exist) it insert a new entry
> in
> > > the
> > > > table. I don't think it is good.
> > >
> > >
> > > +1
> > >
> > >
> > > > Emmanuel
> > > >
> > >
> > > Thanks,
> > > Deng
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message