lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shay Banon <>
Subject Re: FieldCacheImpl concurrency
Date Fri, 12 Feb 2010 06:50:17 GMT
On Thu, Feb 11, 2010 at 5:41 PM, Yonik Seeley <>wrote:

> On Thu, Feb 11, 2010 at 9:54 AM, Shay Banon <> wrote:
> >    I would like to try and improve concurrency in Lucene in several
> places,
> > and thought I would start with FieldCacheImpl. The implementation is
> heavily
> > synchronized on both a global map and on creation values for a pretty
> > heavily used path (get).
> It really shouldn't be heavily used.
> For a sorted search, get() is called once per segment in the index.
> There is no synchronization to retrieve per-document values.

Sorting is not the only reason why FieldCache would be used...

> We have just barely moved to Java5 though, and so it would probably be
> pretty easy to improve the concurrency of the read path if it did
> become problematic.

I know you just moved to Java 5, and my first email talked about the
concurrency aspect.

> > I think the weak hash map is a good solution (but
> > also think that the ability to choose to use soft hash map would add even
> > greater flexibility), the problem is that there is no built in concurrent
> > weak/soft hash map.
> It might be a better idea to have interfaces that allow one to
> implement their own policies rather than to try and push all the
> different options into Lucene:
I think that trying to provide an extension that either syncs or not is a
pain in terms of extension point (it will be very convoluted) and many times
hinders the optimizations that can be done in concurrent systems... . Just
see what is done in the FieldCacheImpl and the creation marker which is not
really needed with google collections (they allow for single entry be
created atomically).

> -Yonik
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message