lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shay Banon <kim...@gmail.com>
Subject Re: FieldCacheImpl concurrency
Date Fri, 12 Feb 2010 06:50:17 GMT
On Thu, Feb 11, 2010 at 5:41 PM, Yonik Seeley <yonik@lucidimagination.com>wrote:

> On Thu, Feb 11, 2010 at 9:54 AM, Shay Banon <kimchy@gmail.com> wrote:
> >    I would like to try and improve concurrency in Lucene in several
> places,
> > and thought I would start with FieldCacheImpl. The implementation is
> heavily
> > synchronized on both a global map and on creation values for a pretty
> > heavily used path (get).
>
> It really shouldn't be heavily used.
> For a sorted search, get() is called once per segment in the index.
> There is no synchronization to retrieve per-document values.
>

Sorting is not the only reason why FieldCache would be used...


>
> We have just barely moved to Java5 though, and so it would probably be
> pretty easy to improve the concurrency of the read path if it did
> become problematic.
>

I know you just moved to Java 5, and my first email talked about the
concurrency aspect.


>
> > I think the weak hash map is a good solution (but
> > also think that the ability to choose to use soft hash map would add even
> > greater flexibility), the problem is that there is no built in concurrent
> > weak/soft hash map.
>
> It might be a better idea to have interfaces that allow one to
> implement their own policies rather than to try and push all the
> different options into Lucene:
> https://issues.apache.org/jira/browse/LUCENE-831
>
>
I think that trying to provide an extension that either syncs or not is a
pain in terms of extension point (it will be very convoluted) and many times
hinders the optimizations that can be done in concurrent systems... . Just
see what is done in the FieldCacheImpl and the creation marker which is not
really needed with google collections (they allow for single entry be
created atomically).


> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
View raw message