lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jed Wesley-Smith (JIRA)" <>
Subject [jira] Commented: (LUCENE-1609) Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
Date Thu, 04 Jun 2009 01:04:07 GMT


Jed Wesley-Smith commented on LUCENE-1609:

We get hit by this too. We'd love to see a fix and we'd agree that up-front initialisation
would work for us.

AFAICT there are a number of other potential subtle concurrency issues with {{TermInfosReader}}:

# lack of {{final}} on fields - a number of fields ({{directory}}, {{segment}}, {{fieldInfos}},
{{origEnum}}, {{enumerators}} etc.) are never written to after construction and should be
declared {{final}} for better publication semantics
# unsafe publication of {{indexDivisor}} and {{totalIndexInterval}} these fields are not written
to under lock and in a worst-case could be unstable under use.
# {{close()}} calls {{enumerators.set(null)}} which only clears the value for the calling

Making the {{TermInfosReader}} more immutable would address some of these issues.

As far as the root problem goes, uncontended synchronisation is generally _very fast_, but
significantly slows down once a lock becomes contended. The kind of pattern employed here
(do something quite expensive but only once) is not an ideal use of synchronisation as it
commonly leads to a contended lock, which remains a slow lock well after it is required\*.
That being said, it isn't easy to do correctly and performantly under 1.4. 

\* An alternative approach is something like this [LazyReference|]
class, although this kind of thing really requires Java5 for full value.

> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead

> ---------------------------------------------------------------------------------------------------
>                 Key: LUCENE-1609
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.9
>         Environment: Solr 
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
>            Reporter: Dan Rosher
>             Fix For: 2.9
>         Attachments: LUCENE-1609.patch, LUCENE-1609.patch
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search
e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress
test application, and later, examining the Thread dump (kill -3) , many threads are blocked
on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation
uses a state pattern, where only one thread can move the object from IndexNotRead state to
IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the
index nolonger needs a synchronized method. 
> In my particular test, this uncreased throughput at least 30 times.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message