lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1609) Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead
Date Mon, 04 May 2009 10:08:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705547#action_12705547
] 

Michael McCandless commented on LUCENE-1609:
--------------------------------------------

bq. Since the indexWriter now does deletes, doesn't it sometimes need the term index too?

True, so when IW opens the SR in order to apply deletes it should request that the terms index
be loaded?

Though, because iW now internally pools open SR's (for NRT search), it's entirely possible
that an SR already opened for merging is re-used when it's time to apply deletes (even in
the NRT case).  To handle that we could add a package private method to load the terms index,
which only IW would call (we do something similar with the doc stores).  IW's synchronization
ensures only one thread would call the method.

It would then not be necessary to check on every Term lookup whether the index was loaded.

In general I prefer up-front initialization when possible.  Ie, when I call IndexReader.open,
it should do as much initializing as it can instead of deferring initialization until the
first query.

> Eliminate synchronization contention on initial index reading in TermInfosReader ensureIndexIsRead

> ---------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1609
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1609
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.9
>         Environment: Solr 
> Tomcat 5.5
> Ubuntu 2.6.20-17-generic
> Intel(R) Pentium(R) 4 CPU 2.80GHz, 2Gb RAM
>            Reporter: Dan Rosher
>             Fix For: 2.9
>
>         Attachments: LUCENE-1609.patch, LUCENE-1609.patch
>
>
> synchronized method ensureIndexIsRead in TermInfosReader causes contention under heavy
load
> Simple to reproduce: e.g. Under Solr, with all caches turned off, do a simple range search
e.g. id:[0 TO 999999] on even a small index (in my case 28K docs) and under a load/stress
test application, and later, examining the Thread dump (kill -3) , many threads are blocked
on 'waiting for monitor entry' to this method.
> Rather than using Double-Checked Locking which is known to have issues, this implementation
uses a state pattern, where only one thread can move the object from IndexNotRead state to
IndexRead, and in doing so alters the objects behavior, i.e. once the index is loaded, the
index nolonger needs a synchronized method. 
> In my particular test, this uncreased throughput at least 30 times.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message