lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yueyu lin" <popeye...@gmail.com>
Subject Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.
Date Wed, 10 May 2006 01:39:58 GMT
My assumption is that every query is relatively quick. If the times lapsed
in other process when querying, the ensureIndexIsRead() function will not
cause a lot of problems. If not, the ensureIndexIsRead() function will be a
bottle neck.
I could understand that a lot of systems' queries are quiet complex, so the
problem may be gone. But for our system, more than 150 queries per seconds
on a dual CPUs linux box, that's a problem.

If the queries became more complicated, we would ignore it in most of time.

On 5/10/06, Doug Cutting <cutting@apache.org> wrote:
>
> The best search performance is achieved using a single IndexSearcher
> shared by multiple threads.  Peter Keegan has demonstrated rates of up
> to 400 searches per second on eight-CPU machines using this approach:
>
> http://www.mail-archive.com/java-user@lucene.apache.org/msg05074.html
>
> So the synchronization is probably not hurting your performance.
>
> Doug
>
> yueyu lin wrote:
> > One IndexSearcher is one IndexSearcher instance. The instance has a lot
> of
> > functions. Unfortunately they will call another synchronized function in
> > other class's instance (TermInfosReader). That's the point why we need
> two
> > IndexSearchers. But two searchers will cost double cache memory. It's
> not
> > worthy. So if Lucene team can modify the codes slightly, the
> > synchronization
> > problem will be gone.
> >
> > On 5/9/06, Chris Hostetter <hossman_lucene@fucit.org> wrote:
> >
> >>
> >>
> >> :   We found if we were using 2 IndexSearcher, we would get 10%
> >> performance
> >> : benefit.
> >> :   But if we increased the number of IndexSearcher from 2, the
> >> performance
> >> : improvement became slight even worse.
> >>
> >> Why use more then 2 IndexSearchers?
> >>
> >> Typically 1 is all you need, except for when you want to open and "warm
> >> up" a new Searcher because you know your index has changed on disk and
> >> you're ready for those changes to be visible.
> >>
> >> (I'm not arguing against your change -- concurrancy isn't my forte so i
> >> have no opinion on wether your suggesting is good or not, i'm just
> >> questioning the goal)
> >>
> >> Acctually .. i don't know a lot about the internals of IndexSearcher
> and
> >> TermInfosReader, but according to your description of the problem...
> >>
> >> :   The class org.apache.lucene.index.TermInfosReader , as you know,
> >> every
> >> : IndexSearcher will have one TermInfosReader. Every query, one method
> in
> >> the
> >> : class must be called:
> >> : private synchronized void ensureIndexIsRead() throws IOException .
> >> Notice
> >>
> >> If the method isn't static, then how can two differnet instances of
> >> IndexSearcher, each with their own TermInfosReader, block one another?
> >>
> >>
> >>
> >>
> >> -Hoss
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >
> >
> > --
> > --
> > Yueyu Lin
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>


--
--
Yueyu Lin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message