Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <15256173.post@talk.nabble.com>
Date: Sun, 3 Feb 2008 10:04:51 -0800 (PST)
From: phiras <phiras@gmail.com>
To: java-dev@lucene.apache.org
Subject: Re: high lookup time on single lucene index
In-Reply-To: <22662B86-4CFB-47F1-B59C-75A2B3AD869D@apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <15255763.post@talk.nabble.com>
 <22662B86-4CFB-47F1-B59C-75A2B3AD869D@apache.org>


In fact we are following lucene performance hints (
http://wiki.apache.org/lucene-java/BasicsOfPerformance )

Our queries are not that complicated , we are expanding user query, let's
say between 2 to 5 new keyword for each keyword in the user query.

in the test we did , queries are not more than 8 'OR-ed' keyword.


Grant Ingersoll-6 wrote:
> 
> Have a look at http://wiki.apache.org/lucene-java/BasicsOfPerformance
> 
> Your times sound pretty high, but it depends on the complexity of the  
> query, etc.  Can you share information about your queries, etc.?  Are  
> you opening the IndexSearcher on every query?
> 
> -Grant
> 
> On Feb 3, 2008, at 12:24 PM, phiras wrote:
> 
>>
>> Hi,
>> We have a single Lucene index consist of 4.5 million document with 20
>> million different term.
>> The lookup time is too high , (more than 3 seconds) for a none-simple
>> boolean query, this high lookup time is not acceptable.
>>
>> We did a simple test to figure out the relation between ( documents  
>> count &
>> lookup time ) and the results was amazing.
>> note :  queries and documents are generated randomly
>>
>> Doc count  /  lookup time  for 1000 multi term query ( in  
>> milliseconds )
>> 5000	    /   1683
>> 10000	   /   2898
>> 15000	   /   23667
>> 20000	   /   25002
>> 25000	   /   31282
>> 30000	   /   43849
>> 35000	   /   53526
>> 40000	   /   69321
>> 45000	   /   73319
>> 50000	   /   80006
>>
>> My questions now are :
>> 1- What's the search complexity in Lucene (driven by what ?).
>> 2- How can we enhance search performance ?
>>
>> Thank you.
>>
>> --------------------------------------------------
>> Firas Kassem,
>> d1g corp.
>>
>>
>>
>> -- 
>> View this message in context:
>> http://www.nabble.com/high-lookup-time-on-single-lucene-index-tp15255763p15255763.html
>> Sent from the Lucene - Java Developer mailing list archive at  
>> Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
> 
> --------------------------
> Grant Ingersoll
> http://lucene.grantingersoll.com
> http://www.lucenebootcamp.com
> 
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
> 
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
> 
> 

-- 
View this message in context: http://www.nabble.com/high-lookup-time-on-single-lucene-index-tp15255763p15256173.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org