lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie <ja...@mailarchiva.com>
Subject Re: search performance
Date Tue, 03 Jun 2014 09:47:16 GMT
Sure... see below:

     protected void search(Query query, Filter queryFilter, Sort sort)
             throws BlobSearchException {
         try {
             logger.debug("start search  {searchquery='" + 
getSearchQuery() + 
"',query='"+query.toString()+"',filterQuery='"+queryFilter+"',sort='"+sort+"'}");
             Thread.currentThread().setPriority(Thread.MAX_PRIORITY);
             results.clear();

             int max;

             if (getPagination()) {
                 max = start + length;
             } else {
                 max = getMaxResults();
             }

             // release the old volume searchers
             IndexReader indexReader = initIndexReader();
             searcher = new IndexSearcher(indexReader,executor);
             TopFieldCollector fieldCollector = 
TopFieldCollector.create(sort, max,true, false, false, true);

             searcher.search(query, queryFilter, fieldCollector);

             TopDocs topDocs;

             if (getPagination()) {
                 topDocs = fieldCollector.topDocs(start,length);
             } else {
                 topDocs = fieldCollector.topDocs();
             }

             int count = 0;
             for (int i = 0; i < topDocs.scoreDocs.length; i++) {
                 if ((getMaxResults()>0 && count > getMaxResults()) || 
(getPagination() && count++>=length)) { break; }
                 results.add(topDocs.scoreDocs[i]);
             }

             totalHits = fieldCollector.getTotalHits();

             logger.debug("search executed successfully {query='"+ 
getSearchQuery() + "',returnedresults='" + results.size()+ "'}");
         } catch (Exception io) {
             throw new BlobSearchException("failed to execute search 
query {searchquery='"+ getSearchQuery() + "}", io, logger, 
ChainedException.Level.DEBUG);
         }
     }
On 2014/06/03, 11:41 AM, Rob Audenaerde wrote:
> Hi Jamie,
>
> What is included in the 5 minutes?
>
> Just the call to the searcher?
>
> seacher.search(...) ?
>
> Can you show a bit more of the code you use?
>
>
>
> On Tue, Jun 3, 2014 at 11:32 AM, Jamie <jamie@mailarchiva.com> wrote:
>
>> Vitaly
>>
>> Thanks for the contribution. Unfortunately, we cannot use Lucene's
>> pagination function, because in reality the user can skip pages to start
>> the search at any point, not just from the end of the previous search. Even
>> the
>> first search (without any pagination), with a max of 1000 hits, takes 5
>> minutes to complete.
>>
>> Regards
>>
>> Jamie
>>
>> On 2014/06/03, 10:54 AM, Vitaly Funstein wrote:
>>
>>> Something doesn't quite add up.
>>>
>>> TopFieldCollector fieldCollector = TopFieldCollector.create(sort,
>>> max,true,
>>>
>>>> false, false, true);
>>>>
>>>> We use pagination, so only returning 1000 documents or so at a time.
>>>>
>>>>
>>>>   You say you are using pagination, yet the API you are using to create
>>> your
>>> collector isn't how you would utilize Lucene's built-in "pagination"
>>> feature (unless misunderstand the API). If the max is the snippet above is
>>> 1000, then you're simply returning top 1000 docs every time you execute
>>> your search. Otherwise... well, could you actually post a bit more of your
>>> code that runs the search here, in particular?
>>>
>>> Assuming that the max is much larger than 1000, however, you could call
>>> fieldCollector.topDocs(int, int) after accumulating hits using this
>>> collector, but this won't work multiple times per query execution,
>>> according to the javadoc. So you either have to re-execute the full
>>> search,
>>> and then get the next chunk of ScoreDocs, or use the proper API for this,
>>> one that accepts as a parameter the end of the previous page of results,
>>> i.e. IndexSearcher.searchAfter(ScoreDoc, ...)
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message