lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject Re: is multi-threads searcher feasible idea to speed up?
Date Tue, 28 Sep 2010 12:16:28 GMT
yes, there is a multisearcher in lucene. but it's idf in 2 indexes are
not global. maybe I can modify it and also the index like:
    term1  df=5 doc1 doc3 doc5
    term1  df=5 doc2 doc4

2010/9/28 Li Li <fancyerii@gmail.com>:
> hi all
>    I want to speed up search time for my application. In a query, the
> time is largly used in reading postlist(io with frq files) and
> calculate scores and collect result(cpu, with Priority Queue). IO is
> hardly optimized or already part optimized by nio. So I want to use
> multithreads to utilize cpu. of course, it may be decrease QPS, but
> the response time will also decrease-- that what I want. Because cpu
> is easily obtained compared to faster hard disk.
>    I read the codes of searching roughly and find it's not an easy
> task to modify search process. So I want to use other easy method .
>    One is use solr distributed search and dispatch documents to many
> shards. but due to the network and global idf problem,it seems not a
> good method for me.
>    Another one is to modify the index structure and averagely
> dispatch frq files.
>    e.g    term1 -> doc1,doc2, doc3,doc4,doc5 in _1.frq
>    I create to 2 indexes with
>            term1->doc1,doc3,doc5
>            term1->doc2,doc4
>    when searching, I create 2 threads with 2 PriorityQueues to
> collect top N docs and merging their results
>    Is the 2nd idea feasible? Or any one has related idea? thanks.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message