lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject is multi-threads searcher feasible idea to speed up?
Date Tue, 28 Sep 2010 11:24:30 GMT
hi all
    I want to speed up search time for my application. In a query, the
time is largly used in reading postlist(io with frq files) and
calculate scores and collect result(cpu, with Priority Queue). IO is
hardly optimized or already part optimized by nio. So I want to use
multithreads to utilize cpu. of course, it may be decrease QPS, but
the response time will also decrease-- that what I want. Because cpu
is easily obtained compared to faster hard disk.
    I read the codes of searching roughly and find it's not an easy
task to modify search process. So I want to use other easy method .
    One is use solr distributed search and dispatch documents to many
shards. but due to the network and global idf problem,it seems not a
good method for me.
    Another one is to modify the index structure and averagely
dispatch frq files.
    e.g    term1 -> doc1,doc2, doc3,doc4,doc5 in _1.frq
    I create to 2 indexes with
            term1->doc1,doc3,doc5
            term1->doc2,doc4
    when searching, I create 2 threads with 2 PriorityQueues to
collect top N docs and merging their results
    Is the 2nd idea feasible? Or any one has related idea? thanks.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message