lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kafka0102 <kafka0...@163.com>
Subject Re:why sorl is slower than lucene so much?
Date Thu, 21 Oct 2010 08:11:59 GMT
I found the problem's cause.It's the DocSetCollector. my fitler query result's size is about
3000000,so the DocSetCollector.getDocSet() is OpenBitSet. And 3000000 OpenBitSet.fastSet(doc)
op is too slow. So I used SolrIndexSearcher's TopFieldDocs search(Query query, Filter filter,
int n,
                    Sort sort), and it's normal.




At 2010-10-20 19:21:27,kafka0102 <kafka0102@163.com> wrote:

>For solr's SolrIndexSearcher.search(QueryResult qr, QueryCommand cmd), I find it's too
slowly.my index's size is about 500M, and record's num is 3984274.my query is like q=xx&fq=fid:1&fq=atm:[int_time1
TO int_time2].
>fid's type is <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true"
positionIncrementGap="0"/>
>atm's type is  <fieldType name="sint" class="solr.TrieIntField" precisionStep="8" omitNorms="true"
positionIncrementGap="0"/>.
>for the test, I closed solr's cache's config and used another lucene's code like bottom:
>
> private void test2(final ResponseBuilder rb) {
>    try {
>      final SolrQueryRequest req = rb.req;
>      final SolrIndexSearcher searcher = req.getSearcher();
>      final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
>      final ExecuteTimeStatics timeStatics = ExecuteTimeStatics.getExecuteTimeStatics();
>      final ExecuteTimeUnit staticUnit = timeStatics.addExecuteTimeUnit("test2");
>      staticUnit.start();
>      final List<Query> query = cmd.getFilterList();
>      final BooleanQuery booleanFilter = new BooleanQuery();
>      for (final Query q : query) {
>        booleanFilter.add(new BooleanClause(q,Occur.MUST));
>      }
>      booleanFilter.add(new BooleanClause(cmd.getQuery(),Occur.MUST));
>      logger.info("q:"+query);
>      final Sort sort = cmd.getSort();
>      final TopFieldDocs docs = searcher.search(booleanFilter,null,20,sort);
>      final StringBuilder sbBuilder = new StringBuilder();
>      for (final ScoreDoc doc :docs.scoreDocs) {
>        sbBuilder.append(doc.doc+",");
>      }
>      logger.info("hits:"+docs.totalHits+",result:"+sbBuilder.toString());
>      staticUnit.end();
>    } catch (final Exception e) {
>      throw new RuntimeException(e);
>    }
>  }
>
>for the test, I first called above's code and then solr's search(...). The result is :
lucence's about 20ms and solr's about 70ms.
>I'm so confused.
>And,I wrote another code using filter like bottom,but the range query's result num is
not correct.
>Can anybody knows the reasons?
>
>  private void test1(final ResponseBuilder rb) {
>    try {
>      final SolrQueryRequest req = rb.req;
>      final SolrIndexSearcher searcher = req.getSearcher();
>      final SolrIndexSearcher.QueryCommand cmd = rb.getQueryCommand();
>      final ExecuteTimeStatics timeStatics = ExecuteTimeStatics.getExecuteTimeStatics();
>      final ExecuteTimeUnit staticUnit = timeStatics.addExecuteTimeUnit("test1");
>      staticUnit.start();
>      final List<Query> query = cmd.getFilterList();
>      final BooleanFilter booleanFilter = new BooleanFilter();
>      for (final Query q : query) {
>        setFilter(booleanFilter,q);
>      }
>      final Sort sort = cmd.getSort();
>      final TopFieldDocs docs = searcher.search(cmd.getQuery(),booleanFilter,20,sort);
>      logger.info("hits:"+docs.totalHits);
>     
>      staticUnit.end();
>    } catch (final Exception e) {
>      throw new RuntimeException(e);
>    }
>  }
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message