lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Woodward <alan.woodw...@romseysoftware.co.uk>
Subject Re: Can't getLiveDocs in Solr Component from LUCENE-2878--positions branch
Date Mon, 23 Jul 2012 21:30:21 GMT
Hi Sean,

From the javadocs on AtomicReader:

 /** Returns the {@link Bits} representing live (not
   *  deleted) docs.  A set bit indicates the doc ID has not
   *  been deleted.  If this method returns null it means
   *  there are no deleted documents (all documents are
   *  live).

So I guess you probably don't have any deleted docs in your index.

-Alan

On 23 Jul 2012, at 02:44, Sean O'Connor wrote:

> Hello all,
>    I think I am missing something fairly basic. I have been checking out the LUCENE-2878
branch and really like it. Somewhere along the line though it seems I have foo'ed something
up. I can't successfully call getLiveDocs().
> 
>    I have enough knowledge of solr/lucene to be dangerous, so this is probably just me
missing a memo somewhere : -). I've whittled down my test code to inserting the following
lines (3 variations of trying to getLiveDocs) into QueryComponent.process(ResponseBuilder
rb)... in the Lucene-2878 branch:
> 
>        AtomicReaderContext trc = searcher.getAtomicReader().getTopReaderContext();
>        Bits liveDocs = trc.reader().getLiveDocs();
>        System.out.println("liveDocs: " + liveDocs);
> 
>        AtomicReader fooReader = searcher.getAtomicReader();
>        Bits fooBits = fooReader.getLiveDocs();
>        System.out.println("fooBits: " + fooBits);
> 
>        for (final AtomicReaderContext leaf : trc.leaves()) {
>            final AtomicReader reader = leaf.reader();
>            final Bits ld = reader.getLiveDocs();
>            System.out.println("ld: " + ld);
>        }
> 
> 
> So my hacked version looks like:
>    @Override
>    public void process(ResponseBuilder rb) throws IOException {
>        SolrQueryRequest req = rb.req;
>        SolrQueryResponse rsp = rb.rsp;
>        SolrParams params = req.getParams();
>        if (!params.getBool(COMPONENT_NAME, true)) {
>            return;
>        }
>        SolrIndexSearcher searcher = req.getSearcher();
> 
>        AtomicReaderContext trc = searcher.getAtomicReader().getTopReaderContext();
>        Bits liveDocs = trc.reader().getLiveDocs();
>        System.out.println("liveDocs: " + liveDocs);
> 
>        AtomicReader fooReader = searcher.getAtomicReader();
>        Bits fooBits = fooReader.getLiveDocs();
>        System.out.println("fooBits: " + fooBits);
> 
>        for (final AtomicReaderContext leaf : trc.leaves()) {
>            final AtomicReader reader = leaf.reader();
>            final Bits ld = reader.getLiveDocs();
>            System.out.println("ld: " + ld);
>        }
> 
> 
> Everything turns up null (liveDocs, fooBits, and ld). Can anyone point me in the right
direction?
> 
> My test schema.xml is:
> <?xml version="1.0" encoding="UTF-8" ?>
> 
> <schema name="position test example" version="1.5">
> 
>  <fields>
>    <field name="id" type="text_en" indexed="true" stored="true" required="true"/>
>    <field name="subject" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
>    <field name="text" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
>    <field name="test_posofftv" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
> 
>  </fields>
>  <uniqueKey>id</uniqueKey>
> 
>  <types>
>    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
>    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
>      <analyzer type="index">
>        <tokenizer class="solr.StandardTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory"/>
>        <filter class="solr.EnglishPossessiveFilterFactory"/>
>      </analyzer>
>      <analyzer type="query">
>        <tokenizer class="solr.StandardTokenizerFactory"/>
>        <filter class="solr.LowerCaseFilterFactory"/>
>        <filter class="solr.EnglishPossessiveFilterFactory"/>
>      </analyzer>
>    </fieldType>
> 
>   </types>
> 
> </schema>
> 
> solrconfig.xml is:
> <?xml version="1.0" encoding="UTF-8" ?>
> <config>
>  <luceneMatchVersion>LUCENE_50</luceneMatchVersion>
>  <lib path="/opt/SDRSolrQuery2.jar" />
>  <!--<lib path="/opt/SDRSolrQuery-bad.jar"/>-->
>  <dataDir>${solr.data.dir:}</dataDir>
> 
> 
>  <directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
>  <!--<directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
-->
>  <indexConfig>
>  </indexConfig>
> 
>  <jmx/>
>  <updateHandler class="solr.DirectUpdateHandler2">
> 
>    <autoCommit>
>      <maxTime>15000</maxTime>
>      <openSearcher>false</openSearcher>
>    </autoCommit>
>  </updateHandler>
> 
>  <requestHandler name="/update" class="solr.UpdateRequestHandler">
>  </requestHandler>
> 
> 
>  <query>
>    <maxBooleanClauses>1024</maxBooleanClauses>
>    <filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"/>
>    <queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/>
>    <documentCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/>
> <enableLazyFieldLoading>true</enableLazyFieldLoading>
>    <queryResultWindowSize>20</queryResultWindowSize>
> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>    <listener event="newSearcher" class="solr.QuerySenderListener">
>      <arr name="queries">
>      </arr>
>    </listener>
>    <listener event="firstSearcher" class="solr.QuerySenderListener">
>      <arr name="queries">
>        <lst>
>          <str name="q">static firstSearcher warming in solrconfig.xml</str>
>        </lst>
>      </arr>
>    </listener>
> 
>    <useColdSearcher>false</useColdSearcher>
>    <maxWarmingSearchers>2</maxWarmingSearchers>
> 
>  </query>
> 
>  <requestDispatcher handleSelect="false">
>    <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048000"/>
> 
>    <httpCaching never304="true"/>
>  </requestDispatcher>
> 
>  <requestHandler name="/select" class="solr.SearchHandler">
>    <!-- default values for query parameters can be specified, these
>         will be overridden by parameters in the request
>      -->
>    <lst name="defaults">
>      <str name="echoParams">explicit</str>
>      <int name="rows">10</int>
>      <str name="df">text</str>
>    </lst>
>  </requestHandler>
> 
>  <!-- A request handler that returns indented JSON by default -->
>  <requestHandler name="/query" class="solr.SearchHandler">
>    <lst name="defaults">
>      <str name="echoParams">explicit</str>
>      <str name="wt">json</str>
>      <str name="indent">true</str>
>      <str name="df">text</str>
>    </lst>
>  </requestHandler>
> 
>  <!-- Admin Handlers
> 
>       Admin Handlers - This will register all the standard admin
>       RequestHandlers.
>    -->
>  <requestHandler name="/admin/" class="solr.admin.AdminHandlers"/>
> 
> 
>  <!-- Legacy config for the admin interface -->
>  <admin>
>    <defaultQuery>*:*</defaultQuery>
>  </admin>
> 
> 
>  <searchComponent name="cmComponent" class="com.oconeco.sdrprep.solr.CMQueryComponent">
>    <lst name="defaults">
>      <str name="fl">content</str>
>    </lst>
>  </searchComponent>
> 
>  <requestHandler name="cm" class="org.apache.solr.handler.component.SearchHandler">
>    <lst name="defaults">
>      <!--<str name="defType">sdr</str>-->
>      <str name="fl">id,fileName,title,score</str>
>      <str name="rows">1000000</str>
>    </lst>
>    <arr name="components">
>      <str>cmComponent</str>
>    </arr>
>  </requestHandler>
> 
> 
>  <requestHandler name="/cm" class="org.apache.solr.handler.component.SearchHandler">
>    <lst name="defaults">
>      <!--<str name="defType">sdr</str>-->
>      <str name="fl">id,subject, test_posofftv</str>
>    </lst>
>    <arr name="components">
>      <str>cmComponent</str>
>    </arr>
>  </requestHandler>
> 
> 
> </config>
> 
> And my ./post.sh test.xml is
> 
> <add>
> <doc>
>  <field name="id">1</field>
>  <field name="subject">one two three four test electronics</field>
>  <field name="test_posofftv">one two three four test</field>
> </doc>
> 
> <doc>
>  <field name="id">2</field>
>  <field name="subject">one two three four test electronics</field>
>  <field name="test_posofftv">one two three four test</field>
> </doc>
> 
> <doc>
>  <field name="id">3</field>
>  <field name="subject">one two three four test electronics</field>
>  <field name="test_posofftv">one two three four test</field>
> </doc>
> 
> <doc>
>  <field name="id">4</field>
>  <field name="subject">one two three four test electronics</field>
>  <field name="test_posofftv">one two three four test</field>
> </doc>
> 
> </add>
> 
> 
> Thank you,
> 
> Sean
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message