lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean O'Connor <sean...@gmail.com>
Subject Re: Can't getLiveDocs in Solr Component from LUCENE-2878--positions branch
Date Tue, 24 Jul 2012 13:40:25 GMT
Ah... right. That couldn't be much more obvious : -).

So it looks like my problem is now in navigating the new API. I am 
getting null Intervals from the Scorer. I'll start a new thread since 
getLiveDocs is not part of my difficulty.
Thanks,

Sean


On 07/23/2012 05:30 PM, Alan Woodward wrote:
> Hi Sean,
>
>  From the javadocs on AtomicReader:
>
>   /** Returns the {@link Bits} representing live (not
>     *  deleted) docs.  A set bit indicates the doc ID has not
>     *  been deleted.  If this method returns null it means
>     *  there are no deleted documents (all documents are
>     *  live).
>
> So I guess you probably don't have any deleted docs in your index.
>
> -Alan
>
> On 23 Jul 2012, at 02:44, Sean O'Connor wrote:
>
>> Hello all,
>>     I think I am missing something fairly basic. I have been checking out the LUCENE-2878
branch and really like it. Somewhere along the line though it seems I have foo'ed something
up. I can't successfully call getLiveDocs().
>>
>>     I have enough knowledge of solr/lucene to be dangerous, so this is probably just
me missing a memo somewhere : -). I've whittled down my test code to inserting the following
lines (3 variations of trying to getLiveDocs) into QueryComponent.process(ResponseBuilder
rb)... in the Lucene-2878 branch:
>>
>>         AtomicReaderContext trc = searcher.getAtomicReader().getTopReaderContext();
>>         Bits liveDocs = trc.reader().getLiveDocs();
>>         System.out.println("liveDocs: " + liveDocs);
>>
>>         AtomicReader fooReader = searcher.getAtomicReader();
>>         Bits fooBits = fooReader.getLiveDocs();
>>         System.out.println("fooBits: " + fooBits);
>>
>>         for (final AtomicReaderContext leaf : trc.leaves()) {
>>             final AtomicReader reader = leaf.reader();
>>             final Bits ld = reader.getLiveDocs();
>>             System.out.println("ld: " + ld);
>>         }
>>
>>
>> So my hacked version looks like:
>>     @Override
>>     public void process(ResponseBuilder rb) throws IOException {
>>         SolrQueryRequest req = rb.req;
>>         SolrQueryResponse rsp = rb.rsp;
>>         SolrParams params = req.getParams();
>>         if (!params.getBool(COMPONENT_NAME, true)) {
>>             return;
>>         }
>>         SolrIndexSearcher searcher = req.getSearcher();
>>
>>         AtomicReaderContext trc = searcher.getAtomicReader().getTopReaderContext();
>>         Bits liveDocs = trc.reader().getLiveDocs();
>>         System.out.println("liveDocs: " + liveDocs);
>>
>>         AtomicReader fooReader = searcher.getAtomicReader();
>>         Bits fooBits = fooReader.getLiveDocs();
>>         System.out.println("fooBits: " + fooBits);
>>
>>         for (final AtomicReaderContext leaf : trc.leaves()) {
>>             final AtomicReader reader = leaf.reader();
>>             final Bits ld = reader.getLiveDocs();
>>             System.out.println("ld: " + ld);
>>         }
>>
>>
>> Everything turns up null (liveDocs, fooBits, and ld). Can anyone point me in the
right direction?
>>
>> My test schema.xml is:
>> <?xml version="1.0" encoding="UTF-8" ?>
>>
>> <schema name="position test example" version="1.5">
>>
>>   <fields>
>>     <field name="id" type="text_en" indexed="true" stored="true" required="true"/>
>>     <field name="subject" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
>>     <field name="text" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
>>     <field name="test_posofftv" type="text_en" indexed="true" stored="true" termVectors="true"
termPositions="true" termOffsets="true"/>
>>
>>   </fields>
>>   <uniqueKey>id</uniqueKey>
>>
>>   <types>
>>     <fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
>>     <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
>>       <analyzer type="index">
>>         <tokenizer class="solr.StandardTokenizerFactory"/>
>>         <filter class="solr.LowerCaseFilterFactory"/>
>>         <filter class="solr.EnglishPossessiveFilterFactory"/>
>>       </analyzer>
>>       <analyzer type="query">
>>         <tokenizer class="solr.StandardTokenizerFactory"/>
>>         <filter class="solr.LowerCaseFilterFactory"/>
>>         <filter class="solr.EnglishPossessiveFilterFactory"/>
>>       </analyzer>
>>     </fieldType>
>>
>>    </types>
>>
>> </schema>
>>
>> solrconfig.xml is:
>> <?xml version="1.0" encoding="UTF-8" ?>
>> <config>
>>   <luceneMatchVersion>LUCENE_50</luceneMatchVersion>
>>   <lib path="/opt/SDRSolrQuery2.jar" />
>>   <!--<lib path="/opt/SDRSolrQuery-bad.jar"/>-->
>>   <dataDir>${solr.data.dir:}</dataDir>
>>
>>
>>   <directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
>>   <!--<directoryFactory name="DirectoryFactory" class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
-->
>>   <indexConfig>
>>   </indexConfig>
>>
>>   <jmx/>
>>   <updateHandler class="solr.DirectUpdateHandler2">
>>
>>     <autoCommit>
>>       <maxTime>15000</maxTime>
>>       <openSearcher>false</openSearcher>
>>     </autoCommit>
>>   </updateHandler>
>>
>>   <requestHandler name="/update" class="solr.UpdateRequestHandler">
>>   </requestHandler>
>>
>>
>>   <query>
>>     <maxBooleanClauses>1024</maxBooleanClauses>
>>     <filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"/>
>>     <queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/>
>>     <documentCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"/>
>> <enableLazyFieldLoading>true</enableLazyFieldLoading>
>>     <queryResultWindowSize>20</queryResultWindowSize>
>> <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>>     <listener event="newSearcher" class="solr.QuerySenderListener">
>>       <arr name="queries">
>>       </arr>
>>     </listener>
>>     <listener event="firstSearcher" class="solr.QuerySenderListener">
>>       <arr name="queries">
>>         <lst>
>>           <str name="q">static firstSearcher warming in solrconfig.xml</str>
>>         </lst>
>>       </arr>
>>     </listener>
>>
>>     <useColdSearcher>false</useColdSearcher>
>>     <maxWarmingSearchers>2</maxWarmingSearchers>
>>
>>   </query>
>>
>>   <requestDispatcher handleSelect="false">
>>     <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048000"/>
>>
>>     <httpCaching never304="true"/>
>>   </requestDispatcher>
>>
>>   <requestHandler name="/select" class="solr.SearchHandler">
>>     <!-- default values for query parameters can be specified, these
>>          will be overridden by parameters in the request
>>       -->
>>     <lst name="defaults">
>>       <str name="echoParams">explicit</str>
>>       <int name="rows">10</int>
>>       <str name="df">text</str>
>>     </lst>
>>   </requestHandler>
>>
>>   <!-- A request handler that returns indented JSON by default -->
>>   <requestHandler name="/query" class="solr.SearchHandler">
>>     <lst name="defaults">
>>       <str name="echoParams">explicit</str>
>>       <str name="wt">json</str>
>>       <str name="indent">true</str>
>>       <str name="df">text</str>
>>     </lst>
>>   </requestHandler>
>>
>>   <!-- Admin Handlers
>>
>>        Admin Handlers - This will register all the standard admin
>>        RequestHandlers.
>>     -->
>>   <requestHandler name="/admin/" class="solr.admin.AdminHandlers"/>
>>
>>
>>   <!-- Legacy config for the admin interface -->
>>   <admin>
>>     <defaultQuery>*:*</defaultQuery>
>>   </admin>
>>
>>
>>   <searchComponent name="cmComponent" class="com.oconeco.sdrprep.solr.CMQueryComponent">
>>     <lst name="defaults">
>>       <str name="fl">content</str>
>>     </lst>
>>   </searchComponent>
>>
>>   <requestHandler name="cm" class="org.apache.solr.handler.component.SearchHandler">
>>     <lst name="defaults">
>>       <!--<str name="defType">sdr</str>-->
>>       <str name="fl">id,fileName,title,score</str>
>>       <str name="rows">1000000</str>
>>     </lst>
>>     <arr name="components">
>>       <str>cmComponent</str>
>>     </arr>
>>   </requestHandler>
>>
>>
>>   <requestHandler name="/cm" class="org.apache.solr.handler.component.SearchHandler">
>>     <lst name="defaults">
>>       <!--<str name="defType">sdr</str>-->
>>       <str name="fl">id,subject, test_posofftv</str>
>>     </lst>
>>     <arr name="components">
>>       <str>cmComponent</str>
>>     </arr>
>>   </requestHandler>
>>
>>
>> </config>
>>
>> And my ./post.sh test.xml is
>>
>> <add>
>> <doc>
>>   <field name="id">1</field>
>>   <field name="subject">one two three four test electronics</field>
>>   <field name="test_posofftv">one two three four test</field>
>> </doc>
>>
>> <doc>
>>   <field name="id">2</field>
>>   <field name="subject">one two three four test electronics</field>
>>   <field name="test_posofftv">one two three four test</field>
>> </doc>
>>
>> <doc>
>>   <field name="id">3</field>
>>   <field name="subject">one two three four test electronics</field>
>>   <field name="test_posofftv">one two three four test</field>
>> </doc>
>>
>> <doc>
>>   <field name="id">4</field>
>>   <field name="subject">one two three four test electronics</field>
>>   <field name="test_posofftv">one two three four test</field>
>> </doc>
>>
>> </add>
>>
>>
>> Thank you,
>>
>> Sean
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message