lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanislav Sandalnikov <s.sandalni...@gmail.com>
Subject Re: Problems with stored/not-stored field filter queries
Date Wed, 25 Jan 2017 01:15:26 GMT
Hi Shawn, 

Thanks a lot for your valuable input. Of course you were right, the data was changed after
reindex step, I completely forgot that categories are done by separate application and this
application was pushing empty IndexDate field after update, because it couldn’t extract
a value from it since the field was not stored. By the way is there any way to see if there
is a index for some particular field of a document?

Stanislav

> 24 янв. 2017 г., в 23:43, Shawn Heisey <apache@elyograg.org> написал(а):
> 
> On 1/24/2017 8:29 AM, Stanislav Sandalnikov wrote:
>> With indexed but not stored IndexDate field:
>> 
>> 1) With this query everything works fine, I’m getting the results back:
>> /select?fl=taskid,docid,score&q=*:*&fq=((((category:"Security")))+AND+(datasource:(sites)))&fq={!frange+l%3D0}query($q)&sort=IndexDate+desc&rows=100&start=0
>> 
>> 2) With this query I get nothing:
>> /select?fl=taskid,docid,score&q=*:*&fq=((((category:"Security")))+AND+(datasource:(sites))+AND+(IndexDate:[NOW/DAY-27DAYS+TO+NOW/DAY%2B1DAY]))&fq={!frange+l%3D0}query($q)&sort=IndexDate+desc&rows=100&start=0
>> 
>> Document’s IndexDate fit specified timeframe for sure. 
>> 
>> 3) With this query there is no category filter, but there is timeframe filter and
everything works fine:
>> /select?fl=taskid,docid,score&q=*:*&fq=((datasource:(sites))+AND+(IndexDate:[NOW/DAY-27DAYS+TO+NOW/DAY%2B1DAY]))&fq={!frange+l%3D0}query($q)&sort=IndexDate+desc&rows=100&start=0
> 
> It sounds like there are no documents that fit all three conditions --
> the correct datasource, the correct category, AND that date range.
> 
>> Then I decided that it might be related to IndexDate as it is not stored. I reindexed
data with stored IndexDate field and now query number 2 works just fine.
> 
> Whether or not a field is stored does not affect searches, only which
> fields are returned in the results.  My guess is that when you do this
> reindex, there's some difference, so different information is being sent
> to Solr and incorporated into the index.  There is also the possibility
> that when you change the schema, that the changes are not limited to the
> "stored" parameter on one field.  If you are changing the "type"
> parameter for the field, the effect may be larger than you realize.
> 
> When you find that the search doesn't work, check the schema browser for
> allthe fields used in your query, and load the top N terms on each one. 
> You may find that there aren't any terms to load for some reason, or
> that the terms that get loaded are not compatible with the part of the
> query which is being done for that field.  The term info that the schema
> browser shows is not be affected by whether or not the field is stored. 
> Also in the schema browser, check all the settings for those fields when
> it's working compared to when it's not working.  There may be a
> difference other than the "stored" checkbox.
> 
> Here's a schema browser screenshot with terms loaded from a field that
> would be compatible with your date range query:
> 
> https://www.dropbox.com/s/t5b4b6hrk0wz6oz/trie-date-schema-browser.png?dl=0
> 
> This screenshot comes from version 6.3.0 running in standalone mode.
> 
> If everything appears to be correct on your system when it's not working
> compared to when it is working, then a bug is always a possibility, but
> a bug like that would affect a LOT of people.  This list would have
> heard from those people on the problem.  Such a bug should also get
> caught by the numerous tests that are part of the Lucene/Solr source
> code, tests that are frequently run by automated systems and Solr
> developers working on the code.
> 
> Although your question did include more information than many do, it was
> missing a number of important details.  Some of the more important
> pieces are the version of Solr and concrete information from your
> config/schema.
> 
> Thanks,
> Shawn
> 


Mime
View raw message