lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phillip Farber <pfar...@umich.edu>
Subject Re: Huge increase in index size adding just 2 fields
Date Thu, 06 Nov 2008 18:56:16 GMT
Hi Otis and Hoss,

My dates are not too granular.  They're always YYYY-MM-DD 00:00:00 but I 
see that I did not omitNorms on the date field and hlb field.  Thanks 
for pointing me in the right direction.

Phil


Chris Hostetter wrote:
> : We added the following 2 fields to the above schema as follows:
> : 
> : <field name="date" type="date" indexed="true" stored="true" required="true"/>
> : <field name="hlb" type="string" indexed="true" stored="true"
> : multiValued="true"/>
> : 
> : where the "hlb" field consists of not more than 3-4 strings such as "Social
> : Sicence"/
> : 
> : Our 500,000 document index size increased to 166G!  This seems completely
> 
> if you don't need fieldNorms for these fields (it almost never makes sense 
> for dates and based on your description of hlb i doesn't sound like you'd 
> need it there either) make sure that's disabled (you might already be 
> doing that in the fieldType declarations, but i'm not sure)
> 
> another way to reduce the amount of space (and improve date range query 
> speed) is to reduce the granulatiry of hte dates you index (ie: round off 
> to the nearest second, minute, hour, or day) so the number of unique terms 
> in the field is reduced.
> 
> -Hoss
> 

Mime
View raw message