lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <yo...@lucidimagination.com>
Subject Re: Questions on FieldValueCache
Date Mon, 03 Aug 2009 21:56:42 GMT
On Mon, Aug 3, 2009 at 5:45 PM, Stephen Duncan
Jr<stephen.duncan@gmail.com> wrote:
> What will cause me to exceed the limit and fail during faceting using the fc
> method is if the documents within a block of 65536 combine to take up too
> much space.  And this (generally speaking) going to be a function on the
> average number of unique terms in the documents?

Yep... pretty much.  The comment at the head of UnInvertedField tries
to lay out the data structure.
Also subtract out "big terms" that match > 5% of the index... they
aren't uninverted.
And the block of 65536 docs wraps around after 256 blocks (since there
are only 256 arrays).

If you needed just a little more headroom, it wouldn't be to hard for
you to modify from 256 arrays to 512 (which would double the number of
terms allowed per block).  Or if you had much less than 16M docs in
the index, you could also lower the number of docs per block (all
custom code of course... it can't be done with a parameter today).

-Yonik
http://www.lucidimagination.com

Mime
View raw message