lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <ysee...@gmail.com>
Subject Re: scalability w/ number of fields
Date Tue, 05 Apr 2005 16:08:24 GMT
Optimize performance update (with tons of indexed fields):

We had a timing bug... ignore the hour I first reported.  Here are the
current numbers:

indexed_fields=6791  index_size=3.9GB  optimize_time=21min
indexed_fields=3216  index_size=2.0GB  optimize_time=9min
indexed_fields=2080  index_size=1.4GB  optimize_time=4min

It's a little apples-to-oranges since we simply removed some of the
fields to test a lower field count (and hence the index size also goes
down).

-Yonik

On Apr 4, 2005 5:38 PM, Yonik Seeley <yseeley@gmail.com> wrote:
> I know Lucene is very scalable in many ways, but how about number of fieldnames?
> 
> We have an index using around 6000 unique fieldnames,
> 450,000 documents, and a total index size of 4GB.   It's very
> sparse... documents don't have that many fields, but the number of
> different fieldtypes is huge.
> 
> An optimize of this index took about an hour (mergefactor 10, compound index)
> This is on enterprise hardware (fast SCSI raid, 6GB RAM, dual 2.8GHz Xeon).
> The JVM was Java5 with 2.5GB heap.
> 
> This seems very long... anyone have any insights?
> We'll be running more tests to see if decreasing the number of fields
> has an impact.
> 
> -Yonik
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message