lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toke Eskildsen ...@statsbiblioteket.dk>
Subject RE: search time & number of segments
Date Sat, 17 May 2014 18:04:20 GMT
De Simone, Alessandro [Alessandro.DeSimone@bvdinfo.com] wrote:
> We have a performance issue ever since we stopped optimizing the index. We are using
Lucene 4.8 (jvm 32bits for searching, 64bits for indexing) on Windows 2008R2.

How much RAM does your search machine have?

> For instance, a search with (2 termQuery + 1 spanquery) x 6 fields made 143 IO calls.
Now with 16 segments  we have 2432 IO calls and the search time is really bad.

[...]

That sounds right. Although each segment is 1/16 of the full index size, the number of seeks
per segment is not 1/16: Larger indexes require relatively fewer seeks. Think binary search
and log(values_in_field), although that is highly simplified.

> The size of the Index is ~24gb (14millions documents). No field are stored, only indexed.

Normally the penalty of running un-optimized is not that great, so it sounds like your machine
cannot provide the I/O speed it needs (as opposed to having a great logistics overhead from
the multiple segments). I am guessing that you are using spinning drives and that there is
not much RAM in the machine? The easy solution is either to throw RAM at the problem or switch
to SSD.

- Toke Eskildsen
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message