lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [lucy-user] Perfomance issue
Date Sat, 08 Aug 2015 16:58:13 GMT
On Sat, Aug 8, 2015 at 7:48 AM,  <> wrote:

>> First, I'd try to find out which call into Lucy takes so long,
> As far as I can see it is the loop that reads the result with $hits->next

All that Hits#next does is read a document from disk.  At that point, the
search has been completed and we have a list of document IDs; each call to
next() takes one of those IDs, finds the location of the data and reads from
mmap'd memory to deserialize it into a hash.

Performance of next() scales with the size of the document fetched, not with
the size of the index (except insofar as index size affects OS cache
performance) or the complexity of the search query.  Text data in the document
store is not compressed.

How large are these documents?  If they are small, yet each iteration of the
$hits->next loop is unacceptably slow, it it hard to understand how the code
Lucy is executing could be the source of the slowdown.

> It's a production system (it not happens in the test system)


> 2.) The second issue is, that from time to time the search time increases
>     drastically, so it goes up to serveral seconds and more

You had mentioned before that restarting the process fixes this issue.  How
much memory is the process consuming when you restart it?

> P.S. The system is a vm with 16GB ram, 6 cores and is running on a SSD and
>      most times 98% idle, so it should not be a performance issues of the
>      host itself.

What do graphs of memory and IO on this box look like?

Marvin Humphrey

View raw message