lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Po-Yu Chuang <ratbert.chu...@gmail.com>
Subject Solr slow start up (tlog is small)
Date Mon, 03 Nov 2014 16:03:56 GMT
Hi,

I am using Solr 4.9 with Tomcat and it works fine except that the
deployment of solr.war is too long. While deploying Solr, all webapps on
Tomcat stop responding which is unacceptable. Most articles I found say
that it might result from big transaction log because of uncommitted
documents, but this is not my case.

At first, the Solr data is 280G and the start up time is 30 minutes. Then I
set a field to stored="false" and re-index whole data. The data size became
185G and the start up time reduced to 17 minutes, but it is still too long.

Here are some numbers I measured:

1)
Solr home: 280G
tlog: 500K
30 min to start up
While starting up, disk read is constantly about 50MB/s (according to
dstat). So it seems that Solr reads 30m * 60s * 50MB/s = 90GB of data while
starting up, which is 30% of index data size.

2)
Solr home: 185G
tlog: 5M
17 minutes to start up
While starting up, disk read is constantly about 5MB/s (according to
dstat). So it seems that Solr reads 17m * 60s *5MB/s = 5GB of data while
starting up, which is about 3% of index data size.

p.s. I did commit each time 1000 documents being added and did optimization
after all documents are added.

Any ideas or suggestions would be appreciated.

Thanks,
Po-Yu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message