lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Venzal <bven...@yahoo.com.INVALID>
Subject Solr Cloud: Massive indexing
Date Tue, 08 Sep 2015 16:28:12 GMT
Hello,
I am indexing lots of big documents thanks to Solr Cloud in a map reduce job: so every day
it is 10000 - 20000 documents (avg:8Mb, max 100Mb, total ~ 100 Gb). This is done is 20 minutes. We
have 5 nodes, Solr server is launched with 20 Gb of Ram (and GC1). We add in parallel around
200 SolrDocuments.Unfortunately, Solr Cloud does not accept so much data and it fails (org.apache.solr.client.solrj.SolrServerException:
IOException occured when talking to server at:).That still indexed many documents thanks to
the multiple attempts, so if I launch multiple times my Map Reduce, I finally get all my documents
indexed ...
Is there a way to be check availability of Solr Cloud before adding a document or maybe synchronize
with the Solr Server ?What do you think ?
ThanksBest RegardsBertrand
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message