lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anand.mahajan" <an...@zerebral.co.in>
Subject Re: SolrCloud Scale Struggle
Date Sat, 02 Aug 2014 20:46:27 GMT
Thank you everyone for your responses. Increased the hard commit to 10mins
and autoSoftCommit to 10 secs. (I wont really need a real time get - tweaked
the app code to cache the doc and use the app side cached version instead of
fetching it from Solr) Will watch it for a day or two and clock the
throughput.

For this deployment the peak is throughout the day as more data keeps
streaming in - there are no direct users with search queries here (as of
now) - but every incoming doc is compared against the existing set of docs
in Solr - to check whether its a new one or an updated version of an
existing one and only then the doc is inserted/updated. Right now its adding
about 1100 docs a minute (~20 docs a second) [But thats because it has to
run a search before to determine whether its an insert/update]

Also, since there are already 18 JVMs per machine - How do I go about
merging these existing cores under just 1 JVM? Would it be that I'd need to
create 1 Solr instance with 18 cores inside and then migrate data from these
separate JVMs into the new instance?



--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Scale-Struggle-tp4150592p4150810.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message