Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Date: Sat, 2 Aug 2014 13:46:27 -0700 (PDT)
From: "anand.mahajan" <anand@zerebral.co.in>
To: solr-user@lucene.apache.org
Message-ID: <1407012387728-4150810.post@n3.nabble.com>
In-Reply-To: <A12B645D-4CFA-400B-A752-FD88E759FEFC@gmail.com>
References: <1406888372702-4150592.post@n3.nabble.com>
 <2DD08AA3-B766-4DDE-9041-60FB552C03F2@gmail.com>
 <A12B645D-4CFA-400B-A752-FD88E759FEFC@gmail.com>
Subject: Re: SolrCloud Scale Struggle
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Thank you everyone for your responses. Increased the hard commit to 10mins
and autoSoftCommit to 10 secs. (I wont really need a real time get - tweaked
the app code to cache the doc and use the app side cached version instead of
fetching it from Solr) Will watch it for a day or two and clock the
throughput.

For this deployment the peak is throughout the day as more data keeps
streaming in - there are no direct users with search queries here (as of
now) - but every incoming doc is compared against the existing set of docs
in Solr - to check whether its a new one or an updated version of an
existing one and only then the doc is inserted/updated. Right now its adding
about 1100 docs a minute (~20 docs a second) [But thats because it has to
run a search before to determine whether its an insert/update]

Also, since there are already 18 JVMs per machine - How do I go about
merging these existing cores under just 1 JVM? Would it be that I'd need to
create 1 Solr instance with 18 cores inside and then migrate data from these
separate JVMs into the new instance?


--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Scale-Struggle-tp4150592p4150810.html
Sent from the Solr - User mailing list archive at Nabble.com.