lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Problem with new solr.xml format and core swaps
Date Sun, 05 Apr 2015 21:05:47 GMT
I'm having two problems with Solr 4.9.1.  I can't upgrade yet, because
we are using a third-party plugin component that is not yet explicitly
qualified for anything newer than 4.9.0.  The point release upgrade
seemed like a safe bet, because I know that we don't do API changes in
point releases.  These are transient problems, and do not seem to be
affecting the index at this time.

Some background info:

Ubuntu 14, Java 8u40 from the webupd8 PPA, Solr 4.9.1.  It is *NOT*
SolrCloud.

Full rebuilds on my index involve building a new index in cores that I
have designated "build" cores, then swapping those cores with "live"
cores.  This always worked flawlessly before I updated to Solr 4.9.1 and
migrated the config to use core discovery.

root@idxb4:~# cat /index/solr4/cores/sparkinc_0/core.properties
name=sparkinclive
dataDir=../../data/sparkinc_0

root@idxb4:~# cat /index/solr4/cores/sparkinc_1/core.properties
name=sparkincbuild
dataDir=../../data/sparkinc_1

The first problem:  Sometimes, in a completely unpredictable manner, the
new solr.xml format seems to behave like using the old format with
persistent=false.

When I restarted Solr yesterday, that action swapped the live cores with
the build cores and I lost half my index because it swapped back to the
previous build cores.  Just now when I tried a restart, everything
worked flawlessly and the cores did not swap.

The second problem:  Sometimes old index segments do not get deleted,
even though they are not part of the index.

Another part of the full rebuild process involves clearing the build
cores before beginning the full import.  The code does a deleteByQuery
with *:* and then optimizes the core.  Sometimes this action fails to
delete the old segment files, but when I checked the core Overview in
the admin UI, numDocs only reflected the newly indexed docs and
deletedDocs was 0.

It was actually while trying to fix/debug this second problem that I
discovered the first problem.  Once the rebuild finished, I wanted to
see what would happen if I restarted Solr while one of my cores had 32GB
of segment files that were not part of the index ... but that's when the
indexes swapped.  At that point, I deleted all the dataDirs on both
machines (it's a distributed index), restarted Solr again, and began a
full rebuild.  Everything seems to be fine now.

Are either of these problems anything that anyone has seen?  I don't
recall seeing anything come across the list before.  Are there existing
issues in Jira?  Is there any information that I can provide which would
help in narrowing down the problem?

Thanks,
Shawn


Mime
View raw message