lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Keegan <peterlkee...@gmail.com>
Subject Re: Solr hangs on distributed updates
Date Mon, 15 Dec 2014 19:52:10 GMT
I added distribUpdateConnTimeout and distribUpdateSoTimeout to solr.xml and
the commit did timeout.(btw, is there any way to view solr.xml in the admin
console?).

Also, although we do have an init.d start/stop script for Solr, the 'stop'
command was not executed during shutdown because there was no lock file for
the script in '/var/lock/subsys'. I didn't know about this until I google'd
around and found '
http://www.redhat.com/magazine/008jun05/departments/tips_tricks'. When I
added the lock file, both the AWS 'stop' and 'terminate' actions did result
in an orderly shutdown of the replica which caused the Solr-leader to get
an exception and update the live_nodes, gracefully.

So now, the timeouts should only play a backup role.

Thanks for the help,
Peter


On Fri, Dec 12, 2014 at 5:21 PM, Chris Hostetter <hossman_lucene@fucit.org>
wrote:
>
>
> : No, I wasn't aware of these. I will give that a try. If I stop the Solr
> : jetty service manually, things recover fine, but the hang occurs when I
> : 'stop' or 'terminate' the EC2 instance. The Zookeeper leader reports a
>
> I don't know squat about AWS Auto-Scaling, (and barely anything about AWS)
> but what you describe makes it sound like maybe your "machine" (ie AMI?)
> isn't really configured very well?
>
> Do you have some init.d/systemd type scripts to ensure a clean shutdown of
> Solr when the machine is shutdown/rebooted?  That seems like a pretty good
> idea in general (in dependent of wether you are using Auto-Scaling) and --
> assuming AWS auto-scaling does clean OS shutdowns when terminating
> instances -- would probably solve your problem.  It would help ensure you
> would never have to wait on the timeouts -- the nodes will each explicitly
> tell ZK they are going bye-bye.
>
> if you do have things setup so that *manually* shutting down your
> instances executes a clean shutdown of solr, but AWS Auto-Scaling is
> actaully totally brutal and doesn't even do a clean shutdown of your
> virtual machines -- just yanks the virtual power cord -- perhaps you could
> implement one of these "LifecycleHook" options that poped up when i did
> some googling for "AWS Auto-Scale termination" to explicitly do a clean
> shutdown of the Solr process" before the machine vanishes into thin air?
>
>
>
> -Hoss
> http://www.lucidworks.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message