zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Keeney <...@fitterweb.com>
Subject Re: Ensemble fails when one node looses connectivity
Date Fri, 02 Mar 2018 13:54:39 GMT
Shawn -

Thanks for jumping in on the ZK side as well.

I will take a hard look at my config files but I checked and I do not have
any one file over 1MB. The combined files (10 indexes) is 2.2MB.

I am using micros for the nodes which are very limited in memory.

I'm not currently using a java.env file so I guess I'm using the default
values for the JVM which is typically xmx512M if I remember correctly.

Could it be just a memory issue?

JiM K.

On Thu, Mar 1, 2018 at 11:13 PM, Shawn Heisey <apache@elyograg.org> wrote:

> On 3/1/2018 7:59 PM, Jim Keeney wrote:
>> Read about the maxbuffer and am pretty sure that this might explain the
>> behavior we are seeing since it occurs when there has been a significant
>> reboot of all the servers. We have over 2 mb of config files for all of
>> our
>> indexes and if all the Solr nodes are sync ing their configs at once it
>> seems like that might overflow the buffer.
> You probably recognize me from the Solr side.  Hello again.  I do know
> enough to handle this part, so I'm answering. I didn't consider the
> maxbuffer setting, because I didn't see anything about large packets in the
> logs you shared on the Solr mailing list, and it's very rare for Solr users
> to need to increase it.
> You only need to worry about the maxbuffer if any single part of the
> config in ZK (what is called a "znode") is over 1MB. Each file in the
> configs that you upload will go into its own znode.  So if none of the
> individual files in your configs is really large, you probably won't need
> to set jute.maxbuffer.
> As for the other things that Solr puts in ZK:  Unless you have a REALLY
> huge cluster (tons of collections, shards, replicas, servers, etc) then
> that information should be quite small.
> Newbie question, where would i set the -Djute.maxbuffer ? Should I update
>> the zkServer.sh file so this is applied every time zookeeper is started or
>> restarted.
> If jute.maxbuffer is needed, it must be set on the startup options for
> every ZK server and every client that will access large znodes.  Which
> means all your ZK servers, all your Solr servers, and any invocations of
> things like the scripts Solr includes for uploading configs.
> Thanks,
> Shawn

Jim Keeney
President, FitterWeb
E: jim@fitterweb.com
M: 703-568-5887 <(703)%20568-5887>

*FitterWeb Consulting*
*Are you lean and agile enough? *

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message