zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamas Penzes <tam...@cloudera.com.INVALID>
Subject Re: Configuring jute.maxbuffer on ZooKeeper Nodes
Date Tue, 18 Sep 2018 19:27:05 GMT
Hi Elbek,

jute.maxbuffer is stored as an int value, which means it's maximum value is
231-1 = 2147483647.
The buffer itself is an array, where the theoretical maximum size is the
same 231-1 = 2147483647 (but it depends on the JVM)
Of course it doesn't mean you should use that large values since you have
to store the buffer on the Java heap, which also limits the buffer size.
Just as Steph mentioned I would set it to 4MB too, you can go a bit higher,
but we should never use much higher numbers.

If you need a huge jute.maxbuffer it is a good idea to check your
application why you need it.

Regards, Tamaas

On Tue, Sep 18, 2018 at 12:31 PM Kamoliddinov, Elbek <
elbek.kamoliddinov@sap.com> wrote:

> Hi Zookeeper community, we had a system issue and we hit max buffer limit
> on zk side, Now to analyze I am trying to set up zk from snapshot that has
> big znode in it. When I start the standalone zk I get:
> 2018-09-18 10:21:13,358 [myid:] - INFO  [main:Util@190] - Invalid
> snapshot /path/tmp/version-2/snapshot.520000fe8e len = 808529968 byte = 56
>
> I tried various ways to set -Djute.maxbuffer=1610612736 but still get the
> same error.
> I changed zoo.cfg and added following line:
> jute.maxbuffer=1610612736
>
> and created zookeeper-env.sh file under conf directory where zoo.cfg and
> the file content is:
> JVMFLAGS="$JVMFLAGS -Djute.maxbuffer=1610612736"
>
> Also created java.env file under conf dir and it has:
> export JVMFLAGS="-Djute.maxbuffer=1610612736"
>
> still no luck.
>
> I tried to change zkServer.sh to have -Djute.maxbuffer=1610612736 as extra
> jvm param without any luck.
>
> On solr page it mentions 3 ways which I tried without any luck.
> https://lucene.apache.org/solr/guide/7_4/setting-up-an-external-zookeeper-ensemble.html
>
> Anybody has any idea what I might be doing wrong.
>
> Thanks.
> Elbek.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message