accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dickson, Matt MR" <matt.dick...@defence.gov.au>
Subject RE: 200k fate locks in zookeeper [SEC=UNOFFICIAL]
Date Tue, 27 May 2014 23:04:51 GMT
UNOFFICIAL

Setting this via the accumulo-env.sh fixed the problem.

Thanks again for the swift response to my questions.

________________________________
From: Keith Turner [mailto:keith@deenlo.com]
Sent: Wednesday, 28 May 2014 01:48
To: user@accumulo.apache.org
Subject: Re: 200k fate locks in zookeeper [SEC=UNOFFICIAL]

The zookeeper admin manual say that prop is a java system property.  So one way to set it
is via the java command line.   Could modify accumulo-env.sh and modify the servers java options
to add -Djute.maxbuffer=XX  I am not sure what formats that property accepts.


On Mon, May 26, 2014 at 7:48 PM, Dickson, Matt MR <matt.dickson@defence.gov.au<mailto:matt.dickson@defence.gov.au>>
wrote:

UNOFFICIAL

I'm seeing an issue caused by there being almost 200k fate locks in zookeeper.  The problem
is that zookeeper is configured to return no more than 1mb of data for each query.  When the
Accumulo master queries the fate directory on startup the response exceeds the 1mb limit and
zookeeper disconnects the session.  Accumulo then retries the query, and it all happens again.
 We have managed to increase the query limit for the zkcli client using the jute.maxbuffer
setting, but are not clear how to setup Accumulo to use the larger query size when calling
the zookeeper jar.  We tried setting this through the zoo.cfg, but it doesn't seem to make
a difference.

For completeness, the reason there are so many fate locks is because the Accumulo binaries
were inadvertantly deleted while ingest and queries were running :( which has left things
in this state.

Any ideas would be great.

Thanks in advance,
Matt


Mime
View raw message