accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject HA namenode questions
Date Fri, 14 Mar 2014 19:18:04 GMT
For those of you running HA NN on large clusters, I'm looking for some
advice.

I was looking at an HA NN config today.  Either by default, or by following
the configuration instructions, I saw that the zookeeper timeout was set to
5 seconds.

* is this a reasonable timeout?
* do you provide HA NN its own set of zookeepers?

We have seen problems with large GC pauses with tablet servers.  This
happens less and less as we have learned more tricks, but I'm constantly
talking to users who want their zookeeper timeout as high as two minutes.

We have also had to increase the number of zookeepers on our largest
clusters in order to handle the "thundering herd" load when large
map/reduce jobs kick off and they all start talking to accumulo, which
requires reading information from zookeeper.

Any experience you can share about HA NN configuration at scales over few
hundred nodes would be appreciated.

-Eric

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message