zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Recovery log on /dev/shm?
Date Thu, 01 Sep 2011 20:51:06 GMT
It has come up before, and it's a reasonable option in some cases,
however many ZK users are concerned about the failure cases this
doesn't handle - such as when all machines in the cluster are
restarted concurrently. (power failure say)

The other reason it's not more popular is that most users are not
concerned about the small amount of latency introduced by the
persistent storage.

Patrick

On Thu, Sep 1, 2011 at 1:31 PM, Ashwin Jayaprakash
<ashwin.jayaprakash@gmail.com> wrote:
> This is a follow up to a question I posted in July 2010 - about whether the
> recovery log could be redirected to /dev/null or some such low latency drive
> [http://markmail.org/thread/3edpfmvn2fhyg5wv].
>
> I just found out that Linux distros with 2.6 kernel have a built-in shared
> memory file system that runs purely in memory - /dev/shm. Since this
> provides recovery across JVM restarts (but not OS restarts) wouldn't this be
> sufficient to run the ZK nodes off this drive? Replicas are on other
> machines anyway so FT/HA is already there. It's just that disk latency can
> be eliminated completely.
>
> [Apologies if you guys are already aware of /dev/shm]
>
> I ran some simple Java IO tests (not full ZK) on this drive and I wanted to
> share my results. Might be of some use to apps that require low latency:
> http://javaforu.blogspot.com/2011/09/ram-disk-is-already-in-linux-and-nobody.html
>
> Ashwin.
>

Mime
View raw message