zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: Recovery log on /dev/shm?
Date Thu, 01 Sep 2011 22:32:49 GMT
I had done quite a bit of benchmarks on not logging to disk.
See ZOOKEEPER-866

and noticed that it actually wasnt of much help. It doesnt improve the latency by much and
also the throughput was still the same.

hope that helps.

mahadev

On Sep 1, 2011, at 2:24 PM, Benjamin Reed wrote:

> have you benchmarked the difference between using the ram disk and
> turning off sync with forceSync=no? i would think that the performance
> will be similar. you will need to use the code in trunk because
> https://issues.apache.org/jira/browse/ZOOKEEPER-1087 isn't in any of
> the releases yet.
> 
> thanx
> ben
> 
> On Thu, Sep 1, 2011 at 1:31 PM, Ashwin Jayaprakash
> <ashwin.jayaprakash@gmail.com> wrote:
>> This is a follow up to a question I posted in July 2010 - about whether the
>> recovery log could be redirected to /dev/null or some such low latency drive
>> [http://markmail.org/thread/3edpfmvn2fhyg5wv].
>> 
>> I just found out that Linux distros with 2.6 kernel have a built-in shared
>> memory file system that runs purely in memory - /dev/shm. Since this
>> provides recovery across JVM restarts (but not OS restarts) wouldn't this be
>> sufficient to run the ZK nodes off this drive? Replicas are on other
>> machines anyway so FT/HA is already there. It's just that disk latency can
>> be eliminated completely.
>> 
>> [Apologies if you guys are already aware of /dev/shm]
>> 
>> I ran some simple Java IO tests (not full ZK) on this drive and I wanted to
>> share my results. Might be of some use to apps that require low latency:
>> http://javaforu.blogspot.com/2011/09/ram-disk-is-already-in-linux-and-nobody.html
>> 
>> Ashwin.
>> 


Mime
View raw message