Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@zookeeper.apache.org
Received-SPF: pass (athena.apache.org: domain of metacret@gmail.com designates
 209.85.216.50 as permitted sender)
MIME-Version: 1.0
Date: Wed, 4 Dec 2013 11:29:42 -0800
Message-ID: 
 <CAKe7ALdndwP_YTcbbvdZZ+GHCNTcT9W2j3NBQcMMaoTNY=m3WA@mail.gmail.com>
Subject: Recurring "Failed to process transaction type: 1 error:
 KeeperErrorCode = NoNode for..."
From: "Bae, Jae Hyeon" <metacret@gmail.com>
To: user@zookeeper.apache.org
Content-Type: multipart/alternative; boundary=001a11c2b266541d6804ecba6f45

--001a11c2b266541d6804ecba6f45
Content-Type: text/plain; charset=ISO-8859-1

Hi Zookeeper users

When the leader zookeeper instance is killed, theoretically, all followers
should restart and elect the new leader gracefully. But I am observing a
little frequently, all followers cannot start due to

"Failed to process transaction type: 1 error: KeeperErrorCode = NoNode
for..." "Unable to load database on disk"

So, I had to manually copy the file from the observer to each instance and
restart them.

What would be the root cause? Even though a few clients are heavily sending
the write requests including delete, create, setData, zookeeper shouldn't
make the corrupted data correct?

Is this disk IO performance or write cache problem? We're running zookeeper
on AWS EC2 environment, so we are using the same disk for snapshot
directory and log directory, also we didn't turn off disk write cache.

Thank you
Best, Jae

--001a11c2b266541d6804ecba6f45--