hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kihwal Lee <kih...@yahoo-inc.com>
Subject Re: corrupted edits log after power failure
Date Thu, 22 Sep 2011 17:37:04 GMT
Does the backup process include syncing? On-drive write cache can also trick you.
For absolutely critical data, it is a good idea to use a controller with battery-backed write
cache or a service/product that guarantees durability.


On 9/22/11 3:48 AM, "Gabi Kazav" <Gabi.Kazav@pursway.com> wrote:


I had Power Failure.
I have backup of files: edits, fsimage.

I am backing it up with:

curl -s http://nameNode:50070/getimage?getimage=1 > fsimage
curl -s http://nameNode:50070/getimage?getedits=1 > edits

When I am trying to start the HDFS with the recovered files, I got error about the edits file
: "Error replaying edit log at offset 1921"

Also, I have edits.new file, when I rename it to edits I got: "ERROR org.apache.hadoop.hdfs.server.common.Storage:
Error replaying edit log at offset 2494103"

What is the problem?!

And from now on, how can I do a backup that works?! :)


Gabi Kazav
IT Manager And Infrastructure Engineer
Gabi.Kazav@pursway.com<mailto:Gabi.Kazav@pursway.com> | www.pursway.com<http://www.pursway.com/>
Mailing address PO Box 4125, Herzliya 46140
Address 8 Hamada St., Herzliya, IL | Tel +972 527 772457| Fax + 972 9 958 4736

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message