hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Name Node Corruption When Shutdown Too Soon
Date Tue, 09 Feb 2010 00:45:24 GMT
Hey Jonathan,

As Konstantin mentioned, I've been looking into a couple issues that
could be related. At first glance it doesn't sound like you've run
into quite the same thing.

What version did you see this on? The steps to reproduce are something like:

1) Start a NN
2) Perform a bunch of edits so there is a large edit log
3) kill -9 the NN
4) start the NN again
5) while it is in the middle of replaying edits, kill -9 it again
6) start the NN, and lose all the previous edits?

Or did I misunderstand what happened? If that sounds right, I'll give
it a go and see if I can reproduce.

Thanks
-Todd

On Sun, Feb 7, 2010 at 8:45 AM, Allen, Jonathan <jonathan.allen2@hp.com> wrote:
> I've come across a name node bug and just wanted to check if it's a known issue before
I formally raise it (I've had a quick look through the database but couldn't see anything
obvious).
>
> If the name node is shut down before it has completed reading through the edit log then
the edit log gets removed without the image file being updated.  This results in name node
reverting to its previously saved state (out of sync with the data nodes) and the most recent
edits getting lost.
>
> Does anybody recognise this as a known issue or should I raise it?
>
> Thanks,
> Jonathan Allen
> UKGP, NS&R, Defence and Security
> HP Enterprise Services
> Telephone +44 1682 292101
> Email jonathan.allen.uk@hp.com
> Street address, Unit 29, Alexandra Way, Ashchurch Business Park, Tewkesbury, Gloucestershire.
GL20 8NB
>
> Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN
> Registered No: 690597 England
> The contents of this message and any attachments to it are confidential and may be legally
privileged. If you have received this message in error, you should delete it from your system
immediately and advise the sender.
> To any recipient of this message within HP, unless otherwise stated you should consider
this message and attachments as "HP CONFIDENTIAL".
>
>
>
>

Mime
View raw message