Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <45f85f71002081645n3fe41e7btcbe0c214db6b6fd@mail.gmail.com>
References: 
 <BDDEFE43307872449C0847F2AC9729374F4BA8AFCD@GVW1122EXC.americas.hpqcorp.net>
	<45f85f71002081645n3fe41e7btcbe0c214db6b6fd@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Mon, 8 Feb 2010 17:10:42 -0800
Message-ID: <45f85f71002081710v5dc21a4fifdc986c9974dcf4f@mail.gmail.com>
Subject: Re: Name Node Corruption When Shutdown Too Soon
To: hdfs-dev@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Jonathan,

Another question: how have you configured dfs.name.dir? Do you have
several directories configured?

Thanks
-Todd

On Mon, Feb 8, 2010 at 4:45 PM, Todd Lipcon <todd@cloudera.com> wrote:
> Hey Jonathan,
>
> As Konstantin mentioned, I've been looking into a couple issues that
> could be related. At first glance it doesn't sound like you've run
> into quite the same thing.
>
> What version did you see this on? The steps to reproduce are something li=
ke:
>
> 1) Start a NN
> 2) Perform a bunch of edits so there is a large edit log
> 3) kill -9 the NN
> 4) start the NN again
> 5) while it is in the middle of replaying edits, kill -9 it again
> 6) start the NN, and lose all the previous edits?
>
> Or did I misunderstand what happened? If that sounds right, I'll give
> it a go and see if I can reproduce.
>
> Thanks
> -Todd
>
> On Sun, Feb 7, 2010 at 8:45 AM, Allen, Jonathan <jonathan.allen2@hp.com> =
wrote:
>> I've come across a name node bug and just wanted to check if it's a know=
n issue before I formally raise it (I've had a quick look through the datab=
ase but couldn't see anything obvious).
>>
>> If the name node is shut down before it has completed reading through th=
e edit log then the edit log gets removed without the image file being upda=
ted. =A0This results in name node reverting to its previously saved state (=
out of sync with the data nodes) and the most recent edits getting lost.
>>
>> Does anybody recognise this as a known issue or should I raise it?
>>
>> Thanks,
>> Jonathan Allen
>> UKGP, NS&R, Defence and Security
>> HP Enterprise Services
>> Telephone +44 1682 292101
>> Email jonathan.allen.uk@hp.com
>> Street address, Unit 29, Alexandra Way, Ashchurch Business Park, Tewkesb=
ury, Gloucestershire. GL20 8NB
>>
>> Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks R=
G12 1HN
>> Registered No: 690597 England
>> The contents of this message and any attachments to it are confidential =
and may be legally privileged. If you have received this message in error, =
you should delete it from your system immediately and advise the sender.
>> To any recipient of this message within HP, unless otherwise stated you =
should consider this message and attachments as "HP CONFIDENTIAL".
>>
>>
>>
>>
>