Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 296B0101C7 for ; Thu, 3 Oct 2013 10:06:37 +0000 (UTC) Received: (qmail 79789 invoked by uid 500); 3 Oct 2013 10:06:29 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 79697 invoked by uid 500); 3 Oct 2013 10:06:28 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 79688 invoked by uid 99); 3 Oct 2013 10:06:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Oct 2013 10:06:26 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jeetuyadav200890@gmail.com designates 209.85.217.174 as permitted sender) Received: from [209.85.217.174] (HELO mail-lb0-f174.google.com) (209.85.217.174) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Oct 2013 10:06:22 +0000 Received: by mail-lb0-f174.google.com with SMTP id w6so1795957lbh.19 for ; Thu, 03 Oct 2013 03:06:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0aBfGW6+EZ7owE6lD7BIKcThg99rG5qb0iUgUhw0urg=; b=Cb/noxwo2s9XrWsQH60m/wPYgY35sfVZxuziMkcQx7Z0KbhNrMK9lu3orh0Av+IlGh WVxJsGTcJWbDvcH1fsAR06NKke40ucK1FaBpdypy4sZnhxwolRgbyiDs8wOcyjKiLAQZ 7W25bBAG1wV6lAnvD2Vnm2Crx61rXnWneQHkHrqfe20ttQksD6o4D9Z4yCR5Qj4INy6P NMpQGE/MB/YmhFkfRPfruvl05YvbDdGI7lyjn/fLqLslDxVNw3E3Ld9R0GTb3vhe+BHr ftaVIw6972MzN6KAY2xrOBD3gmqdup15HKa5ouW9ASYWj8qQRnXpRTUtwzkMiWd3mtat pREQ== MIME-Version: 1.0 X-Received: by 10.152.120.228 with SMTP id lf4mr634644lab.44.1380794760545; Thu, 03 Oct 2013 03:06:00 -0700 (PDT) Received: by 10.112.180.70 with HTTP; Thu, 3 Oct 2013 03:06:00 -0700 (PDT) In-Reply-To: <524D3EE9.6070305@gmail.com> References: <524D2069.5080900@gmail.com> <524D222E.10107@gmail.com> <524D2D41.8080806@gmail.com> <524D3EE9.6070305@gmail.com> Date: Thu, 3 Oct 2013 15:36:00 +0530 Message-ID: Subject: Re: Secondary NameNode doCheckpoint Exception From: Jitendra Yadav To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e0117691535cb5d04e7d35532 X-Virus-Checked: Checked by ClamAV on apache.org --089e0117691535cb5d04e7d35532 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable When you said "started from java" does that means you are doing some development work using eclipse and hadoop? May be I'm not familiar with this kind of environment. Regards Jitendra On Thu, Oct 3, 2013 at 3:24 PM, Furkan B=C4=B1=C3=A7ak = wrote: > I don't have any problem when starting and stoping the cluster with > scripts. I am having this problem when starting the cluster from Java. > > Also, when I start the cluster from Java, I can run map-reduce jobs and i= t > succesfully finishes. However, after some time about 5 minutes when the > secondary namenode is started, it tries the merge and then gives that err= or. > > Thanks, > Frkn. > > On 03-10-2013 12:10, Jitendra Yadav wrote: > > Can you restart your cluster using below scripts and check the logs? > > # stop-all.sh > #start-all.sh > > Regards > Jitendra > > > On Thu, Oct 3, 2013 at 2:09 PM, Furkan B=C4=B1=C3=A7ak wrote: > >> No, I started from scratch. >> >> Thanks, >> Frkn. >> >> >> On 03-10-2013 11:32, Jitendra Yadav wrote: >> >> Did you upgraded your cluster ? >> >> Regards >> Jitendra >> >> >> On Thu, Oct 3, 2013 at 1:22 PM, Furkan B=C4=B1=C3=A7ak wrote: >> >>> On 03-10-2013 10:44, Furkan B=C4=B1=C3=A7ak wrote: >>> >>>> Hi Everyone, >>>> >>>> I am starting my hadoop cluster manually from Java which works fine >>>> until some time. When secondary NameNode tries to do merge it gives me= the >>>> following exception: >>>> >>>> --------------------------- >>>> 13/10/03 10:29:38 ERROR namenode.SecondaryNameNode: Throwable Exceptio= n >>>> in doCheckpoint: >>>> 13/10/03 10:29:38 ERROR namenode.SecondaryNameNode: >>>> java.lang.AssertionError: Negative layout version is expected. >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.jav= a:857) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointSto= rage.doMerge(SecondaryNameNode.java:780) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointSto= rage.access$600(SecondaryNameNode.java:680) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doMerge(Secon= daryNameNode.java:557) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(= SecondaryNameNode.java:521) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(Second= aryNameNode.java:396) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(Secondary= NameNode.java:360) >>>> at java.lang.Thread.run(Thread.java:724) >>>> >>>> java.lang.AssertionError: Negative layout version is expected. >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.jav= a:857) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointSto= rage.doMerge(SecondaryNameNode.java:780) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointSto= rage.access$600(SecondaryNameNode.java:680) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doMerge(Secon= daryNameNode.java:557) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(= SecondaryNameNode.java:521) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(Second= aryNameNode.java:396) >>>> at >>>> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(Secondary= NameNode.java:360) >>>> at java.lang.Thread.run(Thread.java:724) >>>> --------------------------- >>>> >>>> What causes this exception to occur? >>>> Any help would be appreciated! >>>> >>>> Kind Regards, >>>> >>>> Furkan Bicak. >>>> >>>> I forgot to mention the hadoop version. I am using hadoop 1.2.1 >>> version. >>> >>> Thanks. >>> >> >> >> > > --089e0117691535cb5d04e7d35532 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
When you said "started from java" does that mean= s you are doing some development work using eclipse and hadoop?

May be I'm not familiar with this kind of environment.

Regards
Jitendra


On Thu, Oct 3, 2013 at 3:24 PM, Fur= kan B=C4=B1=C3=A7ak <bicak.cs@gmail.com> wrote:
=20 =20 =20
I don't have any problem when starting and stoping the cluster with scripts. I am having this problem when starting the cluster from Java.

Also, when I start the cluster from Java, I can run map-reduce jobs and it succesfully finishes. However, after some time about 5 minutes when the secondary namenode is started, it tries the merge and then gives that error.

Thanks,
Frkn.

On 03-10-2013 12:10, Jitendra Yadav wrote:
Can you restart your cluster using below scripts and check the logs?

# stop-all.sh
#start-all.sh

Regards
Jitendra


On Thu, Oct 3, 2013 at 2:09 PM, Furkan B=C4=B1=C3=A7ak <bicak.cs@gmail.com> wrote:
No, I started from scratch.

Thanks,
Frkn.


On 03-10-2013 11:32, Jitendra Yadav wrote:
Did you upgraded your cluster ?

Regards
Jitendra


On Thu, Oct 3, 2013 at 1:22 PM, Furkan B=C4=B1=C3=A7ak &= lt;bicak.cs@gmail.c= om> wrote:
On 03-10-2013 10:44, Furkan B=C4=B1=C3=A7a= k wrote:
Hi Everyone,

I am starting my hadoop cluster manually from Java which works fine until some time. When secondary NameNode tries to do merge it gives me the following exception:

---------------------------
13/10/03 10:29:38 ERROR namenode.SecondaryNameNode: Throwable Exception in doCheckpoint:
13/10/03 10:29:38 ERROR namenode.SecondaryNameNode: java.lang.AssertionError: Negative layout version is expected.
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:857= )
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointStorage.= doMerge(SecondaryNameNode.java:780)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointStorage.= access$600(SecondaryNameNode.java:680)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doMerge(SecondaryN= ameNode.java:557)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(Secon= daryNameNode.java:521)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNa= meNode.java:396)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameN= ode.java:360)
=C2=A0 =C2=A0 at java.lang.Thread.run(Thread.java:724)

java.lang.AssertionError: Negative layout version is expected.
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:857= )
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointStorage.= doMerge(SecondaryNameNode.java:780)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode$CheckpointStorage.= access$600(SecondaryNameNode.java:680)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doMerge(SecondaryN= ameNode.java:557)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(Secon= daryNameNode.java:521)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNa= meNode.java:396)
=C2=A0 =C2=A0 at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameN= ode.java:360)
=C2=A0 =C2=A0 at java.lang.Thread.run(Thread.java:724)
---------------------------

What causes this exception to occur?
Any help would be appreciated!

Kind Regards,

Furkan Bicak.

I forgot to mention the hadoop version. I am using hadoop 1.2.1 version.

Thanks.





--089e0117691535cb5d04e7d35532--