Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of stucchio@gmail.com designates
 209.85.220.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=N0i58r3jrO4dFIiOiVJ2BbZem40ge2936biP/LWId47Zrjsk9OltQRyydKFifwq99d
         iaQjfrvRSXZJE7dM4L3arRXLGVlu6Lyb+fQd2UFLGLEvYwqfxQojEhnu02YljO1815KA
         JvGT3iQib08Do9E20HSpaTdpzPjyWNJo0J3M4=
MIME-Version: 1.0
In-Reply-To: <AANLkTimus2EHrh1a5et6EZTtG-nDfVB8s-boPjEpnrEC@mail.gmail.com>
References: <AANLkTi=O3DWzjQsZNnCpAvoSw0nj4j4CozMY86KOXpyc@mail.gmail.com>
	<AANLkTimfDJOKz_-VM+B0NKZAtLfi0V8tva-zFRQXSoQ9@mail.gmail.com>
	<AANLkTimus2EHrh1a5et6EZTtG-nDfVB8s-boPjEpnrEC@mail.gmail.com>
Date: Sun, 20 Feb 2011 14:14:42 -0500
Message-ID: <AANLkTimzxtF_GD7bz4whCHU+THn8OkcNCSyuTmbf-ttG@mail.gmail.com>
Subject: Re: Hadoop fsck failure
From: Christian Stucchio <stucchio@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

It's definitely uninitialized. From the namenode log:

2011-02-20 22:38:31,789 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2011-02-20 22:38:31,835 INFO org.apache.hadoop.http.HttpServer: Port
returned by webServer.getConnectors()[0].getLocalPort() before open()
is -1. Opening the listener on 50070
2011-02-20 22:38:31,836 INFO org.apache.hadoop.http.HttpServer:
listener.getLocalPort() returned 50070
webServer.getConnectors()[0].getLocalPort() returned 50070
2011-02-20 22:38:31,836 INFO org.apache.hadoop.http.HttpServer: Jetty
bound to port 50070
2011-02-20 22:38:31,836 INFO org.mortbay.log: jetty-6.1.14
2011-02-20 22:38:31,932 WARN org.mortbay.log: Failed startup of
context org.mortbay.jetty.webapp.WebAppContext@7c19f9d2{/,file:/usr/local/hadoop/webapps/hdfs}
java.util.zip.ZipException: error in opening zip file
	at java.util.zip.ZipFile.open(Native Method)
	at java.util.zip.ZipFile.<init>(ZipFile.java:114)
	at java.util.jar.JarFile.<init>(JarFile.java:135)
	at java.util.jar.JarFile.<init>(JarFile.java:99)
	at org.mortbay.jetty.webapp.TagLibConfiguration.configureWebApp(TagLibConfiguration.java:168)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1231)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:460)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
	at org.mortbay.jetty.Server.doStart(Server.java:222)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.apache.hadoop.http.HttpServer.start(HttpServer.java:461)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:246)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:202)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
2011-02-20 22:38:31,939 INFO org.mortbay.log: Started
SelectChannelConnector@0.0.0.0:50070

Googling this error only found this page, which suggests that the
JVM's cache of jar files might have been modified somehow. I don't
fully understand the error.

http://docs.codehaus.org/display/JETTY/ZIP+exception+reading+jar+or+war+files

However, on the theory that something was wrong with a jar file
somewhere, I reinstalled the hadoop jars on my server. This resolved
the issue.


Thanks for your help,


Chris


On Sun, Feb 20, 2011 at 11:47 AM, Harsh J <qwertymaniac@gmail.com> wrote:
> Hello,
>
> There could be something wrong with the Jetty webserver that starts
> along with a daemon (Used by the Reducer to fetch map outputs, and
> seemingly also by fsck from NN's server). Perhaps the logs (of NN, TT)
> can tell you more on what may have happened when it was initializing
> the Jetty HTTP service. Look at the initializing part (early up in the
> logs, during start-up).
>
> Quite possibly, it may be down/uninitialized due to some reason. Are
> you able to view the Web UIs of each service (services that publish
> one)?
>
> --
> Harsh J
> www.harshj.com
>