Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Received-SPF: pass (athena.apache.org: domain of misapache@gmail.com
 designates 209.85.214.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=IWnkyg4DNB28xkl4nsI+z7emctdeOnhFGP5xcU/XOfcDKOi7V2crVJRoN0Wca0aiXx
         1L7HGxU0Nae1P/rHozs62uEpvs0N8jrAnFrlr1g8IwGGL+YHg7AjoqMDWu1g6OTpbhks
         B9VKMtxHSABIStDrdGj3rWZlR3k+WMdEVX8SQ=
MIME-Version: 1.0
Date: Mon, 23 May 2011 15:26:06 +0530
Message-ID: <BANLkTinb4C6KDiN7CQuXcks=bZ+h=5K_Fg@mail.gmail.com>
Subject: An issue with Hive on hadoop cluster
From: MIS <misapache@gmail.com>
To: user@hive.apache.org, hive <dev@hive.apache.org>
Content-Type: multipart/alternative; boundary=20cf3054a9f3eeacbe04a3ee7a9d

--20cf3054a9f3eeacbe04a3ee7a9d
Content-Type: text/plain; charset=ISO-8859-1

I'm getting into an issue when trying to run hive over the hadoop cluster.

The hadoop cluster is working fine, in a stand alone manner.
I'm using hadoop 0.20.2 and hive 0.7.0 versions.

The problem is that the hive is not considering the fs.default.name property
that I am setting in the core-site.xml or the mapred.job.tracker in the
mapred-site.xml files.
It always considers that namenode can be accessed at localhost (refer to the
stack trace below)
So I have specified these properties in the hive-site.xml file as well. I
tried making them as final in the hive-site.xml file, but didn't get the
intended result.
Further, I set the above properties through command line as well. Again, no
success.

I looked at the hive code for 0.7.0 branch to debug the issue, to see if it
getting fs.default.name property from the file hive-site.xml, which it does
through clone of the JobConf. So no issues here.

Further, in hive-site.xml, if I make any of the properties as final, then
hive gives me a WARNING log. as below :

*WARN  conf.Configuration (Configuration.java:loadResource(1154)) -
file:/usr/local/hive-0.7.0/conf/hive-site.xml:a attempt to override final
parameter: hive.metastore.warehouse.dir;  Ignoring.*

>From the above message I can assume that it has already read the
property(don't know from where, or it may be trying to read the property
multiple times), but I have explicitly specified the hive conf folder in the
hive-env.sh.

Below is the stack trace I'm getting in the log file:
*2011-05-23 15:11:00,793 ERROR CliDriver (SessionState.java:printError(343))
- Failed with exception java.io.IOException:java.net.ConnectException: Call
to localhost/127.0.0.1:54310 failed on connection exception:
java.net.ConnectException: Connection refused
java.io.IOException: java.net.ConnectException: Call to localhost/
127.0.0.1:54310 failed on connection exception: java.net.ConnectException:
Connection refused
    at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:341)
    at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:133)
    at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1114)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.net.ConnectException: Call to
localhost/127.0.0.1:54310failed on connection exception:
java.net.ConnectException: Connection
refused
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:767)
    at org.apache.hadoop.ipc.Client.call(Client.java:743)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
    at $Proxy4.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
    at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
    at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
    at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
    at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextPath(FetchOperator.java:241)
    at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:259)
    at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:320)
    ... 10 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
    at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
    at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304)
    at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:860)
    at org.apache.hadoop.ipc.Client.call(Client.java:720)
    ... 25 more
*
Has anybody encountered similar issues earlier ? any thoughts towards
resolving the above issue would be helpful

Thanks.

--20cf3054a9f3eeacbe04a3ee7a9d--