hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lauren Yang <Lauren.Y...@microsoft.com>
Subject RE: hive-site.xml not found on classpath
Date Fri, 30 Nov 2012 18:46:45 GMT
You can see if the classpath is being passed correctly to hadoop by putting in an echo statement
around line 150 of the hive cli script where it passes the CLASSPATH variable to HADOOP_CLASSPATH.
# pass classpath to hadoop
export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"

You could also echo the classpath in the hadoop script (in your HADOOP_HOME\bin directory)
to see if the classpath is being passed correctly to the time when the cli jar is invoked.

As far as the logs location, if this is not set in your hive-site.xml, you can set it by passing
 in HIVE_OPTS when you invoke the command line.

Like so:
EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"
Then run "hive"

Or:
Run "hive --hiveconf hive.log.dir=$ HIVE_HOME\logs"

Thanks,
Lauren


From: Stephen Boesch [mailto:javadba@gmail.com]
Sent: Friday, November 30, 2012 12:16 AM
To: user@hive.apache.org
Subject: Re: hive-site.xml not found on classpath

running 0.9.0 (you can see it from the classpath shown below);

steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve  1615 2012-11-13 23:37 hive-env.bullshit.sh<http://hive-env.bullshit.sh>
-rw-r--r-- 1 steve steve  1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve  1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve  1637 2011-12-13 00:36 hive-exec-log4j.properties.template
-rw-r--r-- 1 root  root   2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve  2056 2012-03-25 12:49 hive-log4j.properties.template
-rw-r--r-- 1 steve steve  4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive

also you can see the hive/conf is the first entry

After adding the debug statement:

classpath=/shared/hive/conf:/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:


But even so:

  *   the log dir is still wrong (writing to /tmp/${user}/hive.log instead of $HIVE_HOME/logs)
  *   the following message in the log file
2012-11-30 00:12:31,775 WARN  conf.HiveConf (HiveConf.java:<clinit>(70)) - hive-site.xml
not found on CLASSPATH





2012/11/30 Bing Li <sarah.libing@gmail.com<mailto:sarah.libing@gmail.com>>
which version of hive do you use?

Could you try to add the following debug line in bin/hive before hive real executes, and see
the result?

echo "CLASSPATH=$CLASSPATH"

if [ "$TORUN" = "" ]; then
   echo "Service $SERVICE not found"
   echo "Available Services: $SERVICE_LIST"
   exit 7
else
   $TORUN "$@"
fi

The version I used is 0.9.0


2012/11/30 Stephen Boesch <javadba@gmail.com<mailto:javadba@gmail.com>>
Yes i do mean the log is in the wrong location, since it was set to a persistent path in the
$HIVE_CONF_DIR/lhive-log4j.properties.

None of the files in that directory appear to be picked up properly: neither the hive-site.xml
nor log4j.properties.

I have put echo statements into the 'hive" and the hive-config.sh  shell scripts and the echo
statements prove that  HIVE_CONF_DIR is set properly:  /shared/hive/conf

But even so the following problems occur:

  *   the message "hive-site.xml is not found in the classpath"
  *   none of the hive-site.xml values are taking properly
  *   the log4j.properties in that same directory is not taking effect.


2012/11/29 Bing Li <sarah.libing@gmail.com<mailto:sarah.libing@gmail.com>>
Hi, Stephen
what did you mean the "wrong place under /tmp" in
"I am seeing the following message in the logs (which are in the wrong place under /tmp..)"
?

Did you mean that you set a different log dir but it didn't work?

the log dir should be set in conf/hive-log4j.properties, conf/hive-exec-log4j.properties
and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with 'export" command.

- Bing

2012/11/30 Stephen Boesch <javadba@gmail.com<mailto:javadba@gmail.com>>
thought i mentioned in the posts those were already set and verified.. but yes in any case
that's first thing looked at.

steve@mithril:~$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:~$ echo $HIVE_HOME
/shared/hive

2012/11/29 kulkarni.swarnim@gmail.com<mailto:kulkarni.swarnim@gmail.com> <kulkarni.swarnim@gmail.com<mailto:kulkarni.swarnim@gmail.com>>

Have you tried setting HIVE_HOME and HIVE_CONF_DIR?

On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <javadba@gmail.com<mailto:javadba@gmail.com>>
wrote:
Yes.

2012/11/29 Shreepadma Venugopalan <shreepadma@cloudera.com<mailto:shreepadma@cloudera.com>>
Are you seeing this message when your bring up the standalone hive cli by running 'hive'?

On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <javadba@gmail.com<mailto:javadba@gmail.com>>
wrote:
i am running under user steve.  the latest log (where this shows up ) is  /tmp/steve/hive.log

2012/11/29 Viral Bajaria <viral.bajaria@gmail.com<mailto:viral.bajaria@gmail.com>>
You are seeing this error when you run the hive cli or in the tasktracker logs when you run
a query ?

On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <javadba@gmail.com<mailto:javadba@gmail.com>>
wrote:

I am seeing the following message in the logs (which are in the wrong place under /tmp..)

hive-site.xml not found on classpath

My hive-site.xml is under the standard location  $HIVE_HOME/conf so this should not happen.

Now some posts have talked that the HADOOP_CLASSPATH was mangled.  Mine is not..

So what is the underlying issue here?

Thanks

stephenb







--
Swarnim






Mime
View raw message