flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: no valid hadoop home directory can be found
Date Wed, 23 Sep 2015 11:50:03 GMT
I don't think this is a critical issue. Its logged at DEBUG level.
I think you would have to specify the environment variable on all machines
in your cluster. BUT as far I know, HADOOP_HOME has been deprecated in
newer Hadoop (2.2+) versions.

On Wed, Sep 23, 2015 at 1:41 PM, Lydia Ickler <icklerly@googlemail.com>
wrote:

> Added the -Dhadoop.home.dir= to flink-conf.yaml but still get the same
> error.
> Could it be that it somehow has to be added to HBase directly?
>
>
> > Am 23.09.2015 um 13:27 schrieb Ufuk Celebi <uce@apache.org>:
> >
> > Either specify the env variable HADOOP_HOME or set the JVM property
> ‘hadoop.home.dir’ (-Dhadoop.home.dir=…)
> >
> > – Ufuk
> >
> >> On 23 Sep 2015, at 12:43, Lydia Ickler <icklerly@googlemail.com> wrote:
> >>
> >> Hi all,
> >>
> >> I get the following error message that no valid hadoop home directory
> can be found when trying to initialize the HBase configuration.
> >> Where would I specify that path?
> >>
> >> 12:41:02,043 INFO  org.apache.flink.addons.hbase.TableInputFormat
>           - Initializing HBaseConfiguration
> >> 12:41:02,174 DEBUG org.apache.hadoop.util.Shell
>           - Failed to detect a valid hadoop home directory
> >> java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
> >>        at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:225)
> >>        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:250)
> >>        at
> org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
> >>        at
> org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1514)
> >>        at
> org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:113)
> >>        at
> org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:259)
> >>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:159)
> >>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:134)
> >>        at
> org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.<init>(ZooKeeperKeepAliveConnection.java:43)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(HConnectionManager.java:1816)
> >>        at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:82)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:907)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:701)
> >>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> >>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> >>        at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >>        at
> java.lang.reflect.Constructor.newInstance(Constructor.java:408)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:457)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:436)
> >>        at
> org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:317)
> >>        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:198)
> >>        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:160)
> >>        at
> org.apache.flink.addons.hbase.TableInputFormat.createTable(TableInputFormat.java:89)
> >>        at
> org.apache.flink.addons.hbase.TableInputFormat.configure(TableInputFormat.java:78)
> >>        at
> org.apache.flink.runtime.operators.DataSourceTask.initInputFormat(DataSourceTask.java:273)
> >>        at
> org.apache.flink.runtime.operators.DataSourceTask.registerInputOutput(DataSourceTask.java:79)
> >>        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:501)
> >>        at java.lang.Thread.run(Thread.java:745)
> >>
> >
>
>

Mime
View raw message