hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Niels Basjes (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14750) HBaseConfiguration.create() doesn't load properties
Date Mon, 09 Nov 2015 15:27:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14996708#comment-14996708

Niels Basjes commented on HBASE-14750:

I am working on FLINK-2977 and as a test I did the following.

I added:
# the HBASE_CONF_DIR to the classpath (the -classpath command line parameter) 
# the HBASE_CONF_DIR to the environment variable CLASSPATH
# this code fragment to the part I'm working on:
LOG.info("hbase.zookeeper.quorum = {}", conf.get("hbase.zookeeper.quorum"));
LOG.info("CLASSPATH = {}", System.getenv("CLASSPATH"));

int i = 0;
for(URL url: ((URLClassLoader)ClassLoader.getSystemClassLoader()).getURLs()){
	LOG.info("CLASSPATH [{}]: {}" , i , url.getFile());

String hbaseConfDir = System.getenv("HBASE_CONF_DIR");
LOG.info("HBASE_CONF_DIR = {}", hbaseConfDir );
if (hbaseConfDir == null) {
	hbaseConfDir = "/etc/hbase/conf";
LOG.info("Final HBASE_CONF_DIR = {}", hbaseConfDir );
conf.addResource(new Path(hbaseConfDir + "/hbase-site.xml"));
LOG.info("hbase.zookeeper.quorum = {}", conf.get("hbase.zookeeper.quorum"));

the output looks like this:
hbase.zookeeper.quorum = null
CLASSPATH = :/home/nbasjes/hadoop-environment-configs/hdp-b/hadoop/conf/:/home/nbasjes/hadoop-environment-configs/hdp-b/hadoop/conf/:/home/nbasjes/hadoop-environment-configs/hdp-b/hbase/conf/
CLASSPATH [1]: /home/nbasjes/bin/flink-0.9.1_kerberos/lib/flink-dist-1.0-SNAPSHOT.jar
CLASSPATH [2]: /home/nbasjes/bin/flink-0.9.1_kerberos/lib/flink-python-1.0-SNAPSHOT.jar
CLASSPATH [3]: /home/nbasjes/bin/flink-0.9.1_kerberos/lib/log4j-1.2.17.jar
CLASSPATH [4]: /home/nbasjes/bin/flink-0.9.1_kerberos/lib/slf4j-log4j12-1.7.7.jar
CLASSPATH [5]: /home/nbasjes/
CLASSPATH [6]: /home/nbasjes/hadoop-environment-configs/hdp-b/hadoop/conf/
CLASSPATH [7]: /home/nbasjes/hadoop-environment-configs/hdp-b/hadoop/conf/
CLASSPATH [8]: /home/nbasjes/hadoop-environment-configs/hdp-b/hbase/conf/
HBASE_CONF_DIR = /home/nbasjes/hadoop-environment-configs/hdp-b/hbase/conf/
Final HBASE_CONF_DIR = /home/nbasjes/hadoop-environment-configs/hdp-b/hbase/conf/
hbase.zookeeper.quorum = master-001.xxxxxx.net:2181,master-002.xxxxxx.net:2181,master-003.xxxxxx.net:2181

As you can see the SystemClassLoader has the hbase config directory in the classpath.
Only after I explicitly add the hbase-site.xml as an additional resource does the hbase.zookeeper.quorum
get filled with the correct values.

What am I doing wrong?

> HBaseConfiguration.create() doesn't load properties
> ---------------------------------------------------
>                 Key: HBASE-14750
>                 URL: https://issues.apache.org/jira/browse/HBASE-14750
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Niels Basjes
> While writing an application that uses HBase I ran into the problem that although I have
setup the hbase-site.xml in the {{HBASE_CONF_DIR}}; the settings in this file are not picked
up in my application.
> In several places I find instructions to include things like the {{hbase.zookeeper.quorum}}
in a properties file and the explicitly set these value from within the application (I have
actually done this in the past quite a few times).
> Although this works I expect the system to automatically pickup the settings I have installed
already which are picked up by tools like the hbase shell and pig.
> So I ended up writing this helper method:
> {code:java}
>     public static Configuration createConfiguration() {
>         String hbaseConfDir = System.getenv("HBASE_CONF_DIR");
>         if (hbaseConfDir == null) {
>             hbaseConfDir = "/etc/hbase/conf";
>         }
>         Configuration conf = HBaseConfiguration.create();
>         conf.addResource(new Path(hbaseConfDir + "/hbase-site.xml"));
>         return conf;
>     }
> {code}
> I expect HBaseConfiguration.create() to give me a working config in an environment where
everything for all the other HBase clients has already been setup correctly.
> My proposal is to change the HBaseConfiguration.create() to effectively include what
my helper method does.

This message was sent by Atlassian JIRA

View raw message