hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre Reiter <a.rei...@web.de>
Subject Re: HBase & MapReduce & Zookeeper
Date Thu, 28 Jul 2011 20:55:52 GMT
i guess, i know the reason, why  HConnectionManager.deleteConnection(conf, true); does not
work for me

in the MR job im using TableInputFormat, if you have a look at the source code in the method
public void setConf(Configuration configuration)
there is a line creating the HTable like this :

setHTable(new HTable(new Configuration(conf), tableName));

now the HTable is using a new configuration, why???

HConnectionManager holds all  Connections in a map with the config as a key:
private static final Map<Configuration, HConnectionImplementation> HBASE_INSTANCE =

Configuration does not override the equals and hash methods, so the HConnection created by
the table using the new Configuration can not be referenced...

just looking at executed code using debugger, i found out, that there is even one place more,
where another new configuration is created and used:

JobContext context = new JobContext(jobCopy, jobId);

the JobContext constructor creates a new Configuration object: new org.apache.hadoop.mapred.JobConf(conf)

so there is no real chance for me to get the configuration, which was used during the call
to HConnectionManager.getConnection(conf)

that is why HConnectionManager.deleteConnection(conf, true); does not work for me, or am i
completely wrong???

just going crazy with that! i'm not doing anything very special, all what i want, is to run
a MR job out of my application, and NOT by calling it from the shell like this:
./bin/hadoop jar /tmp/my.jar package.OurClass

View raw message