hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan <danharve...@gmail.com>
Subject TableInputFormat configuration problems with 0.90
Date Wed, 23 Feb 2011 13:57:03 GMT
Hey,

I'm just testing our code to move over to 0.90 and I'm finding some issues
with the map/reduce jobs we've written using TableInputFormat.

We setup the jobs using TableMapReduceUtil.initTableMapperJob(..); which
worked fine in 0.20.6 but now throws the following errors when I try to run
them :-

11/02/23 12:38:44 ERROR zookeeper.ZKConfig: no clientPort found in zoo.cfg
11/02/23 12:38:44 ERROR mapreduce.TableInputFormat:
org.apache.hadoop.hbase.ZooKeeperConnectionException: java.io.IOException:
Unable to determine ZooKeeper ensemble
...
Exception in thread "main" java.io.IOException: No table was provided.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:130)
...

Which I think basically means it can't find the quorum/port for zookeeper in
the hbase configuration object from the hbase-*.xml files.
Looking into this a bit further it seems the TableInputFormat's setConf()
method creates the HTable using the Hadoop configuration object which
doesn't contain any of the HBase configuration...

So am I setting the job up incorrectly? If not it doesn't seem to make sense
to me that the HTable should be constructed with the hadoop configuration so
I guess there might be a bug in the TableInputFormat. Modifying it to not
take the hadoop configuration object allows the job to work correctly and
picks up the hbase configuration I've put on the class path,
https://gist.github.com/840403. If that's the case I'll file a bug report
and add the patch.

Thanks,

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message