hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henning Blohm <henning.bl...@zfabrik.de>
Subject Bug when reusing org.apache.hadoop.conf.Configuration in HbaseConfiguration
Date Wed, 13 Oct 2010 14:09:36 GMT
Hi,

  I used 

Configuration cfg = new Configuration();
cfg.addResource(InputStream) 

on org.apache.hadoop.conf.Configuration to create hadoop configuration.
That configuration is
used to construct HbaseConfiguration objects several times later on:

HbaseConfiguration hbcfg = new HbaseConfiguration(cfg)

At the second this leads to an error as the passed on configuraiton
object
is asked to load its properties again, trying to read from the
inputstream once more
(as the stream is memorized!!).

The fact that the properties of the passed on configuration object have
been reset somewhat 
surprising: When using the hbase configuration object constructed above
in HbaseAdmin, this 
will eventually load JobConf which has a static initializer that calls

Configuration.addDefaultResource("mapred-default.xml");

which in turn goes through the Configuration.REGISTRY which holds on to
any previously
not yet collected Configuration objects to (hold your breath!) force
them to reload their configuration
by calling reloadConfiguration that (now we are there) sets
properties=null.

Did anybody follow that....

It seems there is somewhat surprising side effects in hadoop/hbase
configuration handling.

Wouldn't it be better to have the default resource (pragmatically)
defined once in Configuration
and not (even think about) touch already instantiated config objects
later on?

Thanks,
  Henning


Ps.:
Here is one relevant stack trace:

Caused by: java.lang.RuntimeException: java.io.IOException: Ung├╝ltiger
Dateideskriptor
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1162)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1040)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980)
at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1016)
at
org.apache.hadoop.hbase.HBaseConfiguration.<init>(HBaseConfiguration.java:43)

<some removed>

... 56 more
Caused by: java.io.IOException: Ung├╝ltiger Dateideskriptor
at java.io.FileInputStream.read(Native Method)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager
$RewindableInputStream.read(XMLEntityManager.java:2933)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:704)
at
com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1099)



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message