manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Strange Classloader Problem
Date Mon, 04 Apr 2011 18:22:14 GMT
Hi Fuad,

At first glance this really does seem to be an HBase bug.  Instead of
using the thread classloader, it should be starting with whatever
classloader was used to load the HBase class.  Otherwise, it can't
reliably get at the configuration data in the jar (or anything else,
for that matter, that's located in the jar).

Perhaps you could file an HBase ticket.  For now, though, your
workaround doesn't seem to be too horrible.  Depending on what is
actually in the configuration xml, you may eventually want to include
the ability to set the configuration in the connector anyhow, since
ManifoldCF discourages external configuration files for connectors.

Karl


On Mon, Apr 4, 2011 at 2:12 PM, Fuad Efendi <fuad@efendi.ca> wrote:
> Ok, I located the problem (is it really a problem?) and found workaround.
>
> Scenario
> 1. Output connector has dependency on hbase.jar
> 2. hbase.jar has a file hbase-default.xml in a root (with default
> configuration)
> 3. hbase can't load this
>
> HBase uses this classloader:
>
>    ClassLoader classLoader = Thread.currentThread().getContextClassLoader()
>
>
> Then, it tries this:
>    classLoader.getResource("hbase-default.xml")
>
> and returns null (not found) if it runs under ManifoldCF... (or may by any
> other web container?)
>
>
> Workaround is explicit configuration for HBase
>    Configuration config = new Configuration();
>    File f = new
> File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
>    FileInputStream is = new FileInputStream(f);
>    config.addResource(is);
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: April-04-11 1:42 PM
> To: connectors-user@incubator.apache.org
> Subject: RE: Strange Classloader Problem
>
>
> Ok, I found workaround... HBase configuration should be explicit,
>    Configuration config = new Configuration();
>    File f = new
> File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
>    FileInputStream is = new FileInputStream(f);
>    config.addResource(is);
>
>
> Default code (referred in many guides for beginners) does not work:
>
>                config = HBaseConfiguration.create();
>
>
> It will call implicitly this method of HBase:
>  /**
>   * Add a configuration resource.
>   *
>   * The properties of this resource will override properties of previously
>   * added resources, unless they were marked <a href="#Final">final</a>.
>   *
>   * @param name resource to be added, the classpath is examined for a file
>   *             with that name.
>   */
>  public void addResource(String name) {
>    addResourceObject(name);
>  }
>
>
>
> And... somehow HBase run under ManifoldCF/connector-lib can't find this
> resource (in a classpath)... I tried all imaginable methods...
>
>
>
>
>
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: April-04-11 12:52 PM
> To: connectors-user@incubator.apache.org
> Subject: RE: Strange Classloader Problem
>
> Hi Karl,
>
> I have hbase.jar file in connector-lib folder, and it seems it is loaded
> except hbase-default.xml file inside this hbase.jar file;
>
> 1. java -jar start.jar
> Caused by: java.lang.RuntimeException: hbase-default.xml file seems to be
> for and old version of HBase (null), this version is 0.90.1
>        at
> org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfigu
> ration.java:66)
>        at
> org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfigurat
> ion.java:76)
>
> 2. Explicit "test' command such as this one works just fine (so that nothing
> wrong with hbase.jar/hbase-default.xml):
> java -cp
> mcf-hbaseoutput-connector.jar;junit-4.7.jar;hbase-0.90.1.jar;hadoop-core-0.2
> 0-append-r1056497.jar;commons-logging.jar;zookeeper-3.3.2.jar;log4j-1.2.jar;
> mcf-hbaseoutput-connector.jar
> org.apache.manifoldcf.agents.output.hbaseconnector.HBaseTest
>
>
> Putting hbase.jar into "lib" doesn't help; it should be inside
> "connector-lib"; and ManifoldCF doesn't see this hbase-default.xml file...
>
>
>
>
>
> -----Original Message-----
> From: Karl Wright [mailto:daddywri@gmail.com]
> Sent: April-04-11 12:36 PM
> To: connectors-user@incubator.apache.org
> Subject: Re: Strange Classloader Problem
>
> The ManifoldCF class loader is used for loading classes, not for loading XML
> files.  The only XML files ManifoldCF uses are its properties.xml file and
> its connectors.xml.  There should be no reason to use a special class loader
> to get these files, since they are needed to bootstrap the whole system.
>
> Where are you trying to load XML files, and for what purpose?  The place you
> access a file is where you would need to add support for loading it via the
> class loader.
>
> Karl
>
> On Mon, Apr 4, 2011 at 12:30 PM, Fuad Efendi <fuad@efendi.ca> wrote:
>> Hi,
>>
>>
>>
>> I am developing output connector to HBase.
>>
>>
>>
>> After many hours it seems to me ManifoldCF class loader doesn't see
>> any config files (*.XML) inside connector-lib/*.jar, and I can't even
>> manage it via explicit -classpath variable. is that true?
>>
>> Thanks,
>>
>> Fuad
>>
>>
>
>

Mime
View raw message