Mailing-List: contact connectors-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: connectors-user@incubator.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
From: "Fuad Efendi" <fuad@efendi.ca>
To: <connectors-user@incubator.apache.org>
References: <012001cbf2e5$a1e7b2a0$e5b717e0$@efendi.ca>
 <BANLkTikUWQr_zo_gS3f-fTYxGv0EHOpw-w@mail.gmail.com>
 <013a01cbf2e8$a57fed30$f07fc790$@efendi.ca>
 <014401cbf2ef$a5e9ce60$f1bd6b20$@efendi.ca>
In-Reply-To: <014401cbf2ef$a5e9ce60$f1bd6b20$@efendi.ca>
Subject: RE: Strange Classloader Problem
Date: Mon, 4 Apr 2011 14:12:30 -0400
Message-ID: <014b01cbf2f3$dce59800$96b0c800$@efendi.ca>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
Thread-Index: AQGHIvLnGXjJtkZRJ1W0IiKkLMAyogHXssVQAkNtCbUB1KsPF5SoIhkg
Content-Language: en-ca

Ok, I located the problem (is it really a problem?) and found workaround.

Scenario
1. Output connector has dependency on hbase.jar
2. hbase.jar has a file hbase-default.xml in a root (with default
configuration)
3. hbase can't load this

HBase uses this classloader:

    ClassLoader classLoader = Thread.currentThread().getContextClassLoader()


Then, it tries this:
    classLoader.getResource("hbase-default.xml")

and returns null (not found) if it runs under ManifoldCF... (or may by any
other web container?)


Workaround is explicit configuration for HBase
    Configuration config = new Configuration();
    File f = new
File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
    FileInputStream is = new FileInputStream(f);
    config.addResource(is);


-----Original Message-----
From: Fuad Efendi [mailto:fuad@efendi.ca] 
Sent: April-04-11 1:42 PM
To: connectors-user@incubator.apache.org
Subject: RE: Strange Classloader Problem


Ok, I found workaround... HBase configuration should be explicit,
    Configuration config = new Configuration();
    File f = new
File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
    FileInputStream is = new FileInputStream(f);
    config.addResource(is);


Default code (referred in many guides for beginners) does not work:

		config = HBaseConfiguration.create();


It will call implicitly this method of HBase:
  /**
   * Add a configuration resource. 
   * 
   * The properties of this resource will override properties of previously 
   * added resources, unless they were marked <a href="#Final">final</a>. 
   * 
   * @param name resource to be added, the classpath is examined for a file 
   *             with that name.
   */
  public void addResource(String name) {
    addResourceObject(name);
  }


And... somehow HBase run under ManifoldCF/connector-lib can't find this
resource (in a classpath)... I tried all imaginable methods...


-----Original Message-----
From: Fuad Efendi [mailto:fuad@efendi.ca]
Sent: April-04-11 12:52 PM
To: connectors-user@incubator.apache.org
Subject: RE: Strange Classloader Problem

Hi Karl,

I have hbase.jar file in connector-lib folder, and it seems it is loaded
except hbase-default.xml file inside this hbase.jar file;

1. java -jar start.jar
Caused by: java.lang.RuntimeException: hbase-default.xml file seems to be
for and old version of HBase (null), this version is 0.90.1
        at
org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfigu
ration.java:66)
        at
org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfigurat
ion.java:76)

2. Explicit "test' command such as this one works just fine (so that nothing
wrong with hbase.jar/hbase-default.xml):
java -cp
mcf-hbaseoutput-connector.jar;junit-4.7.jar;hbase-0.90.1.jar;hadoop-core-0.2
0-append-r1056497.jar;commons-logging.jar;zookeeper-3.3.2.jar;log4j-1.2.jar;
mcf-hbaseoutput-connector.jar
org.apache.manifoldcf.agents.output.hbaseconnector.HBaseTest


Putting hbase.jar into "lib" doesn't help; it should be inside
"connector-lib"; and ManifoldCF doesn't see this hbase-default.xml file...


-----Original Message-----
From: Karl Wright [mailto:daddywri@gmail.com]
Sent: April-04-11 12:36 PM
To: connectors-user@incubator.apache.org
Subject: Re: Strange Classloader Problem

The ManifoldCF class loader is used for loading classes, not for loading XML
files.  The only XML files ManifoldCF uses are its properties.xml file and
its connectors.xml.  There should be no reason to use a special class loader
to get these files, since they are needed to bootstrap the whole system.

Where are you trying to load XML files, and for what purpose?  The place you
access a file is where you would need to add support for loading it via the
class loader.

Karl

On Mon, Apr 4, 2011 at 12:30 PM, Fuad Efendi <fuad@efendi.ca> wrote:
> Hi,
>
>
>
> I am developing output connector to HBase.
>
>
>
> After many hours it seems to me ManifoldCF class loader doesn't see 
> any config files (*.XML) inside connector-lib/*.jar, and I can't even 
> manage it via explicit -classpath variable. is that true?
>
> Thanks,
>
> Fuad
>
>