Mailing-List: contact connectors-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: connectors-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of daddywri@gmail.com designates
 209.85.216.175 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=vnjcAqtdR7LtuV2DvOTwj/xmCUaXAhEhu6K2FDq1QuGPZ/l4dcyxrcZ/Ufowu12Ql6
         Pk8zCK0kNliu2K1nj++tp5fZ4FB1ofCSZsyEunFy40ucQbZM++3r/vha42H1QcejQUeO
         m3wiTHd32gzU+rHA8/3oxuIBUNOE7mIgn3P6E=
MIME-Version: 1.0
In-Reply-To: <014b01cbf2f3$dce59800$96b0c800$@efendi.ca>
References: <012001cbf2e5$a1e7b2a0$e5b717e0$@efendi.ca>
	<BANLkTikUWQr_zo_gS3f-fTYxGv0EHOpw-w@mail.gmail.com>
	<013a01cbf2e8$a57fed30$f07fc790$@efendi.ca>
	<014401cbf2ef$a5e9ce60$f1bd6b20$@efendi.ca>
	<014b01cbf2f3$dce59800$96b0c800$@efendi.ca>
Date: Mon, 4 Apr 2011 14:22:14 -0400
Message-ID: <BANLkTimfbkANCGVqr4Vv8_oHjYS6RquVow@mail.gmail.com>
Subject: Re: Strange Classloader Problem
From: Karl Wright <daddywri@gmail.com>
To: connectors-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Fuad,

At first glance this really does seem to be an HBase bug.  Instead of
using the thread classloader, it should be starting with whatever
classloader was used to load the HBase class.  Otherwise, it can't
reliably get at the configuration data in the jar (or anything else,
for that matter, that's located in the jar).

Perhaps you could file an HBase ticket.  For now, though, your
workaround doesn't seem to be too horrible.  Depending on what is
actually in the configuration xml, you may eventually want to include
the ability to set the configuration in the connector anyhow, since
ManifoldCF discourages external configuration files for connectors.

Karl


On Mon, Apr 4, 2011 at 2:12 PM, Fuad Efendi <fuad@efendi.ca> wrote:
> Ok, I located the problem (is it really a problem?) and found workaround.
>
> Scenario
> 1. Output connector has dependency on hbase.jar
> 2. hbase.jar has a file hbase-default.xml in a root (with default
> configuration)
> 3. hbase can't load this
>
> HBase uses this classloader:
>
> =A0 =A0ClassLoader classLoader =3D Thread.currentThread().getContextClass=
Loader()
>
>
> Then, it tries this:
> =A0 =A0classLoader.getResource("hbase-default.xml")
>
> and returns null (not found) if it runs under ManifoldCF... (or may by an=
y
> other web container?)
>
>
> Workaround is explicit configuration for HBase
> =A0 =A0Configuration config =3D new Configuration();
> =A0 =A0File f =3D new
> File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
> =A0 =A0FileInputStream is =3D new FileInputStream(f);
> =A0 =A0config.addResource(is);
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: April-04-11 1:42 PM
> To: connectors-user@incubator.apache.org
> Subject: RE: Strange Classloader Problem
>
>
> Ok, I found workaround... HBase configuration should be explicit,
> =A0 =A0Configuration config =3D new Configuration();
> =A0 =A0File f =3D new
> File("C:\\java\\tools\\ManifoldCF\\example\\conf\\hbase-default.xml");
> =A0 =A0FileInputStream is =3D new FileInputStream(f);
> =A0 =A0config.addResource(is);
>
>
> Default code (referred in many guides for beginners) does not work:
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0config =3D HBaseConfiguration.create();
>
>
> It will call implicitly this method of HBase:
> =A0/**
> =A0 * Add a configuration resource.
> =A0 *
> =A0 * The properties of this resource will override properties of previou=
sly
> =A0 * added resources, unless they were marked <a href=3D"#Final">final</=
a>.
> =A0 *
> =A0 * @param name resource to be added, the classpath is examined for a f=
ile
> =A0 * =A0 =A0 =A0 =A0 =A0 =A0 with that name.
> =A0 */
> =A0public void addResource(String name) {
> =A0 =A0addResourceObject(name);
> =A0}
>
>
>
> And... somehow HBase run under ManifoldCF/connector-lib can't find this
> resource (in a classpath)... I tried all imaginable methods...
>
>
>
>
>
>
>
> -----Original Message-----
> From: Fuad Efendi [mailto:fuad@efendi.ca]
> Sent: April-04-11 12:52 PM
> To: connectors-user@incubator.apache.org
> Subject: RE: Strange Classloader Problem
>
> Hi Karl,
>
> I have hbase.jar file in connector-lib folder, and it seems it is loaded
> except hbase-default.xml file inside this hbase.jar file;
>
> 1. java -jar start.jar
> Caused by: java.lang.RuntimeException: hbase-default.xml file seems to be
> for and old version of HBase (null), this version is 0.90.1
> =A0 =A0 =A0 =A0at
> org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConf=
igu
> ration.java:66)
> =A0 =A0 =A0 =A0at
> org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfigu=
rat
> ion.java:76)
>
> 2. Explicit "test' command such as this one works just fine (so that noth=
ing
> wrong with hbase.jar/hbase-default.xml):
> java -cp
> mcf-hbaseoutput-connector.jar;junit-4.7.jar;hbase-0.90.1.jar;hadoop-core-=
0.2
> 0-append-r1056497.jar;commons-logging.jar;zookeeper-3.3.2.jar;log4j-1.2.j=
ar;
> mcf-hbaseoutput-connector.jar
> org.apache.manifoldcf.agents.output.hbaseconnector.HBaseTest
>
>
> Putting hbase.jar into "lib" doesn't help; it should be inside
> "connector-lib"; and ManifoldCF doesn't see this hbase-default.xml file..=
.
>
>
>
>
>
> -----Original Message-----
> From: Karl Wright [mailto:daddywri@gmail.com]
> Sent: April-04-11 12:36 PM
> To: connectors-user@incubator.apache.org
> Subject: Re: Strange Classloader Problem
>
> The ManifoldCF class loader is used for loading classes, not for loading =
XML
> files. =A0The only XML files ManifoldCF uses are its properties.xml file =
and
> its connectors.xml. =A0There should be no reason to use a special class l=
oader
> to get these files, since they are needed to bootstrap the whole system.
>
> Where are you trying to load XML files, and for what purpose? =A0The plac=
e you
> access a file is where you would need to add support for loading it via t=
he
> class loader.
>
> Karl
>
> On Mon, Apr 4, 2011 at 12:30 PM, Fuad Efendi <fuad@efendi.ca> wrote:
>> Hi,
>>
>>
>>
>> I am developing output connector to HBase.
>>
>>
>>
>> After many hours it seems to me ManifoldCF class loader doesn't see
>> any config files (*.XML) inside connector-lib/*.jar, and I can't even
>> manage it via explicit -classpath variable. is that true?
>>
>> Thanks,
>>
>> Fuad
>>
>>
>
>