Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of amits@infolinks.com
 designates 207.126.144.135 as permitted sender)
MIME-Version: 1.0
Date: Wed, 17 Apr 2013 21:41:26 +0300
Message-ID: 
 <CAAMYKhruxozLpsy-TvgdNRmdt9Tp_zgy7GkqqVSGvhdTgR1rKA@mail.gmail.com>
Subject: Setting up a Hadoop client in OSGI bundle
From: Amit Sela <amits@infolinks.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae9340d0356dc8004da92d5db

--14dae9340d0356dc8004da92d5db
Content-Type: text/plain; charset=ISO-8859-1

Hi all,

I'm trying to setup an Hadoop client for job submissions (and more) as an
OSGI bundle.

I came over a lot of hardships but I'm kinda stuck now.

When I create a new Job for submission I setClassLoader() for the Job
Configuration  so that it would use the bundle's ClassLoader (felix), but
when I submit the job and it tries to load the splits it looks for
DefaultCodec and fails to find that because somewhere along the way,
JobClient creates a new JobContext(Configuration) but although using the
Configuration containing the bundle's ClassLoader, the jobContext
constructor doesn't copy the ClassLoader set in Configuration.
And I guess that could happen in many other places in the Hadoop frameworl.

does anyone have any experience with submitting mapreduce jobs from OSGI
bundle client ? Any tips for this problems (and others I've yet to
encounter) ?

BTW, running Hadoop 1.0.4 and HBase 0.94.2.

Thanks,

Amit.

--14dae9340d0356dc8004da92d5db
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi all,=A0<div><br></div><div style>I&#39;m trying to setu=
p an Hadoop client for job submissions (and more) as an OSGI bundle.</div><=
div style><br></div><div style>I came over a lot of hardships but I&#39;m k=
inda stuck now.</div>
<div style><br></div><div style>When I create a new Job for submission I se=
tClassLoader() for the Job Configuration =A0so that it would use the bundle=
&#39;s ClassLoader (felix), but when I submit the job and it tries to load =
the splits it looks for DefaultCodec and fails to find that because somewhe=
re along the way, JobClient creates a new JobContext(Configuration) but alt=
hough using the Configuration containing the bundle&#39;s ClassLoader, the =
jobContext constructor doesn&#39;t copy the ClassLoader set in Configuratio=
n.</div>
<div style>And I guess that could happen in many other places in the Hadoop=
 frameworl.</div><div style><br></div><div style>does anyone have any exper=
ience with submitting mapreduce jobs from OSGI bundle client ? Any tips for=
 this problems (and others I&#39;ve yet to encounter) ?</div>
<div style><br></div><div style>BTW, running Hadoop 1.0.4 and HBase 0.94.2.=
</div><div style><br></div><div style>Thanks,=A0</div><div style><br></div>=
<div style>Amit.</div></div>

--14dae9340d0356dc8004da92d5db--