Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of lordjoe2000@gmail.com
 designates 209.85.214.47 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAN_uYPji=oF3JAzg0mKeWVueAEBj3oNXUw0W-0gJ0aPwQstGVw@mail.gmail.com>
References: 
 <CAN_uYPji=oF3JAzg0mKeWVueAEBj3oNXUw0W-0gJ0aPwQstGVw@mail.gmail.com>
Date: Mon, 9 Dec 2013 14:00:40 -0800
Message-ID: 
 <CALEj8eNRS9=fkySQnDactz1Qkwp2-bWC9zxtAdfOCxKbNCd-ag@mail.gmail.com>
Subject: Re: Execute hadoop job remotely and programmatically
From: Steve Lewis <lordjoe2000@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=bcaec52bf1ef7392f804ed2120a3

--bcaec52bf1ef7392f804ed2120a3
Content-Type: text/plain; charset=ISO-8859-1

out them in  a lib directory in the jar you pass to Hadoop and they will be
found


On Mon, Dec 9, 2013 at 12:58 PM, Yexi Jiang <yexijiang@gmail.com> wrote:

> Hi, All,
>
> I am working on a project that requires to execute a hadoop job remotely
> and the job requires some third-part libraries (jar files).
>
> Based on my understanding, I tried:
>
> 1. Copy these jar files to hdfs.
> 2. Copy them into the distributed cache using
> DistributedCache.addFileToClassPath so that hadoop can spread these jar
> files to each of the slave nodes.
>
> However, my program still throws ClassNotFoundException. Indicating that
> some of the classes cannot be found when the job is running.
>
> So I'm wondering:
> 1. What is the correct way to run a job remotely and programmatically
> while the job requires some third-party jar files.
> 2. I found DistributedCache is deprecated (I'm using hadoop 1.2.0), what
> is the alternative class?
>
> Regards,
> Yexi
>


-- 
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com

--bcaec52bf1ef7392f804ed2120a3
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">out them in =A0a lib directory in the jar you pass to Hado=
op and they will be found</div><div class=3D"gmail_extra"><br><br><div clas=
s=3D"gmail_quote">On Mon, Dec 9, 2013 at 12:58 PM, Yexi Jiang <span dir=3D"=
ltr">&lt;<a href=3D"mailto:yexijiang@gmail.com" target=3D"_blank">yexijiang=
@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi, All,
<div><br></div><div>I am working on a project that requires to execute a ha=
doop job remotely and the job requires some third-part libraries (jar files=
).</div><div><br></div><div>Based on my understanding, I tried:</div><div>

<br></div><div>1. Copy these jar files to hdfs.</div><div>2. Copy them into=
 the distributed cache using DistributedCache.addFileToClassPath so that ha=
doop can spread these jar files to each of the slave nodes.=A0</div><div>

<br></div><div>However, my program still throws ClassNotFoundException. Ind=
icating that some of the classes cannot be found when the job is running.</=
div><div><br></div><div>So I&#39;m wondering:</div><div>1. What is the corr=
ect way to run a job remotely and programmatically while the job requires s=
ome third-party jar files.</div>

<div>2. I found DistributedCache is deprecated (I&#39;m using hadoop 1.2.0)=
, what is the alternative class?</div><div><br></div><div>Regards,</div><di=
v>Yexi</div></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>Steven M. Le=
wis PhD<div>4221 105th Ave NE</div><div>Kirkland, WA 98033</div><div>206-38=
4-1340 (cell)<br>Skype lordjoe_com<br><br></div>
</div>

--bcaec52bf1ef7392f804ed2120a3--