hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Lewis <lordjoe2...@gmail.com>
Subject Re: Execute hadoop job remotely and programmatically
Date Mon, 09 Dec 2013 22:00:40 GMT
out them in  a lib directory in the jar you pass to Hadoop and they will be
found


On Mon, Dec 9, 2013 at 12:58 PM, Yexi Jiang <yexijiang@gmail.com> wrote:

> Hi, All,
>
> I am working on a project that requires to execute a hadoop job remotely
> and the job requires some third-part libraries (jar files).
>
> Based on my understanding, I tried:
>
> 1. Copy these jar files to hdfs.
> 2. Copy them into the distributed cache using
> DistributedCache.addFileToClassPath so that hadoop can spread these jar
> files to each of the slave nodes.
>
> However, my program still throws ClassNotFoundException. Indicating that
> some of the classes cannot be found when the job is running.
>
> So I'm wondering:
> 1. What is the correct way to run a job remotely and programmatically
> while the job requires some third-party jar files.
> 2. I found DistributedCache is deprecated (I'm using hadoop 1.2.0), what
> is the alternative class?
>
> Regards,
> Yexi
>



-- 
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com

Mime
View raw message