hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabor Makrai <makrai.l...@gmail.com>
Subject Re: Large startup time in remote MapReduce job
Date Tue, 21 Jun 2011 22:15:56 GMT
Fortunately, DistributedCache solved my problem! I put a jar file to
HDFS. which contains the necessary classes for the job and I used this:
*DistributedCache.addFileToClassPath(new Path("/myjar/myjar.jar"), conf);*

Thanks for fast answer!
And sorry for my mistake (about the wrong list), that was my first one!
Thank again!

Best,
Gabor

On Tue, Jun 21, 2011 at 11:02 PM, Harsh J <harsh@cloudera.com> wrote:

> Allen,
>
> On Wed, Jun 22, 2011 at 2:28 AM, Allen Wittenauer <aw@apache.org> wrote:
> >
> > On Jun 21, 2011, at 1:31 PM, Harsh J wrote:
> >
> >> Gabor,
> >>
> >> If your jar does not contain code changes that need to get transmitted
> >> every time, you can consider placing them on the JT/TT classpaths
> >
> >        ... which means you get to bounce your system every time you
> change code.
>
> Its ugly, but if the jar filename remains the same there shouldn't
> need to be any bouncing. Doable if there's no activity at replacement
> point of time?
>
> P.s. Gabor: Moving the discussion to mapreduce-user@hadoop.apache.org
> Please use the general@ list only for general project-wide discussions
> on Hadoop.
>
> --
> Harsh J
>

Mime
View raw message