hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rahul p <rahulpoolancha...@gmail.com>
Subject Re: task jvm bootstrapping via distributed cache
Date Sun, 05 Aug 2012 05:35:04 GMT
Hi Arun,
I am new to hadoop n big data.
Can you help me start working on basics.my experience is into ETL and BI
DWH.

Rahul
 On Aug 4, 2012 12:33 AM, "Stan Rosenberg" <stan.rosenberg@gmail.com> wrote:

> Arun,
>
> I don't believe the symlink is of help.  The symlink is created in the
> task's current working directory (cwd), but I don't know what cwd is
> when I launch with 'hadoop jar ...'.
>
> Thanks,
>
> stan
>
> On Fri, Aug 3, 2012 at 2:39 AM, Arun C Murthy <acm@hortonworks.com> wrote:
> > Stan,
> >
> >  You can ask TT to create a symlink to your jar shipped via DistCache:
> >
> >
> http://hadoop.apache.org/common/docs/r1.0.3/mapred_tutorial.html#DistributedCache
> >
> >  That should give you what you want.
> >
> > hth,
> > Arun
> >
> > On Jul 30, 2012, at 3:23 PM, Stan Rosenberg wrote:
> >
> > Hi,
> >
> > I am seeking a way to leverage hadoop's distributed cache in order to
> > ship jars that are required to bootstrap a task's jvm, i.e., before a
> > map/reduce task is launched.
> > As a concrete example, let's say that I need to launch with
> > '-javaagent:/path/profiler.jar'.  In theory, the task tracker is
> > responsible for downloading cached files onto its local filesystem.
> > However, the absolute path to a given cached file is not known a
> > priori; however, we need the path in order to configure '-javaagent'.
> >
> > Is this currently possible with the distributed cache? If not, is the
> > use case appealing enough to open a jira ticket?
> >
> > Thanks,
> >
> > stan
> >
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>

Mime
View raw message