hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stan Rosenberg <stan.rosenb...@gmail.com>
Subject Re: task jvm bootstrapping via distributed cache
Date Wed, 01 Aug 2012 15:12:12 GMT
On Tue, Jul 31, 2012 at 7:26 PM, Michael Segel
<michael_segel@hotmail.com> wrote:
> Hi Stan,
> If I understood your question... you want to ship a jar to the nodes where the task will
run prior to the start of the task?
> Not sure what it is you're trying to do...
> Your example isn't  really clear.

Correct.  I want to ship a jar to the task, but I need to know its
absolute path before the task jvm is launched.
As an example, -javaagent JVM option expects a jar path.

> See: http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/filecache/DistributedCache.html
> When you pull stuff out of the cache you get the path to the jar.
> Or you should be able to get it.

It would be too late at that point; the task tracker controls the
launching of the JVM.  The path of the shipped jar need to be
available before the task is launched.

> Can you give a better example, there may be a different way to handle this...
Does the example above make sense?

View raw message