Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 053179121 for ; Sun, 5 Aug 2012 05:35:37 +0000 (UTC) Received: (qmail 27027 invoked by uid 500); 5 Aug 2012 05:35:35 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 26667 invoked by uid 500); 5 Aug 2012 05:35:33 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 26634 invoked by uid 99); 5 Aug 2012 05:35:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Aug 2012 05:35:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rahulpoolanchalil@gmail.com designates 209.85.217.176 as permitted sender) Received: from [209.85.217.176] (HELO mail-lb0-f176.google.com) (209.85.217.176) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Aug 2012 05:35:25 +0000 Received: by lboj14 with SMTP id j14so2603642lbo.35 for ; Sat, 04 Aug 2012 22:35:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=iy0nxACTRjKjleAN4RtgbzycHn4e229R9CYWVtk96Xs=; b=NMBvQHvSPWmVB0erz2szNvm49H/6b2DaTkqcDwfSIN2ravyErl7q0rJ7z9/62jETc7 v6OYPBON+FfcAFutt92xVZzjDqQ7qazCjC4z0u9Riq3RHPhsMXrnZXehdZdOJLF65zr4 UViDA1FxbJOBd6VhjK+k0yW/fKoYyiPb4+ByxG7EBlejvPZtjhJN5MZt5y8Kes99/4Ut f6U+aHcoMX6njyvEMAmJQreLAxno4AFu0zWSoVnQ/HxoOiyVuQWvW+EW6ssckcOya+97 RI/3T8DxDOXi5hqACHmgvF4DQwE146us7HWHUuAdwyyy3jOWOKu8yifkafB33/3G7yxz F2Mw== MIME-Version: 1.0 Received: by 10.152.146.101 with SMTP id tb5mr6656726lab.0.1344144905000; Sat, 04 Aug 2012 22:35:05 -0700 (PDT) Received: by 10.112.8.39 with HTTP; Sat, 4 Aug 2012 22:35:04 -0700 (PDT) Received: by 10.112.8.39 with HTTP; Sat, 4 Aug 2012 22:35:04 -0700 (PDT) In-Reply-To: References: <63DFEAC0-93EA-4ED9-9EE3-CCB791D56A0D@hortonworks.com> Date: Sun, 5 Aug 2012 13:35:04 +0800 Message-ID: Subject: Re: task jvm bootstrapping via distributed cache From: rahul p To: mapreduce-user@hadoop.apache.org, stan.rosenberg@gmail.com Content-Type: multipart/alternative; boundary=e89a8f2348b396d59404c67e1f4e --e89a8f2348b396d59404c67e1f4e Content-Type: text/plain; charset=ISO-8859-1 Hi Arun, I am new to hadoop n big data. Can you help me start working on basics.my experience is into ETL and BI DWH. Rahul On Aug 4, 2012 12:33 AM, "Stan Rosenberg" wrote: > Arun, > > I don't believe the symlink is of help. The symlink is created in the > task's current working directory (cwd), but I don't know what cwd is > when I launch with 'hadoop jar ...'. > > Thanks, > > stan > > On Fri, Aug 3, 2012 at 2:39 AM, Arun C Murthy wrote: > > Stan, > > > > You can ask TT to create a symlink to your jar shipped via DistCache: > > > > > http://hadoop.apache.org/common/docs/r1.0.3/mapred_tutorial.html#DistributedCache > > > > That should give you what you want. > > > > hth, > > Arun > > > > On Jul 30, 2012, at 3:23 PM, Stan Rosenberg wrote: > > > > Hi, > > > > I am seeking a way to leverage hadoop's distributed cache in order to > > ship jars that are required to bootstrap a task's jvm, i.e., before a > > map/reduce task is launched. > > As a concrete example, let's say that I need to launch with > > '-javaagent:/path/profiler.jar'. In theory, the task tracker is > > responsible for downloading cached files onto its local filesystem. > > However, the absolute path to a given cached file is not known a > > priori; however, we need the path in order to configure '-javaagent'. > > > > Is this currently possible with the distributed cache? If not, is the > > use case appealing enough to open a jira ticket? > > > > Thanks, > > > > stan > > > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/ > > > > > --e89a8f2348b396d59404c67e1f4e Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Hi Arun,
I am new to hadoop n big data.
Can you help me start working on basics.my experience is into ETL and BI DW= H.

Rahul

On Aug 4, 2012 12:33 AM, "Stan Rosenberg&qu= ot; <stan.rosenberg@gmail.co= m> wrote:
Arun,

I don't believe the symlink is of help. =A0The symlink is created in th= e
task's current working directory (cwd), but I don't know what cwd i= s
when I launch with 'hadoop jar ...'.

Thanks,

stan

On Fri, Aug 3, 2012 at 2:39 AM, Arun C Murthy <acm@hortonworks.com> wrote:
> Stan,
>
> =A0You can ask TT to create a symlink to your jar shipped via DistCach= e:
>
> http://hadoop.apache.org/common/d= ocs/r1.0.3/mapred_tutorial.html#DistributedCache
>
> =A0That should give you what you want.
>
> hth,
> Arun
>
> On Jul 30, 2012, at 3:23 PM, Stan Rosenberg wrote:
>
> Hi,
>
> I am seeking a way to leverage hadoop's distributed cache in order= to
> ship jars that are required to bootstrap a task's jvm, i.e., befor= e a
> map/reduce task is launched.
> As a concrete example, let's say that I need to launch with
> '-javaagent:/path/profiler.jar'. =A0In theory, the task tracke= r is
> responsible for downloading cached files onto its local filesystem. > However, the absolute path to a given cached file is not known a
> priori; however, we need the path in order to configure '-javaagen= t'.
>
> Is this currently possible with the distributed cache? If not, is the<= br> > use case appealing enough to open a jira ticket?
>
> Thanks,
>
> stan
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonwor= ks.com/
>
>
--e89a8f2348b396d59404c67e1f4e--