Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.214.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHg+sbOi-JTPaf5HGFUHuKUc2Z1eBZ3=etJhL36t+2np8T_2Fg@mail.gmail.com>
References: 
 <CAHg+sbOWh+tGUz0nX=F_UGCfPU8s8H==xohhXsSfaLOdz7wteg@mail.gmail.com>
 <CAOcnVr0sc2gzuDJikDH79msApqV_Rwunxctb9Jyx7s5yenLuQA@mail.gmail.com>
 <CAHg+sbNnhia+5x+FbbpWkhGaNfjnkZGBfU9EJ-GmsuiejZWTpA@mail.gmail.com>
 <CAOcnVr14q7L-MPXBjjeLLJqgQOPNr8QEx6z=pjMkgijAn3bK4w@mail.gmail.com>
 <CAHg+sbOi-JTPaf5HGFUHuKUc2Z1eBZ3=etJhL36t+2np8T_2Fg@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Tue, 6 Aug 2013 15:36:43 +0530
Message-ID: 
 <CAOcnVr2Q9kw=rTyTPYDzSBGSqy_mxynw6PJTW1CgDvP7ihWOkQ@mail.gmail.com>
Subject: Re: setLocalResources() on ContainerLaunchContext
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

To be honest, I've never tried loading a HDFS file onto the
LocalResource this way. I usually just pass a local file and that
works just fine. There may be something in the URI transformation
possibly breaking a HDFS source, but try passing a local file - does
that fail too? The Shell example uses a local file.

On Tue, Aug 6, 2013 at 10:54 AM, Krishna Kishore Bonagiri
<write2kishore@gmail.com> wrote:
> Hi Harsh,
>
>   Please see if this is useful, I got a stack trace after the error has
> occurred....
>
> 2013-08-06 00:55:30,559 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: CWD set
> to /tmp/nm-local-dir/usercache/dsadm/appcache/application_1375716148174_0004
> =
> file:/tmp/nm-local-dir/usercache/dsadm/appcache/application_1375716148174_0004
> 2013-08-06 00:55:31,017 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:dsadm (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not
> exist: hdfs://isredeng/kishore/kk.ksh
> 2013-08-06 00:55:31,029 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> DEBUG: FAILED { hdfs://isredeng/kishore/kk.ksh, 0, FILE, null }, File does
> not exist: hdfs://isredeng/kishore/kk.ksh
> 2013-08-06 00:55:31,031 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
> Resource hdfs://isredeng/kishore/kk.ksh transitioned from DOWNLOADING to
> FAILED
> 2013-08-06 00:55:31,034 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> Container container_1375716148174_0004_01_000002 transitioned from
> LOCALIZING to LOCALIZATION_FAILED
> 2013-08-06 00:55:31,035 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl:
> Container container_1375716148174_0004_01_000002 sent RELEASE event on a
> resource request { hdfs://isredeng/kishore/kk.ksh, 0, FILE, null } not
> present in cache.
> 2013-08-06 00:55:31,036 WARN org.apache.hadoop.ipc.Client: interrupted
> waiting to send rpc request to server
> java.lang.InterruptedException
>         at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1290)
>         at
> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:229)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:94)
>         at
> org.apache.hadoop.ipc.Client$Connection.sendRpcRequest(Client.java:930)
>         at org.apache.hadoop.ipc.Client.call(Client.java:1285)
>         at org.apache.hadoop.ipc.Client.call(Client.java:1264)
>         at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
>         at $Proxy22.heartbeat(Unknown Source)
>         at
> org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.client.LocalizationProtocolPBClientImpl.heartbeat(LocalizationProtocolPBClientImpl.java:62)
>         at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.localizeFiles(ContainerLocalizer.java:249)
>         at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:163)
>         at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:106)
>         at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:979)
>
>
>
> And here is my code snippet:
>
>       ContainerLaunchContext ctx =
> Records.newRecord(ContainerLaunchContext.class);
>
>       ctx.setEnvironment(oshEnv);
>
>       // Set the local resources
>       Map<String, LocalResource> localResources = new HashMap<String,
> LocalResource>();
>
>       LocalResource shellRsrc = Records.newRecord(LocalResource.class);
>       shellRsrc.setType(LocalResourceType.FILE);
>       shellRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
>       String shellScriptPath = "hdfs://isredeng//kishore/kk.ksh";
>       try {
>         shellRsrc.setResource(ConverterUtils.getYarnUrlFromURI(new
> URI(shellScriptPath)));
>       } catch (URISyntaxException e) {
>         LOG.error("Error when trying to use shell script path specified"
>             + " in env, path=" + shellScriptPath);
>         e.printStackTrace();
>       }
>
>       shellRsrc.setTimestamp(0/*shellScriptPathTimestamp*/);
>       shellRsrc.setSize(0/*shellScriptPathLen*/);
>       String ExecShellStringPath = "ExecShellScript.sh";
>       localResources.put(ExecShellStringPath, shellRsrc);
>
>       ctx.setLocalResources(localResources);
>
>
> Please let me know if you need anything else.
>
> Thanks,
> Kishore
>
>
>
> On Tue, Aug 6, 2013 at 12:05 AM, Harsh J <harsh@cloudera.com> wrote:
>>
>> The detail is insufficient to answer why. You should also have gotten
>> a trace after it, can you post that? If possible, also the relevant
>> snippets of code.
>>
>> On Mon, Aug 5, 2013 at 6:36 PM, Krishna Kishore Bonagiri
>> <write2kishore@gmail.com> wrote:
>> > Hi Harsh,
>> >  Thanks for the quick and detailed reply, it really helps. I am trying
>> > to
>> > use it and getting this error in node manager's log:
>> >
>> > 2013-08-05 08:57:28,867 ERROR
>> > org.apache.hadoop.security.UserGroupInformation:
>> > PriviledgedActionException
>> > as:dsadm (auth:SIMPLE) cause:java.io.FileNotFoundException: File does
>> > not
>> > exist: hdfs://isredeng/kishore/kk.ksh
>> >
>> >
>> > This file is there on the machine with name "isredeng", I could do ls
>> > for
>> > that file as below:
>> >
>> > -bash-4.1$ hadoop fs -ls kishore/kk.ksh
>> > 13/08/05 09:01:03 WARN util.NativeCodeLoader: Unable to load
>> > native-hadoop
>> > library for your platform... using builtin-java classes where applicable
>> > Found 1 items
>> > -rw-r--r--   3 dsadm supergroup       1046 2013-08-05 08:48
>> > kishore/kk.ksh
>> >
>> > Note: I am using a single node cluster
>> >
>> > Thanks,
>> > Kishore
>> >
>> >
>> >
>> >
>> > On Mon, Aug 5, 2013 at 3:00 PM, Harsh J <harsh@cloudera.com> wrote:
>> >>
>> >> The string for each LocalResource in the map can be anything that
>> >> serves as a common identifier name for your application. At execution
>> >> time, the passed resource filename will be aliased to the name you've
>> >> mapped it to, so that the application code need not track special
>> >> names. The behavior is very similar to how you can, in MR, define a
>> >> symlink name for a DistributedCache entry (e.g. foo.jar#bar.jar).
>> >>
>> >> For an example, checkout the DistributedShell app sources.
>> >>
>> >> Over [1], you can see we take a user provided file path to a shell
>> >> script. This can be named anything as it is user-supplied.
>> >> Onto [2], we define this as a local resource [2.1] and embed it with a
>> >> different name (the string you ask about) [2.2], as defined at [3] as
>> >> an application reference-able constant.
>> >> Note that in [4], we add to the Container arguments the aliased name
>> >> we mapped it to (i.e. [3]) and not the original filename we received
>> >> from the user. The resource is placed on the container with this name
>> >> instead, so thats what we choose to execute.
>> >>
>> >> [1] -
>> >>
>> >> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L390
>> >>
>> >> [2] - [2.1]
>> >>
>> >> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L764
>> >> and [2.2]
>> >>
>> >> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L780
>> >>
>> >> [3] -
>> >>
>> >> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L205
>> >>
>> >> [4] -
>> >>
>> >> https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java#L791
>> >>
>> >> On Mon, Aug 5, 2013 at 2:44 PM, Krishna Kishore Bonagiri
>> >> <write2kishore@gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> >   Can someone please tell me what is the use of calling
>> >> > setLocalResources()
>> >> > on ContainerLaunchContext?
>> >> >
>> >> >   And, also an example of how to use this will help...
>> >> >
>> >> >  I couldn't guess what is the String in the map that is passed to
>> >> > setLocalResources() like below:
>> >> >
>> >> >       // Set the local resources
>> >> >       Map<String, LocalResource> localResources = new HashMap<String,
>> >> > LocalResource>();
>> >> >
>> >> > Thanks,
>> >> > Kishore
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Harsh J
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>


-- 
Harsh J