hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Correct way to unzip locally an archive in Yarn
Date Tue, 19 Feb 2013 18:33:47 GMT
Yes if you can trace this down I would be very interested.  We are running 0.23.6 without any
issues, but that does not mean that there is not some bug in the code that is causing this
to happen in your situation.


From: Sebastiano Vigna <vigna@di.unimi.it<mailto:vigna@di.unimi.it>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Saturday, February 16, 2013 8:39 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Correct way to unzip locally an archive in Yarn

I will as soon as I can understand what happens on the cluster (no access from home). DistributedCache.getLocalCacheFiles()
returns in both cases a local name for the zip file uploaded with -files, but locally my unzip
code works, on the cluster it throws a FileNotFoundException.

On 16 February 2013 15:22, Arun C Murthy <acm@hortonworks.com<mailto:acm@hortonworks.com>>
This could be a bug, mind opening a jira? Thanks.

On Feb 16, 2013, at 2:34 AM, Sebastiano Vigna wrote:

On 15 February 2013 16:57, Robert Evans <evans@yahoo-inc.com<mailto:evans@yahoo-inc.com>>
Are you trying to run a Map/Reduce job or are you writing a new YARN
application?  If it is a MR job, then it should work mostly the same as
before (on 1.x). If you are writing a new YARN application then there is a
separate Map in the ContainerLaunchContext that you need to fill in.

It's a MapReduce job (0.23.6). After two days of useless trials, I'm uploading the zip with
-files and I wrote a stub to unzip it manually. I was positively unable to get the archive
unzipped *to a local directory* in any way.

Unfortunately it works in local but not on the cluster. I have still to discover why. :(


Arun C. Murthy
Hortonworks Inc.

View raw message