Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14AB7DE00 for ; Tue, 19 Feb 2013 18:34:43 +0000 (UTC) Received: (qmail 92602 invoked by uid 500); 19 Feb 2013 18:34:38 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 92499 invoked by uid 500); 19 Feb 2013 18:34:37 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 92492 invoked by uid 99); 19 Feb 2013 18:34:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Feb 2013 18:34:37 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,NO_RDNS_DOTCOM_HELO,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: 98.139.253.104 is neither permitted nor denied by domain of evans@yahoo-inc.com) Received: from [98.139.253.104] (HELO mrout1-b.corp.bf1.yahoo.com) (98.139.253.104) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Feb 2013 18:34:30 +0000 Received: from sp1-ex07cas01.ds.corp.yahoo.com (sp1-ex07cas01.ds.corp.yahoo.com [216.252.116.137]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id r1JIXm7Z040174 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Tue, 19 Feb 2013 10:33:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1361298830; bh=LoNlMucZKnu93c60/1ezrq1bPsuEIoUXXqqKUj/wwpA=; h=From:To:Date:Subject:Message-ID:In-Reply-To:Content-Type: MIME-Version; b=So1nlPRaP2Je1e+9fzpCNjtZrSOAKXHEX2Y921O3EtJWYJI+RvAt5Ql9AIkwMIAWr Apu+Kn0RbnSGl5zv/cwj9WRVi3zwQ8KaM/36Ksx+g4e5KD5qwx4ZfjDHLNiUa3srQ2 VZ2RRxELjA+2fwcwyN0VBoAdwk/JDUnS4qEzJrsg= Received: from SP1-EX07VS02.ds.corp.yahoo.com ([216.252.116.135]) by sp1-ex07cas01.ds.corp.yahoo.com ([216.252.116.137]) with mapi; Tue, 19 Feb 2013 10:33:49 -0800 From: Robert Evans To: "user@hadoop.apache.org" Date: Tue, 19 Feb 2013 10:33:47 -0800 Subject: Re: Correct way to unzip locally an archive in Yarn Thread-Topic: Correct way to unzip locally an archive in Yarn Thread-Index: Ac4Oz6jpdAWze0G0RtaoPDtlkaaZLQ== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.1.130117 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CD49215D642Aevansyahooinccom_" MIME-Version: 1.0 X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 298830000 X-Virus-Checked: Checked by ClamAV on apache.org --_000_CD49215D642Aevansyahooinccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yes if you can trace this down I would be very interested. We are running = 0.23.6 without any issues, but that does not mean that there is not some bu= g in the code that is causing this to happen in your situation. --Bobby From: Sebastiano Vigna > Reply-To: "user@hadoop.apache.org" > Date: Saturday, February 16, 2013 8:39 AM To: "user@hadoop.apache.org" > Subject: Re: Correct way to unzip locally an archive in Yarn I will as soon as I can understand what happens on the cluster (no access f= rom home). DistributedCache.getLocalCacheFiles() returns in both cases a lo= cal name for the zip file uploaded with -files, but locally my unzip code w= orks, on the cluster it throws a FileNotFoundException. On 16 February 2013 15:22, Arun C Murthy > wrote: This could be a bug, mind opening a jira? Thanks. On Feb 16, 2013, at 2:34 AM, Sebastiano Vigna wrote: On 15 February 2013 16:57, Robert Evans > wrote: Are you trying to run a Map/Reduce job or are you writing a new YARN application? If it is a MR job, then it should work mostly the same as before (on 1.x). If you are writing a new YARN application then there is a separate Map in the ContainerLaunchContext that you need to fill in. It's a MapReduce job (0.23.6). After two days of useless trials, I'm upload= ing the zip with -files and I wrote a stub to unzip it manually. I was posi= tively unable to get the archive unzipped *to a local directory* in any way= . Unfortunately it works in local but not on the cluster. I have still to dis= cover why. :( Ciao, -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ --_000_CD49215D642Aevansyahooinccom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Yes if you can trace thi= s down I would be very interested.  We are running 0.23.6 without any = issues, but that does not mean that there is not some bug in the code that = is causing this to happen in your situation.

--Bob= by

From: = Sebastiano Vigna <vigna@di.unimi.it= >
Reply-To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Date: Saturday, February 16, 2013 8:39 AM
= To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: Re: Correct way to unzip locally an archive in Yarn

I will as soon as I can understand w= hat happens on the cluster (no access from home). DistributedCache.getLocalCacheFiles() returns in both cases= a local name for the zip file uploaded with -files, but locally my unzip c= ode works, on the cluster it throws a FileNotFoundException.


On 16 February = 2013 15:22, Arun C Murthy <acm@hortonworks.com> wrote:
=
This cou= ld be a bug, mind opening a jira? Thanks.

On Feb 16, 2013= , at 2:34 AM, Sebastiano Vigna wrote:

On 15 February 2013 16:57, Robert Evans <evans= @yahoo-inc.com> wrote:
Are you trying to run a Map/Reduce job or are you writing a new YARN
app= lication?  If it is a MR job, then it should work mostly the same asbefore (on 1.x). If you are writing a new YARN application then there is = a
separate Map in the ContainerLaunchContext that you need to fill in.
 
It's a MapReduce job (0.23.6). Afte= r two days of useless trials, I'm uploading the zip with -files and I wrote= a stub to unzip it manually. I was positively unable to get the archive un= zipped *to a local directory* in any way.

Unfortunately = it works in local but not on the cluster. I have still to discover why. :(<= div>
Ciao,




--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



--_000_CD49215D642Aevansyahooinccom_--