Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A0D1210940 for ; Fri, 3 Jan 2014 07:53:49 +0000 (UTC) Received: (qmail 84858 invoked by uid 500); 3 Jan 2014 07:53:16 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 84710 invoked by uid 500); 3 Jan 2014 07:53:03 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 84691 invoked by uid 99); 3 Jan 2014 07:53:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Jan 2014 07:53:01 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jiangbinglover@gmail.com designates 209.85.214.177 as permitted sender) Received: from [209.85.214.177] (HELO mail-ob0-f177.google.com) (209.85.214.177) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Jan 2014 07:52:58 +0000 Received: by mail-ob0-f177.google.com with SMTP id vb8so15279755obc.36 for ; Thu, 02 Jan 2014 23:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0TZ5rvIGRfpauwhDt9wsGerJnLRwt+5KucDIB71vDlY=; b=mmlsYQZb5kyRuLFYlG+AlLfzzT+KJv9FRxlSC7Ee77ey5SJp71VF3x3gU0k9B5G3g+ A0QlEXRbPuj4PAtM6m+mYJiGxdU37C7N18v/7ZdUVVSXVaFT9A1SXR9+6r96N1q4NQdw 5emEP8jc+LshMl/D4IcWo5KPfw3s6SpBhvvdM++yBVSr3unAqeAI6I6XfBDdC01obquw RDq9pGQlaG58AqzQhsqrsxWsfSPCLvP5sF0wycu9tvjbv/O60FHt4V9DBCEC7YSYxkgl TOjWiuyQVcZuGJF1HI4JW7Fm2ktdpq7BwAg5JKUXop3bBICISjv6DExFowUK1cUVuJlb bdGQ== MIME-Version: 1.0 X-Received: by 10.182.42.105 with SMTP id n9mr58368442obl.33.1388735556805; Thu, 02 Jan 2014 23:52:36 -0800 (PST) Received: by 10.76.27.41 with HTTP; Thu, 2 Jan 2014 23:52:36 -0800 (PST) In-Reply-To: References: Date: Fri, 3 Jan 2014 15:52:36 +0800 Message-ID: Subject: Re: issue about running hive MR job in hadoop From: Bing Jiang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=14dae939981f8cea5504ef0c31aa X-Virus-Checked: Checked by ClamAV on apache.org --14dae939981f8cea5504ef0c31aa Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Could you check your yarn-local directory authority? From the diagnosis, error occurs at mkdir in local directory. I guess something wrong with local direcotry which is set as yarn local dir. 2014/1/3 ch huang > hi, i submit a MR job through hive ,but when it run stage-2 ,it failed wh= y? > > it seems permission problem ,but i do not know which dir cause the probl= em > > Application application_1388730279827_0035 failed 1 times due to AM > Container for appattempt_1388730279827_0035_000001 exited with exitCode: > -1000 due to: EPERM: Operation not permitted at > org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at > org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.= java:581) > at > org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:38= 8) > at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) a= t > org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java= :150) > at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at > org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at > org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at > org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:= 2325) > at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Cont= ainerLocalizer.initDirs(ContainerLocalizer.java:385) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Cont= ainerLocalizer.runLocalization(ContainerLocalizer.java:130) > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startL= ocalizer(DefaultContainerExecutor.java:103) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Reso= urceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.jav= a:861) > .Failing this attempt.. Failing the application. > --=20 Bing Jiang Tel=A3=BA(86)134-2619-1361 weibo: http://weibo.com/jiangbinglover BLOG: www.binospace.com BLOG: http://blog.sina.com.cn/jiangbinglover Focus on distributed computing, HDFS/HBase --14dae939981f8cea5504ef0c31aa Content-Type: text/html; charset=GB2312 Content-Transfer-Encoding: quoted-printable
Could you check your yarn-local directory authority? = >From the diagnosis, error occurs at mkdir in local directory.
I gu= ess something wrong with  local direcotry which is set as yarn local d= ir.



201= 4/1/3 ch huang <justlooks@gmail.com>
hi, i submit a MR job through hiv= e ,but when it run stage-2 ,it failed why?
 
 it seems permission = problem ,but i do not know which dir cause the problem
 
Application application_138873027= 9827_0035 failed 1 times due to AM Container for appattempt_1388730279827_0= 035_000001 exited with exitCode: -1000 due to: EPERM: Operation not permitt= ed at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at org.ap= ache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581= ) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java= :388) at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:104= 1) at org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.= java:150) at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at org.= apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at org.apache.had= oop.fs.FileContext$4.next(FileContext.java:695) at org.apache.hadoop.fs.Fil= eContext$FSLinkResolver.resolve(FileContext.java:2325) at org.apache.hadoop= .fs.FileContext.mkdir(FileContext.java:695) at org.apache.hadoop.yarn.serve= r.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(Contai= nerLocalizer.java:385) at org.apache.hadoop.yarn.server.nodemanager.contain= ermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.j= ava:130) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecu= tor.startLocalizer(DefaultContainerExecutor.java:103) at org.apache.hadoop.= yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationServ= ice$LocalizerRunner.run(ResourceLocalizationService.java:861) .Failing this= attempt.. Failing the application.



--
Bing J= iang
Tel=A3=BA(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG= : www.binospace.com<= /a>
BLOG: = http://blog.sina.com.cn/jiangbinglover 
Focus on distributed c= omputing, HDFS/HBase
--14dae939981f8cea5504ef0c31aa--