Return-Path: Delivered-To: apmail-hadoop-pig-dev-archive@www.apache.org Received: (qmail 42336 invoked from network); 4 Dec 2008 04:28:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 4 Dec 2008 04:28:40 -0000 Received: (qmail 32587 invoked by uid 500); 4 Dec 2008 04:28:49 -0000 Delivered-To: apmail-hadoop-pig-dev-archive@hadoop.apache.org Received: (qmail 32565 invoked by uid 500); 4 Dec 2008 04:28:49 -0000 Mailing-List: contact pig-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: pig-dev@hadoop.apache.org Delivered-To: mailing list pig-dev@hadoop.apache.org Received: (qmail 31937 invoked by uid 99); 4 Dec 2008 04:28:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Dec 2008 20:28:47 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Dec 2008 04:27:26 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 382A7234C2A1 for ; Wed, 3 Dec 2008 20:27:44 -0800 (PST) Message-ID: <642826835.1228364864215.JavaMail.jira@brutus> Date: Wed, 3 Dec 2008 20:27:44 -0800 (PST) From: "Ian Holsman (JIRA)" To: pig-dev@hadoop.apache.org Subject: [jira] Created: (PIG-555) ORDER by requires write access to /tmp MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org ORDER by requires write access to /tmp -------------------------------------- Key: PIG-555 URL: https://issues.apache.org/jira/browse/PIG-555 Project: Pig Issue Type: Bug Components: grunt Affects Versions: types_branch Reporter: Ian Holsman Priority: Minor Fix For: types_branch when doing an order by in pig, it relies on you having write access to /tmp/ in hdfs. (and i think for it to be present in the first place). this comes down to FileLocalizer creating the default relativeRoot = pigContext.getDfs().asContainer("/tmp/temp" + r.nextInt()) i'm not sure if this is due to someone on our side deleting the /tmp dir accidently, so apolagies for the spam if it is. 2008-12-03 22:36:41,716 [main] ERROR org.apache.pig.tools.grunt.Grunt - You don't have permission to perform the operation. Error from the server: Unable to store for alias: 7 [org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x] java.io.IOException: Unable to store for alias: 7 [org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x] at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:255) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:647) at org.apache.pig.PigServer.execute(PigServer.java:638) at org.apache.pig.PigServer.registerQuery(PigServer.java:278) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:439) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:249) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64) at org.apache.pig.Main.main(Main.java:242) Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x ... 9 more Caused by: org.apache.pig.impl.plan.VisitorException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitSort(MRCompiler.java:858) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSort.visit(POSort.java:304) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:266) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:251) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:193) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:134) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:63) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:245) ... 8 more Caused by: org.apache.hadoop.fs.permission.AccessControlException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:52) at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:704) at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:236) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1116) at org.apache.pig.backend.hadoop.datastorage.HDirectory.create(HDirectory.java:64) at org.apache.pig.backend.hadoop.datastorage.HPath.create(HPath.java:155) at org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:391) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.getTempFileSpec(MRCompiler.java:479) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitSort(MRCompiler.java:846) ... 15 more Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException: Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:175) at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:156) at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:104) at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4228) at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4198) at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1595) at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1564) at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:450) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888) at org.apache.hadoop.ipc.Client.call(Client.java:715) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source) at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:702) ... 22 more shell returned 1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.