pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Holsman (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-555) ORDER by requires write access to /tmp
Date Thu, 04 Dec 2008 04:27:44 GMT
ORDER by requires write access to /tmp
--------------------------------------

                 Key: PIG-555
                 URL: https://issues.apache.org/jira/browse/PIG-555
             Project: Pig
          Issue Type: Bug
          Components: grunt
    Affects Versions: types_branch
            Reporter: Ian Holsman
            Priority: Minor
             Fix For: types_branch


when doing an order by in pig, it relies on you having write access to /tmp/ in hdfs. (and
i think for it to be present in the first place).

this comes down to FileLocalizer creating the default 

    relativeRoot = pigContext.getDfs().asContainer("/tmp/temp" + r.nextInt())

i'm not sure if this is due to someone on our side deleting the /tmp dir accidently, so apolagies
for the spam if it is.


2008-12-03 22:36:41,716 [main] ERROR org.apache.pig.tools.grunt.Grunt - You don't have permission
to perform the operation. Error from the server: Unable to store for alias: 7 [org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x]
java.io.IOException: Unable to store for alias: 7 [org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x]
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:255)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:647)
        at org.apache.pig.PigServer.execute(PigServer.java:638)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:278)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:439)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:249)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
        at org.apache.pig.Main.main(Main.java:242)
Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
        ... 9 more
Caused by: org.apache.pig.impl.plan.VisitorException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitSort(MRCompiler.java:858)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSort.visit(POSort.java:304)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:266)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:251)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:193)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:134)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:63)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:245)
        ... 8 more
Caused by: org.apache.hadoop.fs.permission.AccessControlException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:52)
        at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:704)
        at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:236)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1116)
        at org.apache.pig.backend.hadoop.datastorage.HDirectory.create(HDirectory.java:64)
        at org.apache.pig.backend.hadoop.datastorage.HPath.create(HPath.java:155)
        at org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:391)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.getTempFileSpec(MRCompiler.java:479)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitSort(MRCompiler.java:846)
        ... 15 more
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.fs.permission.AccessControlException:
Permission denied: user=jholsma, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
        at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:175)
        at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:156)
        at org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:104)
        at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4228)
        at org.apache.hadoop.dfs.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4198)
        at org.apache.hadoop.dfs.FSNamesystem.mkdirsInternal(FSNamesystem.java:1595)
        at org.apache.hadoop.dfs.FSNamesystem.mkdirs(FSNamesystem.java:1564)
        at org.apache.hadoop.dfs.NameNode.mkdirs(NameNode.java:450)
        at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

        at org.apache.hadoop.ipc.Client.call(Client.java:715)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
        at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
        at org.apache.hadoop.dfs.$Proxy0.mkdirs(Unknown Source)
        at org.apache.hadoop.dfs.DFSClient.mkdirs(DFSClient.java:702)
        ... 22 more

shell returned 1



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message