Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3485D10A6B for ; Tue, 8 Oct 2013 03:27:42 +0000 (UTC) Received: (qmail 73216 invoked by uid 500); 8 Oct 2013 03:26:57 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 73051 invoked by uid 500); 8 Oct 2013 03:26:53 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 72982 invoked by uid 500); 8 Oct 2013 03:26:47 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 72824 invoked by uid 99); 8 Oct 2013 03:26:42 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Oct 2013 03:26:42 +0000 Date: Tue, 8 Oct 2013 03:26:41 +0000 (UTC) From: "chiyang (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PIG-3507) It fails to run pig in local mode on a Kerberos enabled Hadoop cluster MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chiyang updated PIG-3507: ------------------------- Affects Version/s: 0.11 > It fails to run pig in local mode on a Kerberos enabled Hadoop cluster > ---------------------------------------------------------------------- > > Key: PIG-3507 > URL: https://issues.apache.org/jira/browse/PIG-3507 > Project: Pig > Issue Type: Bug > Affects Versions: 0.10.0, 0.11 > Reporter: chiyang > > It fails to run pig in local mode on a Kerberos enabled Hadoop cluster > *Command* > pig -x local > > *Pig script* > A = load '/etc/passwd'; > dump A; > > *Root cause* > When running pig in local mode, jobConf in HExecutionEngine is initiated with core-default.xml (hadoop.security.authentication = simple), mapred-default.xml, and yarn-default.xml. However, the settings are not passed to UserGroupInformation. That's why obtainTokensForNamenodesInternal() is called in obtainTokensForNamenodes(), and causes the exception to happen. > {noformat} > public static void obtainTokensForNamenodes(Credentials credentials, Path[] ps, Configuration conf) throws IOException { > if (!UserGroupInformation.isSecurityEnabled()) { > return; > } > obtainTokensForNamenodesInternal(credentials, ps, conf); > } > {noformat} > *Error* > Pig Stack Trace > --------------- > ERROR 6000: Output Location Validation Failed for: 'file:/tmp/temp-308998488/tmp-2025176494 More info to follow: > Can't get JT Kerberos principal for use as renewer > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias A > at org.apache.pig.PigServer.openIterator(PigServer.java:841) > at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696) > at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320) > at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194) > at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) > at org.apache.pig.Main.run(Main.java:604) > at org.apache.pig.Main.main(Main.java:157) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:208) > Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias A > at org.apache.pig.PigServer.storeEx(PigServer.java:940) > at org.apache.pig.PigServer.store(PigServer.java:903) > at org.apache.pig.PigServer.openIterator(PigServer.java:816) > ... 12 more > Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 6000: Output Location Validation Failed for: 'file:/tmp/temp-308998488/tmp-2025176494 More info to follow: > Can't get JT Kerberos principal for use as renewer > at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:95) > at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66) > at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) > at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) > at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52) > at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45) > at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:288) > at org.apache.pig.PigServer.compilePp(PigServer.java:1327) > at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1252) > at org.apache.pig.PigServer.storeEx(PigServer.java:936) > ... 14 more > Caused by: java.io.IOException: Can't get JT Kerberos principal for use as renewer > at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:129) > at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:111) > at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:85) > at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:127) > at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80) > ... 24 more > ================================================================================ -- This message was sent by Atlassian JIRA (v6.1#6144)