pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "venkata kamalnath (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3499) Pig job fails when run in local mode with namenode HA(QJM)
Date Fri, 04 Oct 2013 00:31:42 GMT

    [ https://issues.apache.org/jira/browse/PIG-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785735#comment-13785735
] 

venkata kamalnath commented on PIG-3499:
----------------------------------------

yes I am running pig job in local mode using pig -x local test.pig . I am testing pig jobs
with hadoop environment setup in HA mode. 

Steps to reproduce:

Set up hadoop HA namenode setup and run pig jobs in using -x localmode.



> Pig job fails when run in local mode with namenode HA(QJM) 
> -----------------------------------------------------------
>
>                 Key: PIG-3499
>                 URL: https://issues.apache.org/jira/browse/PIG-3499
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt, parser, tools
>    Affects Versions: 0.10.0
>            Reporter: venkata kamalnath
>
> when we run a pig script with namenode HA(QJM) we always get unknown host exception.
The nameserviceID is being considered as host and pig job giving unknown host exception.
> I am working on this fix but want community to validate whether any bug reported similar
to this. If not I will provide the fix as soon as possible.
> The pig script is as below:
> testTable = LOAD 'hdfs://kdvenkata/user/kd/test.csv'
>           USING PigStorage(',')
>           AS (col1:chararray, col2:chararray, col3:int);
> STORE testTable into '/tmp/test_pig_output';
> Exception:
> Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: kdvenkata
>         at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:417)
>         at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
>         at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:412)
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:379)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:123)
>         at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2278)
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:86)
>         at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2312)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2294)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:317)
>         at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:70)
>         at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:53)
>         at org.apache.pig.builtin.JsonMetadata.findMetaFile(JsonMetadata.java:106)
>         at org.apache.pig.builtin.JsonMetadata.getSchema(JsonMetadata.java:188)
>         at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:465)
>         at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
>         at org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:110)
>         at org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68)
>         at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60)
>         at org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84)
>         at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
>         at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
>         at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
>         at org.apache.pig.PigServer$Graph.compile(PigServer.java:1626)
>         at org.apache.pig.PigServer$Graph.compile(PigServer.java:1620)
>         at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1343)
>         at org.apache.pig.PigServer.storeEx(PigServer.java:960)
>         at org.apache.pig.PigServer.store(PigServer.java:928)
>         at org.apache.pig.PigServer.openIterator(PigServer.java:841)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message