pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Araceli Henley (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2791) Can not enter grunt shell when using viewFS filesystem with CSMT and Federation
Date Fri, 13 Jul 2012 00:52:34 GMT

     [ https://issues.apache.org/jira/browse/PIG-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Araceli Henley updated PIG-2791:

    Attachment: asf_test_notes.txt

Short answers: the patch looks good.
Here is a list of the manual tests I attempted. I tried running without a client side mount
table and with a client side table. With hdfs as the default and viewfs as the default. With
two name nodes. Reading and writing accross namenodes. It looks pretty good. There were some
errors, but I believe they are expected due to invalid syntax. 

> Can not enter grunt shell when using viewFS filesystem with CSMT and Federation
> -------------------------------------------------------------------------------
>                 Key: PIG-2791
>                 URL: https://issues.apache.org/jira/browse/PIG-2791
>             Project: Pig
>          Issue Type: Bug
>          Components: grunt
>    Affects Versions: 0.10.0
>         Environment: Pig QE
>            Reporter: patrick white
>            Priority: Blocker
>         Attachments: PIG-2791-0.patch, PIG-2791-1.patch, PIG-2791-2.patch, asf_test_notes.txt
> The Yahoo Pig QE team ran into a blocking issue when trying to test Client-Side Mount
Tables, on a Federated cluster with two NNs, this blocks Pig Testing on Federation. 
> Federation relies strongly on the use of CSMT with viewFS, QE found that in this configuration
it is not possible to enter grunt shell because Pig makes a call to getDefaultReplication()
on the fs, which is ambiguous over viewFS and causes core to throw a org.apache.hadoop.fs.viewfs.NotInMountpointException:
"getDefaultReplication on empty path is invalid".
> This in turn cause Pig to exit with an internal error as follows:
> 2012-07-06 22:20:25,657 [main] INFO  org.apache.pig.Main - Apache Pig version
(r1348169) compiled Jun 08 2012, 17:58:42
> 2012-07-06 22:20:26,074 [main] WARN  org.apache.hadoop.conf.Configuration - mapred.used.genericoptionsparser
is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
> 2012-07-06 22:20:26,076 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to hadoop file system at: viewfs:///
> 2012-07-06 22:20:26,080 [main] WARN  org.apache.hadoop.conf.Configuration - fs.default.name
is deprecated. Instead, use fs.defaultFS
> 2012-07-06 22:20:26,522 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal
error. getDefaultReplication on empty path is invalid
> 2012-07-06 22:20:26,522 [main] WARN  org.apache.pig.Main - There is no log file to write
> 2012-07-06 22:20:26,522 [main] ERROR org.apache.pig.Main - org.apache.hadoop.fs.viewfs.NotInMountpointException:
getDefaultReplication on empty path is invalid
>         at org.apache.hadoop.fs.viewfs.ViewFileSystem.getDefaultReplication(ViewFileSystem.java:482)
>         at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:77)
>         at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:205)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:118)
>         at org.apache.pig.impl.PigContext.connect(PigContext.java:208)
>         at org.apache.pig.PigServer.<init>(PigServer.java:246)
>         at org.apache.pig.PigServer.<init>(PigServer.java:231)
>         at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:47)
>         at org.apache.pig.Main.run(Main.java:487)
>         at org.apache.pig.Main.main(Main.java:111)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message