pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Tieman (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2635) PigServer fails to run mapreduce against hadoop 0.23.
Date Fri, 06 Apr 2012 19:15:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248620#comment-13248620
] 

Brian Tieman commented on PIG-2635:
-----------------------------------

I'm using the Cloudera CDH4b1 build which claims to be compatible with hadoop 0.23.  Everything
has worked fine except PigServer.
                
> PigServer fails to run mapreduce against hadoop 0.23.
> -----------------------------------------------------
>
>                 Key: PIG-2635
>                 URL: https://issues.apache.org/jira/browse/PIG-2635
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.2
>            Reporter: Brian Tieman
>
> Running Pig scripts through Grunt works without hiccup but I have a utility program that
runs commands through PigServer.  When running that way, I run into two problems.
> The first, which I can hack around, is that I need to pass in to PigServer the hbase
configuration.  When I do this with ConfigurationUtil.toProperties(hbaseConfig), the new properties
get passed in-not the deprecated ones.  Later on down the road, Pig looks for fs.default.name
which is the deprecated property for fs.defaultfs and throws [1]  The hack is to back map
the fs.default.name property to the value of fs.defaultfs after calling ConfigurationUtil.toProperties();
> That brings me to the second problem which I haven't been able to hack around -  When
PigServer runs, it throws the exception[2].  Poking through the code, there is no longer an
org.apache.hadoop.mapred.jobcontrol.JobControl.addJob() method distributed with hadoop 0.23-only
an org.apache.hadoop.mapred.jobcontrol.JobControl.addJobs() method-which also appears to be
deprecated. 
> Again, Grunt works fine-just PigServer shows these problems.  Has anyone resolved this?
 Or should I log a bug?
> Thanks!
> [1]
> java.lang.NullPointerException
>         at java.net.URI$Parser.parse(URI.java:3003)
>         at java.net.URI.<init>(URI.java:578)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.setHdfsServers(QueryParser.java:485)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1618)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1366)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:1013)
>         at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:800)
>         at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>         at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1612)
>         at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:558)
>         at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:227)
> [2]
> Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Lja
> va/lang/String;
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:258)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
>         at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
>         at org.apache.pig.PigServer.storeEx(PigServer.java:885)
>         at org.apache.pig.PigServer.store(PigServer.java:827)
>         at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:231)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message