hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Parvulescu (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1236) NPE when trying to cast an AVG null value to int
Date Thu, 11 Feb 2010 08:30:28 GMT
NPE when trying to cast an AVG null value to int 
-------------------------------------------------

                 Key: PIG-1236
                 URL: https://issues.apache.org/jira/browse/PIG-1236
             Project: Pig
          Issue Type: Bug
          Components: grunt
    Affects Versions: 0.6.0
         Environment: Hadoop 0.20.1, Ubuntu 9.04 , Sun Java 1.6.0_16
            Reporter: Alex Parvulescu


When you do AVG on a set of values that contains nulls it will return a null value. If you
cast the AVG to int, not knowing that it will return null, you'll get a NPE.

What I did it is:
 'C = FOREACH B GENERATE group, (int)AVG(A.v) as statsavg;'
The problem here is the AVG ran into some null values and returned null. And consequently
the cast failed with a NPE.

This is the stacktrace
2010-02-09 11:14:36,444 [Thread-85] WARN  org.apache.hadoop.mapred.LocalJobRunner - job_local_0006
java.lang.NullPointerException
    at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:282)
    at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:39)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:208)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:182)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:352)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:277)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:423)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:391)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:371)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:239)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)

Now, because I'm not well aware how this works, I did not realize that the cast throws the
NPE and not the computation of the average function on null values provided by the data set.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message