pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Parvulescu (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1236) NPE when trying to cast an AVG null value to int
Date Thu, 11 Feb 2010 08:30:28 GMT
NPE when trying to cast an AVG null value to int 

                 Key: PIG-1236
                 URL: https://issues.apache.org/jira/browse/PIG-1236
             Project: Pig
          Issue Type: Bug
          Components: grunt
    Affects Versions: 0.6.0
         Environment: Hadoop 0.20.1, Ubuntu 9.04 , Sun Java 1.6.0_16
            Reporter: Alex Parvulescu

When you do AVG on a set of values that contains nulls it will return a null value. If you
cast the AVG to int, not knowing that it will return null, you'll get a NPE.

What I did it is:
 'C = FOREACH B GENERATE group, (int)AVG(A.v) as statsavg;'
The problem here is the AVG ran into some null values and returned null. And consequently
the cast failed with a NPE.

This is the stacktrace
2010-02-09 11:14:36,444 [Thread-85] WARN  org.apache.hadoop.mapred.LocalJobRunner - job_local_0006
    at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:282)
    at org.apache.pig.builtin.IntAvg.getValue(IntAvg.java:39)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:208)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:281)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:182)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:352)
    at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:277)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:423)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:391)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:371)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:239)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)

Now, because I'm not well aware how this works, I did not realize that the cast throws the
NPE and not the computation of the average function on null values provided by the data set.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message