hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Reopened: (PIG-179) On hadoop 0.16, some jobs using combiner fail with an NPE
Date Wed, 02 Apr 2008 23:34:24 GMT

     [ https://issues.apache.org/jira/browse/PIG-179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alan Gates reopened PIG-179:
----------------------------


This patch conflicts with the fix for PIG-55.  I'll regenerate it and attach the new patch.

> On hadoop 0.16, some jobs using combiner fail with an NPE
> ---------------------------------------------------------
>
>                 Key: PIG-179
>                 URL: https://issues.apache.org/jira/browse/PIG-179
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.0.0
>         Environment: Hadoop 0.16
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>             Fix For: 0.1.0
>
>         Attachments: PIG-179.patch
>
>
> Some jobs (it appears to only be larger jobs) now fail with an NPE in the combiner code
on this line:
> {code}
> PigSplit split = PigInputFormat.PigRecordReader.getPigRecordReader().getPigFileSplit();
> {code}
> Looking into the PigRecordReader a comment in the class indicates that, as implemented,
it depends on the mapper and splitter (and in this case the combiner as well) running in the
same thread.  It seems that in some cases in hadoop 0.16 this is no longer the case.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message