pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-253) Integrating Pig with Hadoop 0.18
Date Mon, 23 Jun 2008 17:22:45 GMT

    [ https://issues.apache.org/jira/browse/PIG-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607302#action_12607302

Olga Natkovich commented on PIG-253:

In hadoop18, combiner is no longer guaranteed to be called exactly once. Now it can be called
0 or more times on both map and reduce side.

Pig code as is assumes that combiner is called exactly once. Hadoop guys added a flag to JobConf
to restore backward compatibility for 1 release which will give us a chance to comply with
the change.

Once I made a change and picked new hadoop18.jar, one of the unit tests never finishes. Will
attache the verbose log next

> Integrating Pig with Hadoop 0.18
> --------------------------------
>                 Key: PIG-253
>                 URL: https://issues.apache.org/jira/browse/PIG-253
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>         Attachments: hadoop18.jar, PIG-253.patch, PIG-253_v2.patch, PIG-253_v3.patch,
> Hadoop 0.18 is going into feature freeze by the end of this week. We will need to start
integration soon.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message