hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-253) Integrating Pig with Hadoop 0.18
Date Mon, 23 Jun 2008 17:22:45 GMT

    [ https://issues.apache.org/jira/browse/PIG-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607302#action_12607302
] 

Olga Natkovich commented on PIG-253:
------------------------------------

In hadoop18, combiner is no longer guaranteed to be called exactly once. Now it can be called
0 or more times on both map and reduce side.

Pig code as is assumes that combiner is called exactly once. Hadoop guys added a flag to JobConf
to restore backward compatibility for 1 release which will give us a chance to comply with
the change.

Once I made a change and picked new hadoop18.jar, one of the unit tests never finishes. Will
attache the verbose log next

> Integrating Pig with Hadoop 0.18
> --------------------------------
>
>                 Key: PIG-253
>                 URL: https://issues.apache.org/jira/browse/PIG-253
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>         Attachments: hadoop18.jar, PIG-253.patch, PIG-253_v2.patch, PIG-253_v3.patch,
TEST-org.apache.pig.test.TestStreaming.txt
>
>
> Hadoop 0.18 is going into feature freeze by the end of this week. We will need to start
integration soon.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message