hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-18) Making Pig work with HOD 0.4 and hadoop 0.16
Date Fri, 07 Mar 2008 01:06:58 GMT

    [ https://issues.apache.org/jira/browse/PIG-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575997#action_12575997

Olga Natkovich commented on PIG-18:

Just apploaded the patch that makes pig work with hadoop 0.16 and HOD 0.4. 

This patch contains several things:

(1) New hadoop16.jar and I made it the default in build.xml since this is the current hadoop
(2) Changes to integrate with HOD 0.4
   - complete API rewrite; we no longer need expect script and all the strange inetraction
we had to do. Now we just call hod allocate in the beginning and hod deallocate at the end.
   - added shutdown hook to make sure that we disconnect even during abnormal termination
(3) Moved temp file cleanup from shutdown hook into Main.java. The problem with shutdown hook
is that it might be called after DFS shutdown hook in which case cleanup does not happen.
So at this point we cleanup under normal conditions and don't cleanup when client process
is killed via signal. (I opened a bug with hadoop team to help is deal with this issue.)

> Making Pig work with HOD 0.4 and hadoop 0.16
> --------------------------------------------
>                 Key: PIG-18
>                 URL: https://issues.apache.org/jira/browse/PIG-18
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>            Priority: Critical
>         Attachments: PIG-18.patch
> Pig needs to work with the soon to be released hadoop on demand (HOD) version.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message