hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antonio Magnaghi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-32) Abstraction Layer to decouple Pig from Back-End
Date Thu, 27 Dec 2007 20:31:43 GMT

     [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Antonio Magnaghi updated PIG-32:

    Attachment: patch2007_12_27.diff

This patch includes yesterday's patch (as it has not been committed yet) plus the changes

With these changes I was able to get a full pass on the Pig test suite.

- Fixed the implementation of PigServer::capacity: it was still explicitly using Hadoop DFS,
instead should use the abstract data storage class provided as part of the abstraction API
and there was a dynamic cast that is now wrong.

- Added keys to the DataStorage interface for collecting statistics on capacity and provided
support for those in the HDataStorage

- Fixed POMapredeuce: specifically, the copy operation was not properly handling the copy
of the physical Op table

- Fixed implementation of openDFSFile in FileLocalizer: it was not instantiating a DataStorage
object to access distributed data when running as part of a map-reduce job, causing the job
to fail.

- Have also cleaned up some source files removing packages that were imported but now not
used anymore.

> Abstraction Layer to decouple Pig from Back-End
> -----------------------------------------------
>                 Key: PIG-32
>                 URL: https://issues.apache.org/jira/browse/PIG-32
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>            Reporter: Antonio Magnaghi
>            Assignee: Antonio Magnaghi
>         Attachments: DataStorage.diff, DataStorage20071212.diff, patch2007_12_26.diff,
patch2007_12_26_II.diff, patch2007_12_27.diff
> I'm opening a new issue to track the development work to support an abstraction layer
for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message