hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antonio Magnaghi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-32) Abstraction Layer to decouple Pig from Back-End
Date Thu, 13 Dec 2007 00:13:43 GMT

    [ https://issues.apache.org/jira/browse/PIG-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551166

Antonio Magnaghi commented on PIG-32:

Recording in the bug log some discussion points from a conversation with Utkarsh on the separation
of front-end from back-end:

Data Storage Portion:
- add a "isContainer" method to the DataStorageElement interface

Execution Engine Portion:
- Introduce the concept of Job in the type hierarchy. This would be the result of executing/submitting
a physical plan to the execution engine

- as part of the Local Back-End re-write:

   - currently co-group uses in mem data structures. we should switch over to spillable data
structures instead
   - a general suggestion is to favor clarity of the code. A possibility we discussed was
that some operators like LORead, LOSplits, and intermediate results can be removed.

> Abstraction Layer to decouple Pig from Back-End
> -----------------------------------------------
>                 Key: PIG-32
>                 URL: https://issues.apache.org/jira/browse/PIG-32
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>            Reporter: Antonio Magnaghi
>            Assignee: Antonio Magnaghi
>         Attachments: DataStorage.diff, DataStorage20071212.diff
> I'm opening a new issue to track the development work to support an abstraction layer
for Pig as defined at http://wiki.apache.org/pig/PigAbstractionLayer

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message