Mailing-List: contact pig-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: pig-dev@hadoop.apache.org
Message-ID: <1316971081.1251246839291.JavaMail.jira@brutus>
Date: Tue, 25 Aug 2009 17:33:59 -0700 (PDT)
From: "Daniel Dai (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Subject: [jira] Updated: (PIG-932) Required fields projection in Loader:
 nested fields in bag/tuple, map key lookup more than two levels
In-Reply-To: <847934098.1251243119422.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/PIG-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-932:
---------------------------

    Description: 
To leverage the performance features provided by Zebra, Pig should be able to figure out which input fields are actually used in Pig script, and prune unnecessary inputs. This feature is being implementing in [PIG-922|https://issues.apache.org/jira/browse/PIG-922]. However, there are two limitations currently:

1. Pruning nested fields only apply to map. We do not prune sub-field inside a bag or tuple
2. For map, currently we only go one level deep. Eg, if in Pig script, user uses a#'key0'#'key1', a#'key0' will be asked

These two limitations are in line with current limitation of Zebra loader. Once Zebra loader can handle this, we need to work to lift these limitations.

  was:
To leverage the performance features provided by Zebra, Pig should be able to figure out which input fields are actually used in Pig script, and prune unnecessary inputs. This feature is being implementing in [PIG-922|https://issues.apache.org/jira/browse/PIG-922]. However, there are two limitations currently:

1. Pruning nested fields only apply to map and record. We do not prune sub-field inside a collection
2. For map, currently we only go one level deep. Eg, if in Pig script, user uses a#'key0'#'key1', a#'key0' will be asked

These two limitations are in line with current limitation of Zebra loader. Once Zebra loader can handle this, we need to work to lift these limitations.

        Summary: Required fields projection in Loader: nested fields in bag/tuple, map key lookup more than two levels  (was: Required fields projection in Loader: nested fields in bag, map key lookup more than two levels)

> Required fields projection in Loader: nested fields in bag/tuple, map key lookup more than two levels
> -----------------------------------------------------------------------------------------------------
>
>                 Key: PIG-932
>                 URL: https://issues.apache.org/jira/browse/PIG-932
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Daniel Dai
>
> To leverage the performance features provided by Zebra, Pig should be able to figure out which input fields are actually used in Pig script, and prune unnecessary inputs. This feature is being implementing in [PIG-922|https://issues.apache.org/jira/browse/PIG-922]. However, there are two limitations currently:
> 1. Pruning nested fields only apply to map. We do not prune sub-field inside a bag or tuple
> 2. For map, currently we only go one level deep. Eg, if in Pig script, user uses a#'key0'#'key1', a#'key0' will be asked
> These two limitations are in line with current limitation of Zebra loader. Once Zebra loader can handle this, we need to work to lift these limitations.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.