pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-5085) Support FLATTEN of maps
Date Fri, 17 Feb 2017 00:59:41 GMT

    [ https://issues.apache.org/jira/browse/PIG-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15870970#comment-15870970

Rohini Palaniswamy commented on PIG-5085:

Patch is good. Just 2 minor comments on documentation.
  1) Can skip detail here as explanation for each follows immediately.
   Flatten un-nests tuples as well as bags. It can also be used on maps to break down the
key-value pair structure into a key and a value part within a tuple. -> Flatten un-nests
tuples, bags and maps. 
  2) Can make the map section similar to previous paragraph on bags for easy reading. Would
be good to have details on how to reference them.
   <p>For maps, flatten creates a tuple with two fields containing the key and value.
   If we have a map field named kvpair with input as (m[k1#v1, k2#v2]) and we apply GENERATE
flatten(kvpair), it will generate two tuples (k1,v1) and (k1,v2) which can be accessed as
kvpair::key and kvpair::value.
   When there are additional projections in the expression, a cross product will happen similar
to bags. 
   For example, if we apply the expression GENERATE $0, flatten($1) to the input tuple (a,
m[k1#1, k2#2, k3#3]), we will see (a,k1,1), (a,k2,2) and (a,k3,3) as the result.</p>

> Support FLATTEN of maps
> -----------------------
>                 Key: PIG-5085
>                 URL: https://issues.apache.org/jira/browse/PIG-5085
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Rohini Palaniswamy
>            Assignee: Adam Szita
>             Fix For: 0.17.0
>         Attachments: PIG-5085.0.patch, PIG-5085.1.patch
>   I have come across users asking for this quite a few times. Don't see why we should
not support it with FLATTEN instead of users having to write a UDF for that

This message was sent by Atlassian JIRA

View raw message