datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Hayes (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (DATAFU-12) Implement Lead UDF based on version from SQL
Date Tue, 10 Oct 2017 19:08:00 GMT

     [ https://issues.apache.org/jira/browse/DATAFU-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Matthew Hayes closed DATAFU-12.
-------------------------------
    Resolution: Won't Fix

> Implement Lead UDF based on version from SQL
> --------------------------------------------
>
>                 Key: DATAFU-12
>                 URL: https://issues.apache.org/jira/browse/DATAFU-12
>             Project: DataFu
>          Issue Type: New Feature
>            Reporter: Matthew Hayes
>
> Min Zhou has provided this suggestion ([Issue #88 on GitHub|https://github.com/linkedin/datafu/pull/88]):
> Lead is an analytic function like Oracle's Lead function. It provides access to more
than one tuple of a bag at the same time without a self join. Given a bag of tuple returned
from a query, LEAD provides access to a tuple at a given physical offset beyond that position.
Generates pairs of all items in a bag.
> If you do not specify offset, then its default is 1. Null is returned if the offset goes
beyond the scope of the bag.
> Example 1:
> {noformat}
>    register ba-pig-0.1.jar
>    define Lead datafu.pig.bags.Lead('2');
>    -- INPUT: ({(1),(2),(3),(4)})
>    data = LOAD 'input' AS (data: bag {T: tuple(v:INT)});
>    describe data;
>    -- OUTPUT:  ({((1),(2),(3)),((2),(3),(4)),((3),(4),),((4),,)})
>    -- OUTPUT SCHEMA: data2: {lead_data: {(elem0: (v: int),elem1: (v: int),elem2: (v:
int))}}
>    data2 = FOREACH data GENERATE Lead(data);
>    describe data2;
>    DUMP data2;
> {noformat}
> Example 2
> {noformat}
>    register  ba-pig-0.1.jar
>    define Lead datafu.pig.bags.Lead();
>    -- INPUT: ({(10,{(1),(2),(3)}),(20,{(4),(5),(6)}),(30,{(7),(8)}),(40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})})
>    data = LOAD 'input' AS (data: bag {T: tuple(v1:INT,B: bag{T: tuple(v2:INT)})});
>    --describe data;
>    -- OUPUT: ({((10,{(1),(2),(3)}),(20,{(4),(5),(6)})),((20,{(4),(5),(6)}),(30,{(7),(8)})),((30,{(7),(8)}),(40,{(9),(10),(11)})),((40,{(9),(10),(11)}),(50,{(12),(13),(14),(15)})),((50,{(12),(13),(14),(15)}),)})
>    data2 = FOREACH data GENERATE Lead(data);
>    --describe data2;
>    DUMP data2;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message