Mailing-List: contact dev-help@datafu.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@datafu.incubator.apache.org
Date: Tue, 12 Sep 2017 09:25:00 +0000 (UTC)
From: "Eyal Allweil (JIRA)" <jira@apache.org>
To: dev@datafu.incubator.apache.org
Message-ID: <JIRA.13101580.1505208296000.89125.1505208300174@Atlassian.JIRA>
In-Reply-To: <JIRA.13101580.1505208296000@Atlassian.JIRA>
References: <JIRA.13101580.1505208296000@Atlassian.JIRA> <JIRA.13101580.1505208296617@jira-lw-us.apache.org>
Subject: [jira] [Created] (DATAFU-127) New macro - samply by keys
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 12 Sep 2017 09:25:10 -0000

Eyal Allweil created DATAFU-127:
-----------------------------------

             Summary: New macro - samply by keys
                 Key: DATAFU-127
                 URL: https://issues.apache.org/jira/browse/DATAFU-127
             Project: DataFu
          Issue Type: New Feature
            Reporter: Eyal Allweil
            Assignee: Eyal Allweil


Two macros that return a sample of a larger table based on a list of keys, with the schema of the larger table. One of the macros filters by dates, the other doesn't.

If there are multiple rows with a key that appears in the key list, all of them will be returned (no deduplication is done). The results are returned ordered by the key field in a single file.

The implementation uses a replicated join for efficiency, but this means the key list shouldn't be too large as to not fit in memory.

The first macro's definition looks as follows:

DEFINE sample_by_keys(table, sample_set, join_key_table, join_key_sample) returns out {

- table_name 				- table name to sample
- sample_set 				- a set of keys
- join_key_table 			- join column name in the table
- join_key_sample 			- join column name in the sample


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)