Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.apache.org
Date: Tue, 29 Sep 2015 14:49:05 +0000 (UTC)
From: "Rishav Rohit (JIRA)" <jira@apache.org>
To: dev@falcon.incubator.apache.org
Message-ID: <JIRA.12901330.1443537889000.103086.1443538145356@Atlassian.JIRA>
In-Reply-To: <JIRA.12901330.1443537889000@Atlassian.JIRA>
References: <JIRA.12901330.1443537889000@Atlassian.JIRA>
 <JIRA.12901330.1443537889036@arcas>
Subject: [jira] [Updated] (FALCON-1498) Add more parameters for HCatalogue
 input feed
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/FALCON-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rishav Rohit updated FALCON-1498:
---------------------------------
    Description: 
Add more parameters like falcon_<input_name>_partitions for HCatalogue input feed.

A use case which I can think of now is described below -
Suppose there are two date partitioned input table and you want to join these tables on date partition column - 
{noformat}
INSERT OVERWRITE TABLE ${falcon_output_database}.${falcon_output_table} PARTITION (${falcon_output_partitions_hive})
SELECT "join_count", count(*)
FROM ${falcon_input1_database}.${falcon_input1_table} a,
${falcon_input2_database}.${falcon_input2_table} b
-- these parameters are not available
WHERE a.${falcon_input1_partitions}
AND b.${falcon_input2_partitions};

--- workaround, use output feed parameters
WHERE a.${falcon_output_partitions}
AND b.${falcon_output_partitions}
{noformat}
Some other workaround is also possible, but presence of more parameters would help.

  was:
Add more parameters like falcon_<input_name>_partitions for HCatalogue input feed.

A use case which I can think of now is described below -
Suppose there are two date partitioned input table and you want to join these tables on date partition column - 
{quote}
INSERT OVERWRITE TABLE ${falcon_output_database}.${falcon_output_table} PARTITION (${falcon_output_partitions_hive})
SELECT "join_count", count(*)
FROM ${falcon_input1_database}.${falcon_input1_table} a,
${falcon_input2_database}.${falcon_input2_table} b
-- these parameters are not available
WHERE a.${falcon_input1_partitions}
AND b.${falcon_input2_partitions};

--- workaround, use output feed parameters
WHERE a.${falcon_output_partitions}
AND b.${falcon_output_partitions}
{quote}
Some other workaround is also possible, but presence of more parameters would help.


> Add more parameters for HCatalogue input feed
> ---------------------------------------------
>
>                 Key: FALCON-1498
>                 URL: https://issues.apache.org/jira/browse/FALCON-1498
>             Project: Falcon
>          Issue Type: Improvement
>          Components: feed
>    Affects Versions: 0.6
>         Environment: HDP-2.2.0
>            Reporter: Rishav Rohit
>            Priority: Minor
>
> Add more parameters like falcon_<input_name>_partitions for HCatalogue input feed.
> A use case which I can think of now is described below -
> Suppose there are two date partitioned input table and you want to join these tables on date partition column - 
> {noformat}
> INSERT OVERWRITE TABLE ${falcon_output_database}.${falcon_output_table} PARTITION (${falcon_output_partitions_hive})
> SELECT "join_count", count(*)
> FROM ${falcon_input1_database}.${falcon_input1_table} a,
> ${falcon_input2_database}.${falcon_input2_table} b
> -- these parameters are not available
> WHERE a.${falcon_input1_partitions}
> AND b.${falcon_input2_partitions};
> --- workaround, use output feed parameters
> WHERE a.${falcon_output_partitions}
> AND b.${falcon_output_partitions}
> {noformat}
> Some other workaround is also possible, but presence of more parameters would help.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)