falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Satish Mittal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-357) HCatalog Feed replication: Hive export job fails when table partition contains multiple dated columns
Date Mon, 17 Mar 2014 18:53:44 GMT

    [ https://issues.apache.org/jira/browse/FALCON-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938199#comment-13938199
] 

Satish Mittal commented on FALCON-357:
--------------------------------------

The HQL used by feed replication to perform partition export:

export table ${falconSourceDatabase}.${falconSourceTable} partition ${falconSourcePartition}
to '${falconSourceStagingDir}';

The export fails because the value of falconSourcePartition is of the format:
falconSourcePartition=(minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')

Hive Export/Import DDL does NOT accept a partition value that consists of AND keywords. Instead
it accepts a partition value string of format: (key1='value1',key2='value2',...).

Currently feed replication code uses oozie coord function dataInPartitionFilter() to create
a partition value string. However this function is meant to create partition filters for Pig/Java/Hive.
We need a new oozie coord function that returns a complete partition value string which can
work with Hive export/import scenarios. 

Created OOZIE-1741 for the same.

> HCatalog Feed replication: Hive export job fails when table partition contains multiple
dated columns
> -----------------------------------------------------------------------------------------------------
>
>                 Key: FALCON-357
>                 URL: https://issues.apache.org/jira/browse/FALCON-357
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Satish Mittal
>            Assignee: Satish Mittal
>
> Suppose a falcon feed is based on an HCatalog table partitioned by (year, month, day,
hour, minute). The feed replication fails during hive export with the following error:
> Intercepting System.exit(40000)
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code
[40000]
> stderr logs:
> FAILED: ParseException line 1:51 mismatched input 'AND' expecting ) near ''00'' in export
statement



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message