Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.apache.org
Date: Mon, 9 Jan 2017 14:22:58 +0000 (UTC)
From: "Prashant (JIRA)" <jira@apache.org>
To: dev@falcon.incubator.apache.org
Message-ID: <JIRA.13031481.1483354679000.685850.1483971778522@Atlassian.JIRA>
In-Reply-To: <JIRA.13031481.1483354679000@Atlassian.JIRA>
References: <JIRA.13031481.1483354679000@Atlassian.JIRA> <JIRA.13031481.1483354679399@arcas>
Subject: [jira] [Commented] (FALCON-2234) Falcon feed fails for RDBMS import
 if extract type is incremental
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 09 Jan 2017 14:23:03 -0000


    [ https://issues.apache.org/jira/browse/FALCON-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15811891#comment-15811891 ] 

Prashant commented on FALCON-2234:
----------------------------------

Hi Ajay,

So the desired behaviour is to remove the same from the Falcon UI? 

> Falcon feed fails for RDBMS import if extract type is incremental
> -----------------------------------------------------------------
>
>                 Key: FALCON-2234
>                 URL: https://issues.apache.org/jira/browse/FALCON-2234
>             Project: Falcon
>          Issue Type: Bug
>          Components: falcon-ui
>    Affects Versions: 0.10
>            Reporter: Prashant
>            Priority: Critical
>              Labels: documentaion, newbie
>
> Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException
> when extract type is incremental.
> As per the Apache doc Falcon supports 
> Import policy
> The valid combinations are:
> [full,snapshot] - data is extracted in full and dumped into the feed instance location.
> [incremental, append] - data is extracted incrementally using the key specified in the deltacolumn
> As per the below code , it appears we only check [full,snapshot] and [incremental, append]  goes to exception 
> https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java
> Snippet 
> {code}
> /**
>      * Validate extraction and merge type combination. Currently supported combo:
>      *
>      * ExtractionType = FULL and MergeType = SNAPSHOT.
>      * ExtractionType = INCREMENTAL and MergeType = APPEND.
>      *
>      * @param feed Feed entity
>      * @param cluster Cluster referenced in the Feed definition
>      * @throws FalconException
>      */
>     private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException {
>         Extract extract = cluster.getImport().getSource().getExtract();
>         if (ExtractMethod.FULL == extract.getType())  {
>             if ((MergeType.SNAPSHOT != extract.getMergepolicy())
>                     || (extract.getDeltacolumn() != null)) {
>                 throw new ValidationException(String.format("Feed %s is using FULL "
>                         + "extract method but specifies either a superfluous "
>                         + "deltacolumn  or a mergepolicy other than snapshot", feed.getName()));
>             }
>         }  else {
>             throw new ValidationException(String.format("Feed %s is using unsupported "
>                     + "extraction mechanism %s", feed.getName(), extract.getType().value()));
>         }
>     }
> {code}
> It must have a clause to also check [incremental, append]  .


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)