falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FALCON-2234) Falcon feed fails for RDBMS import if extract type is incremental
Date Fri, 06 Jan 2017 15:05:58 GMT

     [ https://issues.apache.org/jira/browse/FALCON-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ajay Yadava updated FALCON-2234:
--------------------------------
    Labels: documentaion newbie  (was: )

> Falcon feed fails for RDBMS import if extract type is incremental
> -----------------------------------------------------------------
>
>                 Key: FALCON-2234
>                 URL: https://issues.apache.org/jira/browse/FALCON-2234
>             Project: Falcon
>          Issue Type: Bug
>          Components: falcon-ui
>    Affects Versions: 0.10
>            Reporter: Prashant
>            Priority: Critical
>              Labels: documentaion, newbie
>
> Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException
> when extract type is incremental.
> As per the Apache doc Falcon supports 
> Import policy
> The valid combinations are:
> [full,snapshot] - data is extracted in full and dumped into the feed instance location.
> [incremental, append] - data is extracted incrementally using the key specified in the
deltacolumn
> As per the below code , it appears we only check [full,snapshot] and [incremental, append]
 goes to exception 
> https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java
> Snippet 
> {code}
> /**
>      * Validate extraction and merge type combination. Currently supported combo:
>      *
>      * ExtractionType = FULL and MergeType = SNAPSHOT.
>      * ExtractionType = INCREMENTAL and MergeType = APPEND.
>      *
>      * @param feed Feed entity
>      * @param cluster Cluster referenced in the Feed definition
>      * @throws FalconException
>      */
>     private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException
{
>         Extract extract = cluster.getImport().getSource().getExtract();
>         if (ExtractMethod.FULL == extract.getType())  {
>             if ((MergeType.SNAPSHOT != extract.getMergepolicy())
>                     || (extract.getDeltacolumn() != null)) {
>                 throw new ValidationException(String.format("Feed %s is using FULL "
>                         + "extract method but specifies either a superfluous "
>                         + "deltacolumn  or a mergepolicy other than snapshot", feed.getName()));
>             }
>         }  else {
>             throw new ValidationException(String.format("Feed %s is using unsupported
"
>                     + "extraction mechanism %s", feed.getName(), extract.getType().value()));
>         }
>     }
> {code}
> It must have a clause to also check [incremental, append]  .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message