falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-2234) Falcon feed fails for RDBMS import if extract type is incremental
Date Mon, 02 Jan 2017 10:58:58 GMT
Prashant created FALCON-2234:
--------------------------------

             Summary: Falcon feed fails for RDBMS import if extract type is incremental
                 Key: FALCON-2234
                 URL: https://issues.apache.org/jira/browse/FALCON-2234
             Project: Falcon
          Issue Type: Bug
          Components: falcon-ui
    Affects Versions: 0.10
            Reporter: Prashant
            Priority: Critical


Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException
when extract type is incremental.
As per the Apache doc Falcon supports 

Import policy
The valid combinations are:

[full,snapshot] - data is extracted in full and dumped into the feed instance location.
[incremental, append] - data is extracted incrementally using the key specified in the deltacolumn

As per the below code , it appears we only check [full,snapshot] and [incremental, append]
 goes to exception 

https://github.com/apache/falcon/blob/0.10/common/src/main/java/org/apache/falcon/entity/parser/FeedEntityParser.java

Snippet 
[
private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException
{
        Extract extract = cluster.getImport().getSource().getExtract();

        if (ExtractMethod.FULL == extract.getType())  
        {
            if ((MergeType.SNAPSHOT != extract.getMergepolicy())
                    || (extract.getDeltacolumn() != null)) 
                 {
                throw new ValidationException(String.format("Feed %s is using FULL "
                        + "extract method but specifies either a superfluous "
                        + "deltacolumn  or a mergepolicy other than snapshot", feed.getName()));
                }
        }  

         else {
            throw new ValidationException(String.format("Feed %s is using unsupported "
                    + "extraction mechanism %s", feed.getName(), extract.getType().value()));
                  }
    }

]

It must have a clause to also check [incremental, append]  .




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message