falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-2234) Falcon feed fails for RDBMS import if extract type is incremental
Date Mon, 02 Jan 2017 10:58:58 GMT
Prashant created FALCON-2234:

             Summary: Falcon feed fails for RDBMS import if extract type is incremental
                 Key: FALCON-2234
                 URL: https://issues.apache.org/jira/browse/FALCON-2234
             Project: Falcon
          Issue Type: Bug
          Components: falcon-ui
    Affects Versions: 0.10
            Reporter: Prashant
            Priority: Critical

Falcon feed is failing with org.apache.falcon.entity.parser.ValidationException
when extract type is incremental.
As per the Apache doc Falcon supports 

Import policy
The valid combinations are:

[full,snapshot] - data is extracted in full and dumped into the feed instance location.
[incremental, append] - data is extracted incrementally using the key specified in the deltacolumn

As per the below code , it appears we only check [full,snapshot] and [incremental, append]
 goes to exception 


private void validateFeedExtractionType(Feed feed, Cluster cluster) throws FalconException
        Extract extract = cluster.getImport().getSource().getExtract();

        if (ExtractMethod.FULL == extract.getType())  
            if ((MergeType.SNAPSHOT != extract.getMergepolicy())
                    || (extract.getDeltacolumn() != null)) 
                throw new ValidationException(String.format("Feed %s is using FULL "
                        + "extract method but specifies either a superfluous "
                        + "deltacolumn  or a mergepolicy other than snapshot", feed.getName()));

         else {
            throw new ValidationException(String.format("Feed %s is using unsupported "
                    + "extraction mechanism %s", feed.getName(), extract.getType().value()));


It must have a clause to also check [incremental, append]  .

This message was sent by Atlassian JIRA

View raw message