falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkat Ranganathan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1977) Move falcon-unit to addons as it requires a higher hadoop version
Date Sun, 22 May 2016 02:50:12 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295365#comment-15295365

Venkat Ranganathan commented on FALCON-1977:

When i initially tested, there was a falcon-unit-0.10-SNAPSHOT.jar in my local repo that satisfied
the dependency in webapp.   I tried a few maven compiler plugin changes with the new profile,
but it is getting quickly bigger.   So, I think that  the move to addons is not going to go
well and it does not make sense to ignore all tests also.

So, changed the base hadoop.version to 2.7.1 and this is atleast consistent with the unit
test and regression test  requirements now.

If there is further discussions on the reason to stick with 2.6.2 let us discuss separately
(we can still do unit alone to be 2.7.1 as we had before), but that seems counter productive.

If we stick with Hadoop version to be 2.7.1, then we should enable hdfs-snapshot-mirroring
to be available by default also and explore if hivedr can be enabled by default also.

> Move falcon-unit to addons as it requires a higher hadoop version
> -----------------------------------------------------------------
>                 Key: FALCON-1977
>                 URL: https://issues.apache.org/jira/browse/FALCON-1977
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Venkat Ranganathan
>            Assignee: Venkat Ranganathan
> Falcon-unit requires Hadoop 2.7.x APIs.   This is different from the falcon main which
uses 2.6.2 -  There were some discussions on keeping falcon version to 2.6.2
> As part of cleaning up hadoop-2 profile, it would be good to move falcon-unit to addons
like hivedr, hdfs-snapshot-mirroring etc which require higher versions of components and enabled
with a specific profile.   That way, users can build Falcon after explicitly updating the
component versions.
> But given that Falcon unit (an integration test) and falcon regression both require higher
version of Hadoop, it might be good to think about moving Falcon base to also default to Hadoop
2.7.1.  Will create a separate JIRA for that.

This message was sent by Atlassian JIRA

View raw message