hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <>
Subject [jira] Commented: (HIVE-726) Make "ant package -Dhadoop.version=0.17.0" work
Date Wed, 05 Aug 2009 21:29:17 GMT


Todd Lipcon commented on HIVE-726:

I think there is some confusion about how the shims work.

The point of the new shims package as done in HIVE-487 is that, on every build, the shims
are build for all major versions of Hadoop. The versions in the ivy.xml there were picked
as the most recent versions, but as Ashish said, we could use the .0 releases instead - it's
somewhat arbitrary. After the shims have been built with each of the hadoop versions, it makes
a hive_shims.jar which includes *all* of the implementations.

At this point, when Hadoop is built, it can be built against any version of Hadoop using the
-Dhadoop.version flag. In actuality this should not matter - it may be that there is still
some work yet to be done, but in my testing I build with the default hadoop.version (0.19.0)
and then ran the build products against 18 and 20 with no recompile. The ShimLoader class
determines the current hadoop version at runtime and loads the correct implementation class
out of hive_shims.jar.

So, -1 on the patch, since it would result in a hive_shims.jar that only includes one version
of the shims, and thus the build product would only work on that version of Hadoop.

> Make "ant package -Dhadoop.version=0.17.0" work
> -----------------------------------------------
>                 Key: HIVE-726
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Build Infrastructure
>            Reporter: Zheng Shao
>         Attachments: HIVE-726.1.patch, HIVE-726.2.patch
> After HIVE-487, users will have to specify the versions as in "shims/ivy.xml" to make
"ant package -Dhadoop.version=<version>".work.
> Currently it is only running fine with the following versions (from shims/ivy.xml):,
0.18.3, 0.19.0, 0.20.0.
> We used to do "ant package -Dhadoop.version=0.17.0" but it's not working any more, although
we can specify "ant package -Dhadoop.version=" and the package will probably still
work with 0.17.0.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message