hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10101) Update guava dependency to the latest version
Date Mon, 08 May 2017 18:28:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001262#comment-16001262

Junping Du commented on HADOOP-10101:

Thanks [~bpodgursky] for sharing your thoughts. We are glad to hear voices from hadoop users
as well as developers from hadoop and other related projects.

bq. The number of watchers on this ticket should be an indicator in itself, and I'm sure for
every person who bothered creating a jira account, there are a couple hundred who were bitten
by this.
The number of watchers doesn't indicate they are supporting this patch but just they pay attention
to. They have to because this patch break everything. We can bring this topic to hadoop-user
alias to get more audience if needed.

bq. Has anyone contacted the maintainers on the major affected downstream projects like Spark
and HBase? My guess is that they would be more than happy to help work around any breakages
this upgrade causes – if they are like us, they would be overjoyed at being able to finally
I doubt this. I talked with several guys from HBase and Tez community offline - none of them
are happy with doing the same change. That means our change are pushing them to create/maintain
branches for different version of hadoop (if no shading work in HADOOP-14284 which has side
effect though). It also means the releases need to get synchronized, otherwise no downstream
project release can work with Hadoop GA release will be a bigger problem.

bq. Hadoop's ancient Guava dependency is the single largest issue we run into when putting
any other third party jar on our client or task classpath.
Please don't blame hadoop for this. The real problem here is the poor incompatibility of Guava
across different versions and they released 21 major versions across 7 years. The ancient
guava dependency are also suffering from this.

bq. There are many libraries developed outside the Hadoop ecosystem which (rightfully) assume
they can use a newer version of Guava, and we regularly either have to do horrible hacks to
get these onto the client classpath, or tell developers not to use them. This is an incredible
time-waster that shouldn't exist.
Is there any protocol for these libraries outside of hadoop to choose an uniform version of
guava (like version 21)? If not, bump up guava version here does't help as it will still break
these libraries which are using different version. Also, there are also many apps developed
within hadoop ecosystem which should be seen as first class citizenship that get totally break
by this change. As a system software, we should have some better solution like: application
classpath isolation, etc. instead of simply keeping dependency updated.

bq. I understand the concerns about stability, but this is a major upgrade, no? I don't think
it's acceptable to say that Hadoop will be running Guava 11 until Hadoop 4 comes out in 2025.
Guava is not Java, I don't think any hadoop releases should bind with specific guava version.
It is just a third party library - with poor incompatibility - that's it!

> Update guava dependency to the latest version
> ---------------------------------------------
>                 Key: HADOOP-10101
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10101
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Rakesh R
>            Assignee: Tsuyoshi Ozawa
>              Labels: BB2015-05-TBR
>             Fix For: 3.0.0-alpha3
>         Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, HADOOP-10101-005.patch,
HADOOP-10101-006.patch, HADOOP-10101-007.patch, HADOOP-10101-008.patch, HADOOP-10101-009.patch,
HADOOP-10101-009.patch, HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch,
HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, HADOOP-10101.015.patch,
HADOOP-10101.016.patch, HADOOP-10101.017.patch, HADOOP-10101.018.patch, HADOOP-10101.patch,
> The existing guava version is 11.0.2 which is quite old. This issue tries to update the
version to as latest version as possible. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message