hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13866) Upgrade netty-all to 4.1.1.Final
Date Tue, 28 Feb 2017 02:00:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887061#comment-15887061

Junping Du commented on HADOOP-13866:

bq. As we discussed earlier, this is an incompatible change.
>From my observation on above comments, it seems we haven't get confirmed this is incompatible
change yet. I just saw you were asking the same question twice, and someone reply "might be
an incompatible" but without providing any evidence.

bq. Ted only checked a subset of the versions between our current one (4.1.0.Beta5) and the
version proposed in this patch (4.1.8.Final)
>From https://github.com/netty/netty/wiki/New-and-noteworthy-in-4.1, you can check the
details for changes between 4.1.x and 4.0.x. I don't see any outstanding change to break backward
compatibility. If you found anything in suspect, please list here and I will be appreciated
the evidence in details instead of high level comments.

bq. We already discussed the historical precedent for version bumps earlier, and I still don't
think it's a good reason to bump here.
We should bump up any of our dependent library to a stable release instead of an alpha or
beta version. I would more concern on our previous change to update to a beta version before
instead of concerning an update from beta to a stable release.

bq. I'm -1 on inclusion in the branch-2s without shading. Please do the shading if you want
this included.
IIRC, there was no common agreement to do shading on branch-2 before. I heard some people
have concern that some lazy developers/apps could depend on hadoop library and so shading
could make it another incompatible. Actually, shaded dependencies have pros and cons (complex,
hard to debug) and some other projects are giving up shade like ElasticSearch stop shading
since 2.0 (please refer: https://www.elastic.co/blog/to-shade-or-not-to-shade). So I don't
think making dependency shading is something mandatory here. 

Given my comments above, [~andrew.wang], can you provide more arguments/evidence on your -1
to make it looks reasonable? Also, I would like to hear comments from others.

> Upgrade netty-all to 4.1.1.Final
> --------------------------------
>                 Key: HADOOP-13866
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13866
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Blocker
>         Attachments: HADOOP-13866.v1.patch, HADOOP-13866.v2.patch, HADOOP-13866.v3.patch,
HADOOP-13866.v4.patch, HADOOP-13866.v6.patch, HADOOP-13866.v7.patch, HADOOP-13866.v8.patch,
HADOOP-13866.v8.patch, HADOOP-13866.v8.patch, HADOOP-13866.v9.patch
> netty-all 4.1.1.Final is stable release which we should upgrade to.
> See bottom of HADOOP-12927 for related discussion.
> This issue was discovered since hbase 2.0 uses 4.1.1.Final of netty.
> When launching mapreduce job from hbase, /grid/0/hadoop/yarn/local/  usercache/hbase/appcache/application_1479850535804_0008/container_e01_1479850535804_0008_01_000005/mr-framework/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar
(from hdfs) is ahead of 4.1.1.Final jar (from hbase) on the classpath.
> Resulting in the following exception:
> {code}
> 2016-12-01 20:17:26,678 WARN [Default-IPC-NioEventLoopGroup-1-1] io.netty.util.concurrent.DefaultPromise:
An exception was thrown by org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete()
> java.lang.NoSuchMethodError: io.netty.buffer.ByteBuf.retainedDuplicate()Lio/netty/buffer/ByteBuf;
>         at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:272)
>         at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:262)
>         at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
>         at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
>         at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
>         at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message