hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
Date Wed, 15 Mar 2017 21:04:08 GMT
Hi Junping, inline,

>From my understanding, this issue is related to our previous
> improvements with separating client and server jars in HDFS-6200. If we use
> the new "client" jar in NN HA deployment, then we will hit the issue
> reported.
>
>From my read of the poms, hadoop-client depends on hadoop-hdfs-client to
pull in HDFS-related code. It doesn't have its own dependency on
hadoop-hdfs. So I think this affects users of the hadoop-client artifact,
which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would
affect a lot of users.

> I can see two options here:
>
> - Without any change in 2.8.0, if user hit the issue when they deploy HA
> cluster by using new client jar, adding back hdfs jar just like how things
> work previously
>
> - Make the change now in 2.8.0, either moving
> ConfiguredFailoverProxyProvider to client jar or adding dependency
> between client jar and server jar. There must be some arguments there on
> which way to fix is better especially ConfiguredFailoverProxyProvider
> still has some sever side dependencies.
>
>
> I would prefer the first option, given:
>
> - The issue fixing time is unpredictable as there are still discussion on
> how to fix this issue. Our 2.8.0 release shouldn't be an endless journey
> which has been deferred several times for more serious issue.
>
Looks like we have a patch being actively revved and reviewed to fix this
by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and
Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

> - We have workaround for this improvement, no regression happens due to
> this issue. People can still use hdfs jar in old way. The worst case
> is improvement for HDFS doesn't work in some cases - that shouldn't block
> the whole release.
>
Based on the above, I think there is a regression for users of the
hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we
can document it as a Known Issue and fix it later.

Best,
Andrew

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message