hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kuhu Shukla <kshu...@yahoo-inc.com.INVALID>
Subject Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
Date Thu, 16 Mar 2017 14:16:34 GMT
+1 (non-binding)-Downloaded source.-Verified signatures.- Compiled the source.-Ran sample jobs
like MR sleep on pseudo distributed cluster. (Mac OS)
Thanks Junping and others!Regards,Kuhu
On Wednesday, March 15, 2017, 7:25:46 PM CDT, Junping Du <jdu@hortonworks.com> wrote:bq.
From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related
code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of
the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?,
it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <andrew.wang@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


>From my understanding, this issue is related to our previous improvements with separating
client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then
we will hit the issue reported.

>From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related
code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of
the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of
users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using
new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar
or adding dependency between client jar and server jar. There must be some arguments there
on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever
side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this
issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times
for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client
depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People
can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in
some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as
a Known Issue and fix it later.

Best,
Andrew
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message