hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Date Tue, 21 Mar 2017 19:02:53 GMT

I'm an =0 I'm afraid

I think those extra files need to be pulled which triggers a repackage, at which point there's
a couple of 

HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,

of the two, the ADL FS one is the most significant, though if there was going to be a re-roll,
I'd like them in.

I also worry about windows test coverage, as irrespective of whether people are planning to
run it on a windows host in production, the hadoop libs get used client side in job submit,
and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib
problems.. someone needs to look at it a bit more. However, I don't think it should really
be a blocker, given the ASF doesn't officially redistribute windows binaries.

so: some issues with packaging and a couple of patches which need to go in. Test coverage
on windows is potentially longer, and I don't want it to hold things up.

Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the
only way that we'll get all the in-the-field bugreps coming back in. We will need to release
some new versions in the coming weeks to handle those

-Steve




> On 21 Mar 2017, at 18:43, Haibo Chen <haibochen@cloudera.com> wrote:
> 
> Thanks Junping for working on the new release!
> 
> +1 non-binding
> 
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
> 
> Best,
> Haibo Chen
> 
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jdu@hortonworks.com> wrote:
> 
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>> 
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jzhuge@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>>  <property>
>>    <name>fs.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>  </property>
>> 
>>  <property>
>>    <name>fs.AbstractFileSystem.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.Adl</value>
>>  </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> 
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jzhuge@cloudera.com<mailto:jzhuge@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>> 
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>> 
>> I am working on a patch.
>> 
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Mime
View raw message