hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Junping Du <...@hortonworks.com>
Subject Re: [VOTE] Release Apache Hadoop 2.7.2 RC1
Date Thu, 14 Jan 2016 14:53:20 GMT
Thanks Vinod for working hard on this! It is really a huge effort to take care of a release
with 154 commits.

I did a sanity check for commits on 2.7.2, all looks fine except HDFS-8767 is missing from
CHANGES.txt due to original commit issue. I already update it, and I think we are in good
shape now.​



From: Vinod Kumar Vavilapalli <vinodkv@apache.org>
Sent: Wednesday, January 13, 2016 11:15 PM
To: Junping Du
Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; common-dev@hadoop.apache.org;
Subject: Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

Okay, did a whole bunch of things on 2.7.2.

Fixing CHANGES.txt entry that Junping reported on the voting thread
 - 56e2dcc HADOOP-5323. Trash documentation should describe its directory structure and configurations.
(Weiwei Yang via ozawa) Moved entry from HDFS CHANGES.txt to that of common.

Detected one more issue: YARN-2859 addendum’s got missed in 2.7.2
 - 2e1c617 YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt.
 - d738e2e YARN-2859.addendum: fix the remaining issue from the previous patch

Also removed multiple duplicate CHANGES.txt entries for MAPREDUCE-6549

 - 91b7f94 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer
binds to default port 8188. (Meng Ding via wangda)
 - 3c1b25b HADOOP-12413. AccessControlList should avoid calling getGroupNames in isUserInList
with empty groups. Contributed by Zhihai Xu. (cherry picked from commit b2017d9b032af20044fdf60ddbd1575a554ccb79)
 - 17ad3b1 HADOOP-12526. there are duplicate dependency definitions in pom's (sjlee)
 - c3ba72c YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony
 - 096ac9e MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee)
 - 3c0e4de HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy
 - ad829ff MAPREDUCE-6377. JHS sorting on state column not working in webUi. Contributed by
zhihai xu. (cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)
 - 10f23dc HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed
by Kazuho Fujii.
 - 10a6298 YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster
resource calculations. Contributed by Varun Vasudev (cherry picked from commit d36b6e045f317c94e97cb41a163aa974d161a404)
 - 09d3e47 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock.
Contributed by Chang Li.
 - 89f6716 MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading.
Contributed by Nathan Roberts (cherry picked from commit cab3c7c8892ad33a7eb0955b01e99872ab95e192)
 - 4e84a8b YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup.
Contributed by Kuhu Shukla (cherry picked from commit f5acf94ecafb301a0cc8e8f91f19c8bcbc8da701)
 - 4765a65 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records
(wilfreds via rkanter) (cherry picked from commit 7fd00b3db4b7d73afd41276ba9a06ec06a0e1762)
 - 0fc19f6 YARN-4348. ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding
blocking ZK's event thread. (ozawa)
 - 3e192f8 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed
by Weiwei Yang.

Ran these tests that got changed in the above patches, before pushing the backports:
 - TestAccessControlList,DFSTestUtil,TestCommitBlockWithInvalidGenStamp,TestHDFSConcat,TestLineRecordReader,TestMRTimelineEventHandling,TestDistributedShell,TestFileSystemNodeLabelsStore,TestCapacityScheduler

@Junping, mind giving a look at the branch for sanity checks?


On Jan 13, 2016, at 11:02 AM, Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vinodkv@apache.org>>

Thanks for the comments Tsuyoshi / Andrew / Varun / Junping / Akira.

I agree that where possible we should serialize releases and make them incremental w.r.t fixes.

Will roll a new RC for 2.7.2 after the backports.

If there are more thoughts on releases, which I’m sure we will all do have, let’s fork
the thread off from this voting conversation.


On Dec 29, 2015, at 6:50 AM, Junping Du <jdu@hortonworks.com<mailto:jdu@hortonworks.com>>

I am +1 with pulling all of these tickets into 2.7.2.
- For “any fix in 2.6.3 to be there in all releases that get out after 2.6.3 release date”
Shall we conclude this as a general rule - "any fix in 2.x.y to be there in all 2.b.c releases
(while b>=x) that get out after 2.x.y release date"? I am generally fine with this, but
just feel it sounds to set too strong restrictions among branches. Some fixes could be trivial
(test case fix, etc.) enough to deserve more flexibility.​ I would prefer this rule only
applies on critical/blocker fixes, but not applies on minor/trivial issues.
Just 2 cents.



From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vinodkv@apache.org>>
Sent: Thursday, December 24, 2015 12:47 AM
To: Junping Du
Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>; yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>;
common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

I retract my -1. I think we will need to discuss this a bit more.

Beyond those two tickets, there are a bunch more (totaling to 16) that are in 2.6.3 but *not*
in 2.7.2. See this: https://issues.apache.org/jira/issues/?jql=key%20in%20%28HADOOP-12526%2CHADOOP-12413%2CHADOOP-11267%2CHADOOP-10668%2CHADOOP-10134%2CYARN-4434%2CYARN-4365%2CYARN-4348%2CYARN-4344%2CYARN-4326%2CYARN-4241%2CYARN-2859%2CMAPREDUCE-6549%2CMAPREDUCE-6540%2CMAPREDUCE-6377%2CMAPREDUCE-5883%2CHDFS-9431%2CHDFS-9289%2CHDFS-8615%29%20and%20fixVersion%20!%3D%202.7.0<https://issues.apache.org/jira/issues/?jql=key%20in%20(HADOOP-12526,HADOOP-12413,HADOOP-11267,HADOOP-10668,HADOOP-10134,YARN-4434,YARN-4365,YARN-4348,YARN-4344,YARN-4326,YARN-4241,YARN-2859,MAPREDUCE-6549,MAPREDUCE-6540,MAPREDUCE-6377,MAPREDUCE-5883,HDFS-9431,HDFS-9289,HDFS-8615)%20and%20fixVersion%20!=%202.7.0>

Two options here, depending on the importance of ‘causality' between 2.6.x and 2.7.x lines.
 - Ship 2.7.2 as we voted on here
 - Pull these 16 tickets into 2.7.2 and roll a new RC

What do people think? Do folks expect “any fix in 2.6.3 to be there in all releases that
get out after 2.6.3 release date (December 16th)”?


On Dec 23, 2015, at 12:37 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vinodkv@apache.org>>

Sigh. Missed this.

To retain causality ("any fix in 2.6.3 will be there in all releases that got out after 2.6.3”),
I’ll get these patches in.

Reverting my +1, and casting -1 for the RC myself.

Will spin a new RC, this voting thread is marked dead.


On Dec 22, 2015, at 8:24 AM, Junping Du <jdu@hortonworks.com<mailto:jdu@hortonworks.com>>

However, when I look at our commit log and CHANGES.txt, I found something we are missing:
1. HDFS-9470 and YARN-4424 are missing from the 2.7.2 branch and RC1 tag.
2. HADOOP-5323, HDFS-8767 are missing in CHANGE.txt

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message