hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akira AJISAKA <ajisa...@oss.nttdata.co.jp>
Subject Re: [VOTE] Release Apache Hadoop 2.6.1 RC1
Date Mon, 21 Sep 2015 18:54:45 GMT
+1 (non-binding) if MAPREDUCE-6334 is addressed.

- Downloaded and verified signatures and checksums
- Deployed a single node cluster
- Built Tez 0.7.0 with Hadoop 2.6.1 pom
- Built Hive 1.2.1 with Hadoop 2.6.1 pom
- Ran some Hive on Tez queries

Thanks everyone who worked on this release.

-Akira

On 9/22/15 03:19, Eric Payne wrote:
> Hi Vinod and everyone else who helped on this release!
>
> Thank you very much for going through the work and effort to put this release together.
> While doing my testing for Hadoop 2.6.1 RC1, I encountered the following issue:[MAPREDUCE-6334]
Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle
handler - ASF JIRA
>
> |   |
> |   |   |   |   |   |
> | [MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException during
...We are seeing this happen when an NM's disk goes bad during the creation of map output(s)
the reducer's fetcher can read the shuffle header and reserve the memory  |
> |  |
> | View on issues.apache.org | Preview by Yahoo |
> |  |
> |   |
>
>
> This is actually something we ran across frequently when running 2.6. You may want to
consider pulling that in to 2.6.
> Other than that, it looks fine. I did the following manual testing on a one-node cluster:
> - successfully downloaded and compiled the code- successfully ran streaming jobs- successfully
ran distributed shell jobs-- as part of the distributed shell testing, I started the jobs
with the '-keep_containers_across_application_attempts' property set, killed the AM container,
and verified that it would in fact keep the containers running across AM restarts.- sucessfully
ran wordcount jobs.
> Thank you,
> -Eric Payne
>
>        From: Vinod Kumar Vavilapalli <vinodkv@apache.org>
>   To: common-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
mapreduce-dev@hadoop.apache.org
> Cc: vinodkv@apache.org; Sangjin Lee <sjlee0@gmail.com>; Akira AJISAKA <ajisakaa@oss.nttdata.co.jp>
>   Sent: Wednesday, September 16, 2015 9:10 PM
>   Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1
>
> Hi all,
>
> After a nearly month long [1] toil, with loads of help from Sangjin Lee and
> Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
> candidate RC1 for hadoop-2.6.1.
>
> RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
> for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
> YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
> bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
> YARN-2890) that Jeff Zhang raised.
>
> The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
>
> The RC tag in git is: release-2.6.1-RC1
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/orgapachehadoop-1021
>
> Some notes from our release process
>   -  - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
> non-committed but desired patches. 2.6.1 is already big as is and is late
> by any standard, we can definitely include them in the next release.
>   - The 2.6.1 wiki page [3] captures some (but not all) of the context of
> the patches that we pushed in.
>   - Given the number of fixes pushed [4] in, we had to make a bunch of
> changes to our original plan - we added a few improvements that helped us
> backport patches easier (or in many cases made backports possible), and we
> dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
> HDFS-7611, HDFS-7843, HDFS-8850).
>   - I ran all the unit tests which (surprisingly?) passed. (Except for one,
> which pointed out a missing fix HDFS-7552).
>
> As discussed before [5]
>   - This release is the first point release after 2.6.0
>   - I’d like to use this as a starting release for 2.6.2 in a few weeks and
> then follow up with more of these.
>
> Please try the release and vote; the vote will run for the usual 5 days.
>
> Thanks,
> Vinod
>
> [0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
> [1] Hadoop 2.6.1 Release process thread:
> http://markmail.org/thread/wkbgkxkhntx5tlux
> [2] 2.6.1 Pending tickets:
> https://issues.apache.org/jira/issues/?filter=12331711
> [3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
> -Working-Notes
> [4] List of 2.6.1 patches pushed:
> https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
> %20and%20labels%20%3D%20%222.6.1-candidate%22
> [5] Planning Hadoop 2.6.1 release:
> http://markmail.org/thread/sbykjn5xgnksh6wg
>
> PS:
>   - Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
> fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
> 2.6.1.
>   - The additional patches in RC1 that got into 2.6.1 all the way from 2.8
> are NOT in 2.7.2 yet, this will be done as a followup.
>
>
>


Mime
View raw message