Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of zshen@hortonworks.com
 designates 209.85.220.48 as permitted sender)
MIME-Version: 1.0
Date: Fri, 17 Jan 2014 12:53:48 -0800
Message-ID: 
 <CAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com>
Subject: [VOTE] Merge YARN-321 Generic Application History Service to trunk
From: Zhijie Shen <zshen@hortonworks.com>
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7b67328a15c99504f030bd6a

--047d7b67328a15c99504f030bd6a
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi folks,

As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p)=
,
I would like to call a vote to merge the YARN-321 branch for Generic
Application History Server into trunk.

*Scope of the changes*


The changes enable ResourceManager to record the historic information of
the application, the application attempt and the container in terms of
events via a history writer. In addition, the changes setup up an
application history server, which allows users to access the recorded
information via RPC interface, web UI and REST APIs.

*Details of development*


 - Development of the feature is tracked in the jira -
https://issues.apache.org/jira/browse/YARN-321.
 - Development has been done in a separate branch -
https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
 - The feature development involved about 35 subtasks.
 - The up-to-date design is posted at -
https://issues.apache.org/jira/secure/attachment/12619638/Generic
Application
History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachm=
ent/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>

 - The uber merge patch Jira -
https://issues.apache.org/jira/browse/YARN-1587

*Testing*


A number of unit tests have been added as a part of the feature. In
addition, we=92ve also done end-to-end functional tests, and performance
tests for HDFS-based history storage and history events processing. Last
but not least, we have updated branch YARN-321 against the latest trunk,
edited merge conflicts, fixed test failures caused by merge, and corrected
a bunch of bad source code issues. The uber merge patch that contains all
the diff between branch YARN-321 and trunk has been run through Jenkins.

*Pending work*


 - Make it work in secure mode
 - Pending bug fixes

We wish to merge the branch now instead of waiting for later. The main
reason for this is that as the branch grew in size, the cost of its
maintenance became huge. Once the feature is merged into trunk, we will
continue to work on pending work like security stuff, to test and fix any
bugs that may be found on the trunk, and to refactor the code about to
share some pieces in PRC and web interfaces.

*Release status*


If the security stuff and the pending fixes arrive by the time everything
else planned for Release 2.4 is done, we can include it as well. This is
what we are striving for. Otherwise, we will call AHS not-feature-complete
and not stable.

The bulk of the design and implementation was done by Mayank Bansal and me
with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
feedback on the design discussions.


This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.

Thanks,
Zhijie

--=20
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

--=20
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to=
=20
which it is addressed and may contain information that is confidential,=20
privileged and exempt from disclosure under applicable law. If the reader=
=20
of this message is not the intended recipient, you are hereby notified that=
=20
any printing, copying, dissemination, distribution, disclosure or=20
forwarding of this communication is strictly prohibited. If you have=20
received this communication in error, please contact the sender immediately=
=20
and delete it from your system. Thank You.

--047d7b67328a15c99504f030bd6a--