Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of vinodkv@hortonworks.com
 designates 209.85.160.50 as permitted sender)
Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\))
Subject: Re: [VOTE] Merge YARN-321 Generic Application History Service to
 trunk
From: Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>
In-Reply-To: 
 <CAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com>
Date: Fri, 17 Jan 2014 15:39:55 -0800
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Message-Id: <2437C21B-00EA-489B-B007-70346C8193F1@hortonworks.com>
References: 
 <CAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com>
To: yarn-dev@hadoop.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable


Thanks Zhijie!

I've been involved in its design. And reviewed almost every patch. Contribu=
ted a little w.r.t tests etc.

The code is in a decent shape bar some bugs and security. The merge should =
let us avoid the maintenance burden.

Tested the latest code on my single node setup. It mostly looks good.

+1 (binding).

Thanks,
+Vinod


On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zshen@hortonworks.com> wrote:

> Hi folks,
>=20
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6=
p),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
>=20
> *Scope of the changes*
>=20
>=20
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
>=20
> *Details of development*
>=20
>=20
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attac=
hment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf=
>
>=20
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
>=20
> *Testing*
>=20
>=20
> A number of unit tests have been added as a part of the feature. In
> addition, we=92ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and correcte=
d
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
>=20
> *Pending work*
>=20
>=20
> - Make it work in secure mode
> - Pending bug fixes
>=20
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
>=20
> *Release status*
>=20
>=20
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complet=
e
> and not stable.
>=20
> The bulk of the design and implementation was done by Mayank Bansal and m=
e
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
>=20
>=20
>=20
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>=20
> Thanks,
> Zhijie
>=20
> --=20
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>=20
> --=20
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity =
to=20
> which it is addressed and may contain information that is confidential,=
=20
> privileged and exempt from disclosure under applicable law. If the reader=
=20
> of this message is not the intended recipient, you are hereby notified th=
at=20
> any printing, copying, dissemination, distribution, disclosure or=20
> forwarding of this communication is strictly prohibited. If you have=20
> received this communication in error, please contact the sender immediate=
ly=20
> and delete it from your system. Thank You.


--=20
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to=
=20
which it is addressed and may contain information that is confidential,=20
privileged and exempt from disclosure under applicable law. If the reader=
=20
of this message is not the intended recipient, you are hereby notified that=
=20
any printing, copying, dissemination, distribution, disclosure or=20
forwarding of this communication is strictly prohibited. If you have=20
received this communication in error, please contact the sender immediately=
=20
and delete it from your system. Thank You.