hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk
Date Tue, 28 Jan 2014 23:37:43 GMT
Yup, I was building and running tests in the branch.

ALL CLEAR now. YARN-321 merged. I could build branch-2. Ran all the YARN tests successfully.

Let me know if any more issues blow up and/or file JIRA tickets.

+Vinod

On Jan 28, 2014, at 1:08 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> It looks like Vinod just fixed this.  Thanks!
> 
> 
> On Tue, Jan 28, 2014 at 1:00 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:
> 
>> I'm getting build errors on branch-2 because the pom for the new module
>> still refers to 3.0.0-SNAPSHOT:
>> 
>>  <parent>
>>    <artifactId>hadoop-yarn-server</artifactId>
>>    <groupId>org.apache.hadoop</groupId>
>>    <version>3.0.0-SNAPSHOT</version>
>>  </parent>
>>  <modelVersion>4.0.0</modelVersion>
>>  <groupId>org.apache.hadoop</groupId>
>>  <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
>>  <version>3.0.0-SNAPSHOT</version>
>>  <name>hadoop-yarn-server-applicationhistoryservice</name>
>> 
>> -Sandy
>> 
>> 
>> On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>> 
>>> 
>>> Trunk build seems stable now after the merge. A couple of yarn trunk
>>> builds also ran fine (
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is
>>> also cut and is out of the way.
>>> 
>>> I'm now merging this into branch-2 so as to minimize YARN dev effort
>>> across branches as per the discussion thread on YARN-321 merge.
>>> 
>>> Thanks,
>>> +Vinod Kumar Vavilapalli
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> Done. Ran trunk compilation and the new module's (small number of) YARN
>>> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
>>> is broken. And/or file tickets.
>>> 
>>> Thanks,
>>> +Vinod
>>> 
>>> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
>>> merging YARN-321 into trunk now.
>>> 
>>> Thanks everyone for voting!
>>> 
>>> +Vinod
>>> 
>>> On Jan 22, 2014, at 3:29 AM, Devaraj K <devaraj@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> 
>>> 
>>> Thanks
>>> Devaraj K
>>> 
>>> 
>>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <mayank@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> Thanks,
>>> Mayank
>>> 
>>> 
>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <acm@hortonworks.com>
>>> wrote:
>>> 
>>> +1 (binding)
>>> 
>>> Arun
>>> 
>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zshen@hortonworks.com> wrote:
>>> 
>>> Hi folks,
>>> 
>>> As previously discussed here (
>>> 
>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>> 
>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>> Application History Server into trunk.
>>> 
>>> *Scope of the changes*
>>> 
>>> 
>>> The changes enable ResourceManager to record the historic information
>>> 
>>> of
>>> 
>>> the application, the application attempt and the container in terms of
>>> events via a history writer. In addition, the changes setup up an
>>> application history server, which allows users to access the recorded
>>> information via RPC interface, web UI and REST APIs.
>>> 
>>> *Details of development*
>>> 
>>> 
>>> - Development of the feature is tracked in the jira -
>>> https://issues.apache.org/jira/browse/YARN-321.
>>> - Development has been done in a separate branch -
>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>> - The feature development involved about 35 subtasks.
>>> - The up-to-date design is posted at -
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>> Application
>>> History - Design-20131219.pdf<
>>> 
>>> 
>>> 
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>> 
>>> 
>>> 
>>> - The uber merge patch Jira -
>>> https://issues.apache.org/jira/browse/YARN-1587
>>> 
>>> *Testing*
>>> 
>>> 
>>> A number of unit tests have been added as a part of the feature. In
>>> addition, we've also done end-to-end functional tests, and performance
>>> tests for HDFS-based history storage and history events processing.
>>> 
>>> Last
>>> 
>>> but not least, we have updated branch YARN-321 against the latest
>>> 
>>> trunk,
>>> 
>>> edited merge conflicts, fixed test failures caused by merge, and
>>> 
>>> corrected
>>> 
>>> a bunch of bad source code issues. The uber merge patch that contains
>>> 
>>> all
>>> 
>>> the diff between branch YARN-321 and trunk has been run through
>>> 
>>> Jenkins.
>>> 
>>> 
>>> *Pending work*
>>> 
>>> 
>>> - Make it work in secure mode
>>> - Pending bug fixes
>>> 
>>> We wish to merge the branch now instead of waiting for later. The main
>>> reason for this is that as the branch grew in size, the cost of its
>>> maintenance became huge. Once the feature is merged into trunk, we will
>>> continue to work on pending work like security stuff, to test and fix
>>> 
>>> any
>>> 
>>> bugs that may be found on the trunk, and to refactor the code about to
>>> share some pieces in PRC and web interfaces.
>>> 
>>> *Release status*
>>> 
>>> 
>>> If the security stuff and the pending fixes arrive by the time
>>> 
>>> everything
>>> 
>>> else planned for Release 2.4 is done, we can include it as well. This
>>> 
>>> is
>>> 
>>> what we are striving for. Otherwise, we will call AHS
>>> 
>>> not-feature-complete
>>> 
>>> and not stable.
>>> 
>>> The bulk of the design and implementation was done by Mayank Bansal and
>>> 
>>> me
>>> 
>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>> 
>>> providing
>>> 
>>> feedback on the design discussions.
>>> 
>>> 
>>> 
>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>> 
>>> Thanks,
>>> Zhijie
>>> 
>>> --
>>> Zhijie Shen
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or
>>> 
>>> entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the
>>> 
>>> reader
>>> 
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>> 
>> 
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message