hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlo Aldo Curino <carlo.cur...@gmail.com>
Subject Re: [VOTE] Merge YARN-6592 feature branch to trunk
Date Fri, 26 Jan 2018 17:37:22 GMT
+1

On Jan 26, 2018 9:28 AM, "Sunil G" <sunilg@apache.org> wrote:

> +1
>
> - Sunil
>
>
> On Fri, Jan 26, 2018 at 8:58 PM Arun Suresh <asuresh@apache.org> wrote:
>
>> Hello yarn-dev@
>>
>> Based on the positive feedback from the DISCUSS thread [1], I'd like to
>> start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
>> 5 days, and will end Jan 31 7:30AM PDT.
>>
>> This feature adds support for placing containers in YARN using rich
>> placement constraints. For example, this can be used by applications to
>> co-locate containers on a node or rack (*affinity *constraint), spread
>> containers across nodes or racks (*anti-affinity* constraint), or even
>> specify the maximum number of containers on a node/rack (*cardinality *
>> constraint).
>>
>> We have integrated this feature into the Distributed-Shell application
>> for feature testing. We have performed end-to-end testing on
>> moderately-sized clusters to verify that constraints work fine. Performance
>> tests have been done via both SLS tests and Capacity Scheduler performance
>> unit tests, and no regression was found. We have opened a JIRA to track
>> Jenkins acceptance of the aggregated patch [3]. Documentation is in the
>> process of being completed [4]. You can also check our design document for
>> more details [5].
>>
>> Config flags are needed to enable this feature and it should not have any
>> effect on YARN when turned off. Once merged, we plan to work on further
>> improvements, which can be tracked in the umbrella YARN-7812 [6].
>>
>> Kindly do take a look at the branch and raise issues/concerns that need
>> to be addressed before the merge.
>>
>> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
>> their contributions to this effort, as well as Subru, Chris, Carlo, and
>> Vinod for their inputs and discussions.
>>
>>
>> Cheers
>> -Arun
>>
>> [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/
>> 201801.mbox/%3CCAMreUaz%3DGnsjOLZ%3Dem2x%3DQS7qh27euCWNw6Bo_4Cu%
>> 2BfXnXhyNA%40mail.gmail.com%3E
>> [2] https://issues.apache.org/jira/browse/YARN-6592
>> [3] https://issues.apache.org/jira/browse/YARN-7792
>> [4] https://issues.apache.org/jira/browse/YARN-7780
>> [5] https://issues.apache.org/jira/secure/attachment/
>> 12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
>> [6] https://issues.apache.org/jira/browse/YARN-7812
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message