From yarn-issues-return-136422-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Mon Jan 29 20:46:05 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 09332180658 for ; Mon, 29 Jan 2018 20:46:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id ECF90160C2F; Mon, 29 Jan 2018 19:46:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 44D28160C31 for ; Mon, 29 Jan 2018 20:46:04 +0100 (CET) Received: (qmail 31004 invoked by uid 500); 29 Jan 2018 19:46:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 30991 invoked by uid 99); 29 Jan 2018 19:46:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Jan 2018 19:46:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D0AE6CDCAA for ; Mon, 29 Jan 2018 19:46:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id PO5uZB4J5t_G for ; Mon, 29 Jan 2018 19:46:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id B79C35F23D for ; Mon, 29 Jan 2018 19:46:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id C1610E01BE for ; Mon, 29 Jan 2018 19:46:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2D09224102 for ; Mon, 29 Jan 2018 19:46:00 +0000 (UTC) Date: Mon, 29 Jan 2018 19:46:00 +0000 (UTC) From: "Konstantinos Karanasos (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7822) Constraint satisfaction checker support for composite OR and AND constraints MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7822?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16343= 897#comment-16343897 ]=20 Konstantinos Karanasos commented on YARN-7822: ---------------------------------------------- Patch looks good, thanks [~cheersyang]. I have only one ask, which I don't think is a big change. Can we support DN= F? This is just ORs of ANDs. This means that we can allow the following: * AND constraints with no nesting (your patch does that already); * OR constraints, each being either a simple constraint or an AND constrai= nt with no nesting. So essentially, you only have to allow this additional = case in your patch. If we do this, then we can transform any compound constraint with any nesti= ng level to an equivalent DNF form (we can tackle this in a separate JIRA),= and we will be done with the canSatisfyConstraint for all possible constra= ints. =C2=A0 Regarding the max cardinality, you are right it is a bit confusing in the d= esign doc, but it is not a code implementation artifact. The idea is that the constraint has to be satisfied the moment of schedulin= g: * When the source and target tags are different, this happens to be the sa= me both before and after the placement.=C2=A0Assume a constraint=C2=A0that = hb-m should be placed at a node with at most 2 hb-rs.=C2=A0Say node n1 has = before scheduling 2 hb-rs, so we can place hb-m there. After placement, it = will still have 2 hb-rs. * Now when the source and target tags are the same, it is more complicated= . Assume a constraint that hb-rs should be placed at a node with at most 2 = hb-rs. Say node n1 has again 2 hb-rs before scheduling. After scheduling, i= t will have 3. So to unify the two cases above, we say that=C2=A0the constraints should be= satisfied before the placement of the new container happens. As [~asuresh]= mentioned, at some point we tried to create a special case for when source= =3D=3Dtarget, but it the semantics were not straightforward (e.g., if a use= r was saying max cardinality 2, they were expecting 2 after placement; when= a user says max cardinality 0, they think anti-affinity, but it actually h= as to be 1 if we consider cardinality after placement). To unify things, we= opted for cardinality check before placement in all cases. =C2=A0 > Constraint satisfaction checker support for composite OR and AND constrai= nts > -------------------------------------------------------------------------= --- > > Key: YARN-7822 > URL: https://issues.apache.org/jira/browse/YARN-7822 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Arun Suresh > Assignee: Weiwei Yang > Priority: Major > Attachments: YARN-7822-YARN-6592.001.patch, YARN-7822-YARN-6592.0= 02.patch, YARN-7822-YARN-6592.003.patch, YARN-7822-YARN-6592.004.patch > > > JIRA to track changes to {{PlacementConstraintsUtil#canSatisfyConstraints= }} handle OR and AND Composite constaints -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org