apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)
Date Sat, 26 Mar 2016 01:42:25 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212708#comment-15212708
] 

ASF GitHub Bot commented on APEXCORE-10:
----------------------------------------

Github user ishark commented on a diff in the pull request:

    https://github.com/apache/incubator-apex-core/pull/250#discussion_r57506922
  
    --- Diff: api/src/main/java/com/datatorrent/api/DAG.java ---
    @@ -250,6 +250,24 @@
       public abstract <T> void setAttribute(Operator operator, Attribute<T> key,
T value);
     
       /**
    +   * Set affinity between operators
    +   * @param locality
    +   * @param relaxLocality
    +   * @param first operator
    +   * @param one or more operators
    +   */
    +  public abstract void setAffinity(Locality locality, boolean relaxLocality, String firstOperator,
String... operators);
    --- End diff --
    
    Yes, the affinity/anti-affinity can be set from attributes directly. These were added
as convenience methods as per discussion on dev mailing list. Here are sample usages with
these methods:
    https://github.com/ishark/Apex-Samples/blob/master/affinity-example/src/main/java/com/example/affinity/ApplicationDagAPIs.java#L33-L34
    
    Compared to when setting via attributes:
    https://github.com/ishark/Apex-Samples/blob/master/affinity-example/src/main/java/com/example/affinity/ApplicationDagContext.java#L42-L54


> Enable non-affinity of operators per node (not containers)
> ----------------------------------------------------------
>
>                 Key: APEXCORE-10
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-10
>             Project: Apache Apex Core
>          Issue Type: Task
>            Reporter: Amol Kekre
>            Assignee: Isha Arkatkar
>              Labels: roadmap
>
> The issue happens on cloud which provides virtual cores with software like Xen underneath.
In effect if CPU intensive operators land up on same node we have a resource bottleneck,
> Need to create an attribute that does the following
> - Operators A & B should not be on same node
> - Stram should use this attribute to try to get containers on different node
> It is understood that the user is making an explicit choice to use NIC instead of stream
local optimization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message