apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)
Date Thu, 07 Apr 2016 23:30:25 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231331#comment-15231331
] 

ASF GitHub Bot commented on APEXCORE-10:
----------------------------------------

Github user ishark commented on a diff in the pull request:

    https://github.com/apache/incubator-apex-core/pull/250#discussion_r58961724
  
    --- Diff: engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
    @@ -344,9 +346,67 @@ public PhysicalPlan(LogicalPlan dag, PlanContext ctx) {
             addLogicalOperator(n);
           }
         }
    +    
    +    inlinePrefs.prefs.clear();
    +    localityPrefs.prefs.clear();
    +    
    +    // Add inlinePrefs and localityPreds for affinity rules
    --- End diff --
    
    Great point. Did not handle it earlier. 
    Removed clearing of inlinePrefs and localityPrefs. These preference are applied again
on dynamic plan changes. So affinity will be set again on dynamic changes. As for anti-affinity,
added population of antiAffinityPreferences for containers on re-allocation.
    
    Added a test case for the same.


> Enable non-affinity of operators per node (not containers)
> ----------------------------------------------------------
>
>                 Key: APEXCORE-10
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-10
>             Project: Apache Apex Core
>          Issue Type: Task
>            Reporter: Amol Kekre
>            Assignee: Isha Arkatkar
>              Labels: roadmap
>
> The issue happens on cloud which provides virtual cores with software like Xen underneath.
In effect if CPU intensive operators land up on same node we have a resource bottleneck,
> Need to create an attribute that does the following
> - Operators A & B should not be on same node
> - Stram should use this attribute to try to get containers on different node
> It is understood that the user is making an explicit choice to use NIC instead of stream
local optimization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message