apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer
Date Mon, 20 Jun 2016 04:30:05 GMT

    [ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338980#comment-15338980
] 

ASF GitHub Bot commented on APEXCORE-222:
-----------------------------------------

Github user vrozov commented on a diff in the pull request:

    https://github.com/apache/apex-core/pull/350#discussion_r67635187
  
    --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java ---
    @@ -769,7 +769,11 @@ public void processHeartbeatResponse(ContainerHeartbeatResponse rsp)
         }
     
         if (rsp.committedWindowId != lastCommittedWindowId) {
    +
           lastCommittedWindowId = rsp.committedWindowId;
    +
    +      bufferServer.purge(lastCommittedWindowId);
    --- End diff --
    
    While currently buffer server by itself does not make any assumption regarding locality
and communicates with its clients (publisher, subscriber, controller) over network request
API, Apex supports deployment of the buffer server into the same container as the upstream
operator/publisher only. As the buffer server is not fault tolerant, and there is no plan
to make it fault tolerant, in reality, there is no flexibility to deploy buffer server into
another container other than the upstream operator container. Additionally, as long as the
buffer server continues to support purge request submitted over a network, the flexibility
to deploy buffer server in any container still can be exercised once (or if) fault tolerance
is supported by the buffer server.


> Delegate Buffer Server purge to StreamingContainer
> --------------------------------------------------
>
>                 Key: APEXCORE-222
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-222
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: Thomas Weise
>            Assignee: Sandesh
>
> Currently the purge requests are sent to the buffer servers from the app master. This
interaction exists parallel to the heartbeat protocol. Instead, the committed window ID that
is propagated through the heartbeat response can be used in StreamingContainer to initiate
the purge with the local buffer server, similar to how the committed callback on the operator
occurs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message