mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Mahler (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MESOS-6941) Add support for batch processing of status updates, to increase latency / throughput / cluster scalability.
Date Wed, 18 Jan 2017 02:02:26 GMT
Benjamin Mahler created MESOS-6941:
--------------------------------------

             Summary: Add support for batch processing of status updates, to increase latency
/ throughput / cluster scalability.
                 Key: MESOS-6941
                 URL: https://issues.apache.org/jira/browse/MESOS-6941
             Project: Mesos
          Issue Type: Improvement
          Components: agent, framework api, master
            Reporter: Benjamin Mahler


Currently, each task has its own independent status update stream. Within an individual stream,
updates are sent to schedulers in a serial fashion: the agent will send the N+1th status update
only after it receives the scheduler's acknowledgement for the Nth status update.

This approach slows down throughput substantially and has the potential to backlog status
updates when they occur rapidly. Rather, we should add the ability for all available updates
on a stream to be sent together (there should probably be a limit to the size of this "batch")
so that the scheduler can process them together without incurring the round trip acknowledgement
latency in between each update.

In addition, there may be cases of updates where the scheduler only wants the latest information
(e.g. download status per MESOS-2256, or possibly health information). But this should be
tackled separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message