incubator-mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qinghe Jin (JIRA)" <>
Subject [jira] [Commented] (MESOS-204) enable Frameworks level disk IO bandwidth support
Date Tue, 05 Jun 2012 07:55:24 GMT


Qinghe Jin commented on MESOS-204:

Thanks Charles Reiss very much for your suggestion!

Cgroups blkio controller seems to be more powerfully than ioprio_set, and I will consider
to use it. Yes,OOM is really nasty,but you said that Ben had started working on it, so
I'd like to add the IO support first :D

Does the network really similarly to disk ? In my opinion, network prioitization is more difficult
than disk IO because  the network application is point to point and it’s related to the
network topology and the traffic state of the network,which is beyond the ability of single
node. From the view of single node, we probably could set the network bandwidth or priority,
but could we insure that we could really achieve that for application? After all, the node
know more about the disk than the network, right?

Frankly speaking, I know little about HDFS's priority inversion, how does it happened in the
real world? Does the os could do the priority inheritance to avoid this?

At the beginning, I suppose the IO bandwidth control is specially for IO bound applications
with the same priority, and haven't taken priority inversion into account. If it really happens,
I will pay more attention to it.

> enable Frameworks level disk IO bandwidth support
> -------------------------------------------------
>                 Key: MESOS-204
>                 URL:
>             Project: Mesos
>          Issue Type: Brainstorming
>            Reporter: Qinghe Jin
> I am considering to add Frameworks level disk IO bandwidth support to mesos,which behaves
like ionice on the single node,but this time it‘s in multi-nodes。I believe this kind
of disk QoS support will be more user friendly。My initial idea is to allow user to use an
ioprio_set like api to set the io priority when they commit their job. At the same time, here
may need some tools to display the disk IO usage per node/container on the webui.
> I have read the mesos paper carefully and I found that in the comment,it mentioned
that in the future it will support disk and network bandwidth control。But,in my opinion,disk
IO bandwidth is more feasible for it has more locality,and network bandwidth control may
be more difficult for the complexity of the network environment。But I don't know why is
there nobody going to add this feature。Is it useless?Or not feasible?Or just not interested?
> For this feature may need a lot of work,I‘d like to listen to your opinions before
I start to work on it。Anybody who have any questions ,suggestions or ideas, please tell
me and I appreciate it very much,Thanks!

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message