apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: anti-affinity - parameter to control the containers of an operator on a node
Date Fri, 12 Aug 2016 19:16:24 GMT
Eventually in long run Yarn (or other distributed OS) should handle
isolation for all resources. Currently I/O is not isolated by Yarn, and
will need Apex to help out for a year or two. Secondly if Apex works on
another OS, we will also need to check if isolation is available for CPU,
Memory. Overall IMHO this feature though temporary, may exist longer than
we anticipated.

Thks
Amol


On Fri, Aug 12, 2016 at 12:04 PM, Venkatesh Kottapalli <
venkatesh@datatorrent.com> wrote:

> Yes Thomas.
>
> YARN allocates based on “availability” and it also depends on the other
> jobs running in the cluster.
>
> As an example, in a 20 node cluster with 64 partitions of Operator A, I
> want to set a rule saying - not more than 4 containers of Operator A should
> be deployed on the same node because it is processing intensive. Is this
> possible with the current versions?
>
> It is possible that there will be other jobs running in the cluster and 6
> to 7 containers of the operator A get deployed on the same node leading to
> a CPU utilization of 90% on that node while other nodes have cpu
> utilization less than 20 which is not optimal. Instead the job can
> wait/preempt for the resources to get allocated as per the configuration
> above.
>
> I think it is good to provide handle to the users to configure such things
> as they might decide depending on the job priorities. Let me know if I am
> missing something here.
>
> -Venkatesh.
>
>
> > On Aug 12, 2016, at 11:15 AM, Thomas Weise <thomas@datatorrent.com>
> wrote:
> >
> > Venky,
> >
> > Please think about this in terms of resources ("capable of handling").
> YARN
> > controls memory and CPU and you define how much memory and CPU your
> > operator needs. The scheduler uses that information to find a suitable
> node
> > and it is already clear how many containers of an operator fit on a node.
> >
> > Maybe you are thinking of another resource that is not managed by YARN?
> >
> > Thomas
> >
> > On Wed, Aug 10, 2016 at 11:27 AM, Venkatesh Kottapalli <
> > venkatesh@datatorrent.com> wrote:
> >
> >> Hi team,
> >>
> >> On the anti-affinity rules while deploying containers, do we have a
> >> feature which can control the number of containers of the same operator
> >> that get deployed on the same node. If the environment is capable of
> >> handling, then this will be a good feature to have as it is possible
> that
> >> certain operators could be resource hungry and this would distribute the
> >> load uniformly on all the nodes.
> >>
> >> Please share your thoughts on this.
> >>
> >> -Venkatesh.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message