apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Support for Anti-Affinity in Apex
Date Wed, 24 Feb 2016 23:37:00 GMT
To explain the affinity support below seems to be a super-set that can
consume the stream locality setting whether that was the original intention
or not.

On Wed, Feb 24, 2016 at 3:35 PM, Pramod Immaneni <pramod@datatorrent.com>
wrote:

> Why would we want two different ways of specifying the same thing?
>
> On Wed, Feb 24, 2016 at 3:29 PM, Amol Kekre <amol@datatorrent.com> wrote:
>
>> Stream locality is a stream attribute. Affinity is operator attribute.
>> Technically they may not be connected. Though operator affinity can be
>> extended to specify a situation that implicity covers stream locality, it
>> may be better to let user say the two following statements independently.
>>
>> - This stream is heavy on I/O and I want it to be ..._local // This
>> statement is independent of future additions of operators to this stream
>> - I need these logical operators to not be (or be) on the same node
>>
>> Do note that container_local or node_local stream localities only relate
>> to
>> connected operators.
>>
>> Thks
>> Amol
>>
>>
>> On Wed, Feb 24, 2016 at 1:54 PM, Pramod Immaneni <pramod@datatorrent.com>
>> wrote:
>>
>> > Would it make sense to deprecate stream locality in favor of this?
>> >
>> > On Tue, Feb 23, 2016 at 6:05 PM, Isha Arkatkar <isha@datatorrent.com>
>> > wrote:
>> >
>> > > Hi Pramod,
>> > >
>> > >    We can have a list or operators or regex to specify operators list
>> for
>> > > affinity rule. Though, internal implementation will translate both
>> these
>> > > into list of pairs. Since, having pair of operators helps in
>> validation
>> > > phase.
>> > >
>> > >  And yes, validation phase will catch conflicting locality or affinity
>> > > rules and throw validation exception.
>> > >
>> > >  Thanks for input regarding unifiers. It is probably alright to not
>> > include
>> > > it in the affinity settings. Will leave the current implementation as
>> is.
>> > >
>> > > Thanks!
>> > > Isha
>> > >
>> > >
>> > > On Tue, Feb 23, 2016 at 5:56 PM, Pramod Immaneni <
>> pramod@datatorrent.com
>> > >
>> > > wrote:
>> > >
>> > > > Isha,
>> > > >
>> > > > If the implementation would support (eventually) affinity between
>> more
>> > > than
>> > > > 2 logical operators, instead of relying on specifying it as multiple
>> > > rules
>> > > > why not make provision in the API to specify more than 2 logical
>> > > operators?
>> > > > If there is a stream locality rule that conflicts with the affinity
>> > rule
>> > > > will the validation catch this? With unifiers the common case of MxN
>> > > > locality is already chosen based on downstream operator. In other
>> cases
>> > > > such as single downstream or cascade case I am not sure it is all
>> the
>> > > > important to have it.
>> > > >
>> > > > Thanks
>> > > >
>> > > > On Tue, Feb 23, 2016 at 5:37 PM, Isha Arkatkar <
>> isha@datatorrent.com>
>> > > > wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > >     I have opened a review only pull request for handling affinity
>> > > rules
>> > > > in
>> > > > > Apex:
>> > > > >    https://github.com/apache/incubator-apex-core/pull/234
>> > > > >
>> > > > >  Just wanted to confirm with everyone, if the APIs for specifying
>> > > > affinity
>> > > > > rules look alright. Also, wanted to get views on how unifiers
>> should
>> > be
>> > > > > handled in case of operator affinity rules. More details follow:
>> > > > >
>> > > > >  I have added a list of affinity rules as an attribute in Dag
>> > context.
>> > > > For
>> > > > > now, have added AffinityRule to be specified for operator pairs.
>> We
>> > can
>> > > > add
>> > > > > regex support in next iteration.
>> > > > >
>> > > > > Here is sample usage:
>> > > > >
>> > > > >    List<AffinityRule> rules = new ArrayList<>();
>> > > > >    // To add node locality between  2 not connected operators rand
>> > and
>> > > > > console:
>> > > > >    rules.add(new AffinityRule(Type.AFFINITY, new OperatorPair(
>> > "rand",
>> > > > > "console"), Locality.NODE_LOCAL, false));
>> > > > >
>> > > > >    // To add anti-affinity between all partitions of rand operator
>> > with
>> > > > all
>> > > > > partitions of passThru operator:
>> > > > >    rules.add(new AffinityRule(Type.ANTI_AFFINITY, new
>> OperatorPair(
>> > > > "rand",
>> > > > > "passThru"), Locality.NODE_LOCAL, false));
>> > > > >
>> > > > >    // To add anti-affinity between partitions of passThru
>> operator,
>> > > give
>> > > > > same operator name in pair:
>> > > > >     rules.add(new AffinityRule(Type.ANTI_AFFINITY, new
>> OperatorPair(
>> > > > > "passThru", "passThru"), Locality.NODE_LOCAL, false));
>> > > > >
>> > > > >   // Set the rules in dag context
>> > > > >   dag.setAttribute(DAGContext.AFFINITY_RULES, rules);
>> > > > >
>> > > > >
>> > > > > Please find a sample application using these affinity rules here:
>> > > > >
>> https://github.com/ishark/Apex-Samples/tree/master/affinity-example
>> > > > >
>> > > > >
>> > > > > The actual implementation of affinity rules heavily depends node
>> > > specific
>> > > > > requests, that was already being handled in
>> > StreamingAppMasterService.
>> > > To
>> > > > > handle node requests for cloudera, I have added an override to
>> > > blacklist
>> > > > > all other nodes except the ones where container request is to be
>> > > issued.
>> > > > >
>> > > > > There is one open question I wanted to bring up: In case of
>> > partitioned
>> > > > > operators, should the unifiers also follow affinity or
>> anti-affinity
>> > > > rules?
>> > > > > Or should they be independent. For now, I have kept them
>> independent
>> > > and
>> > > > > only the actual operators follow affinity rules. Please suggest
>> what
>> > > > would
>> > > > > make more sense from user's specification perspective.
>> > > > >
>> > > > > Thanks,
>> > > > > Isha
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Mon, Feb 1, 2016 at 2:12 PM, Isha Arkatkar <
>> isha@datatorrent.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi folks,
>> > > > > >
>> > > > > >    Summarizing the proposal for affinity/anti-affinity rules in
>> > apex
>> > > as
>> > > > > > per discussions in this mail thread. Please suggest if I missed
>> > > > > something.
>> > > > > >
>> > > > > > *  For configuration:*
>> > > > > >
>> > > > > >   - We will have application level affinity/anti-affinity rules.
>> > For
>> > > > the
>> > > > > > first iteration, we can support for (anti-)affinity among
>> operators
>> > > > > within
>> > > > > > single application. We can revisit it in the next iteration to
>> > > support
>> > > > > > across applications.
>> > > > > >
>> > > > > >  -  Each rule will consist of 4 components:
>> > > > > >    <List of operators>,   AFFINITY/ANTI-AFFINITY,
>>  STRICT/RELAXED
>> > > > > policy,
>> > > > > > CONTAINER/NODE/RACK
>> > > > > > I have checked that Apex supports container locality between 2
>> > > > operators
>> > > > > > that are not connected by Steam. Though did not find an API for
>> > > setting
>> > > > > it
>> > > > > > through application. So, affinity rules can cover container
>> > affinity
>> > > as
>> > > > > > well.
>> > > > > > We will not support THREAD locality in app level affinity or
>> > > > > > anti-affinity, since it does not apply to unconnected operators.
>> > > > > >
>> > > > > >  The list of operators can also be specified by regex
>> > > > > >
>> > > > > > *For Implementation: *
>> > > > > >
>> > > > > >   - I have verified that Node specific requests are honored by
>> > > > > HortonWorks
>> > > > > > for both Fair and Capacity scheduler. So we can  go with the
>> > approach
>> > > > of
>> > > > > >  node specific container requests to implement affinity rules.
>> > > > > >
>> > > > > >  - However, we may need to fall back on blacklist approach for
>> CDH
>> > > > > distro,
>> > > > > > as node specific requests may not work well. As:
>> > > > > >          -> With Fair scheduler, node specific requests are not
>> > > honored
>> > > > > in
>> > > > > > CDH 4. Application master keeps sending requests to Yarn, but no
>> > > > > containers
>> > > > > > are allocated.
>> > > > > >          -> I could not yet verify the same with Fair and
>> Capacity
>> > > > > > Scheduler on CDH 5. Ran into some configuration issues when
>> setting
>> > > up
>> > > > > CDH
>> > > > > > 5.5.1.  Will update the thread with findings.
>> > > > > >
>> > > > > >   I will try to build a prototype for affinity rules with these
>> > > things
>> > > > in
>> > > > > > mind. Please let me know if anyone would like to collaborate on
>> > this
>> > > > > > effort! :)
>> > > > > >
>> > > > > > Thanks!
>> > > > > > Isha
>> > > > > >
>> > > > > > On Mon, Jan 25, 2016 at 10:23 AM, Vlad Rozov <
>> > > v.rozov@datatorrent.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Yes, with ability to refer not only to a host name, but to
>> > operators
>> > > > > >> (possibly both in the same and other applications). In general
>> > this
>> > > > > should
>> > > > > >> allow to specify both affinity and anti affinity using
>> > expressions.
>> > > > For
>> > > > > >> example LOCALITY_CONTAINER = "{app1.op1 or app1.op2} and
>> > > {!app2.op1}".
>> > > > > >>
>> > > > > >>
>> > > > > >> On 1/25/16 09:39, Thomas Weise wrote:
>> > > > > >>
>> > > > > >>> Agreed. Originally I thought you wanted both operators on the
>> > same
>> > > > > host.
>> > > > > >>> To
>> > > > > >>> pin them to the same container the same idea can be applied,
>> > > through
>> > > > a
>> > > > > >>> "LOCALITY_CONTAINER" attribute.
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> On Mon, Jan 25, 2016 at 9:18 AM, Vlad Rozov <
>> > > v.rozov@datatorrent.com
>> > > > >
>> > > > > >>> wrote:
>> > > > > >>>
>> > > > > >>> Works as a workaround and requires two extra ports that
>> operators
>> > > > > >>>> designer
>> > > > > >>>> may not necessarily provide out of the box. We may apply the
>> > same
>> > > > hack
>> > > > > >>>> to
>> > > > > >>>> anti-affinity and request that any two operators with
>> > > anti-affinity
>> > > > > rule
>> > > > > >>>> are connected by non functional stream, but it does not look
>> > like
>> > > a
>> > > > > good
>> > > > > >>>> option :).
>> > > > > >>>>
>> > > > > >>>>
>> > > > > >>>>
>> > > > > >>>> On 1/25/16 08:52, Thomas Weise wrote:
>> > > > > >>>>
>> > > > > >>>> Sandeep suggested to connect them with a stream.
>> > > > > >>>>>
>> > > > > >>>>> On Mon, Jan 25, 2016 at 8:49 AM, Vlad Rozov <
>> > > > v.rozov@datatorrent.com
>> > > > > >
>> > > > > >>>>> wrote:
>> > > > > >>>>>
>> > > > > >>>>> Sorry, possibly I miss something. X and Z are not directly
>> > > > connected
>> > > > > >>>>> by a
>> > > > > >>>>>
>> > > > > >>>>>> stream, how can I apply THREAD_LOCAL?
>> > > > > >>>>>>
>> > > > > >>>>>>
>> > > > > >>>>>> On 1/25/16 08:44, Thomas Weise wrote:
>> > > > > >>>>>>
>> > > > > >>>>>> Why does THREAD_LOCAL not work?
>> > > > > >>>>>>
>> > > > > >>>>>>> On Mon, Jan 25, 2016 at 7:43 AM, Vlad Rozov <
>> > > > > v.rozov@datatorrent.com
>> > > > > >>>>>>> >
>> > > > > >>>>>>> wrote:
>> > > > > >>>>>>>
>> > > > > >>>>>>> Neither LOCALITY_HOST or THREAD_LOCAL provides required
>> > > > > >>>>>>> functionality.
>> > > > > >>>>>>> The
>> > > > > >>>>>>>
>> > > > > >>>>>>> goal is to deploy X and Z to the same container/JVM. X
>> may be
>> > > > > >>>>>>>> deployed
>> > > > > >>>>>>>> on
>> > > > > >>>>>>>> any node. Z needs to be deployed on the same container
>> as X,
>> > > so
>> > > > X
>> > > > > >>>>>>>> does
>> > > > > >>>>>>>> not
>> > > > > >>>>>>>> have any host affinity, and Z has affinity to X, not to
>> any
>> > > > > specific
>> > > > > >>>>>>>> host.
>> > > > > >>>>>>>> If database client caches writes, reading inside the same
>> > JVM
>> > > > will
>> > > > > >>>>>>>> be
>> > > > > >>>>>>>> in
>> > > > > >>>>>>>> process lookup, while deploying Z to any other host/JVM
>> > leads
>> > > to
>> > > > > >>>>>>>> inter
>> > > > > >>>>>>>> process or inter host lookup.
>> > > > > >>>>>>>>
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> On 1/24/16 21:03, Thomas Weise wrote:
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> There are the attributes LOCALITY_HOST and LOCALITY_RACK
>> for
>> > > an
>> > > > > >>>>>>>> operator
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> to
>> > > > > >>>>>>>>> achieve precisely what you are describing. The value
>> would
>> > be
>> > > > an
>> > > > > >>>>>>>>> alias
>> > > > > >>>>>>>>> that
>> > > > > >>>>>>>>> can be mapped to physical hosts for allocation. The same
>> > > could
>> > > > be
>> > > > > >>>>>>>>> used
>> > > > > >>>>>>>>> of
>> > > > > >>>>>>>>> anti-affinity, by assigning different values.
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> Stream locality is a special case of expressing affinity
>> > for
>> > > 2
>> > > > > >>>>>>>>> operators.
>> > > > > >>>>>>>>> Note that we also need a way to extend this to
>> partitions
>> > of
>> > > > the
>> > > > > >>>>>>>>> same
>> > > > > >>>>>>>>> operator.
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> On Sun, Jan 24, 2016 at 8:52 PM, Vlad Rozov <
>> > > > > >>>>>>>>> v.rozov@datatorrent.com>
>> > > > > >>>>>>>>> wrote:
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> It may be quite extreme use case, but suppose that
>> > operator X
>> > > > > >>>>>>>>> writes
>> > > > > >>>>>>>>> to
>> > > > > >>>>>>>>> a
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>> distributed in-memory database and operator Z reads from
>> > it.
>> > > Z
>> > > > is
>> > > > > >>>>>>>>> not
>> > > > > >>>>>>>>>
>> > > > > >>>>>>>>>> directly connected to X. In such cases it may be
>> necessary
>> > > to
>> > > > > let
>> > > > > >>>>>>>>>> an
>> > > > > >>>>>>>>>> application request container affinity (deploying to
>> the
>> > > same
>> > > > > JVM)
>> > > > > >>>>>>>>>> for
>> > > > > >>>>>>>>>> operators X and Z as writes may be cached and reads
>> from
>> > the
>> > > > > same
>> > > > > >>>>>>>>>> JVM
>> > > > > >>>>>>>>>> will
>> > > > > >>>>>>>>>> be potentially faster.
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> Technically the same may be applied to thread local on
>> > NUMA
>> > > > > >>>>>>>>>> boxes, as
>> > > > > >>>>>>>>>> NUMA
>> > > > > >>>>>>>>>> aware deployment of containers is not supported.
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> Vlad
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> On 1/23/16 18:01, Yogi Devendra wrote:
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> @Isha
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>> In my opinion:
>> > > > > >>>>>>>>>>
>> > > > > >>>>>>>>>>> THREAD_LOCAL, CONTAINER_LOCAL on stream is a special
>> case
>> > > of
>> > > > > >>>>>>>>>>> generic
>> > > > > >>>>>>>>>>> rules
>> > > > > >>>>>>>>>>> for Operator X and Operator Y.
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> We can say that, THREAD_LOCAL, CONTAINER_LOCAL would
>> be
>> > > > > >>>>>>>>>>> applicable
>> > > > > >>>>>>>>>>> only
>> > > > > >>>>>>>>>>> if
>> > > > > >>>>>>>>>>> operator X and Y are connected by stream. But, way to
>> > > express
>> > > > > >>>>>>>>>>> this
>> > > > > >>>>>>>>>>> should
>> > > > > >>>>>>>>>>> be similar to other rules for affinity.
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> ~ Yogi
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> On 24 January 2016 at 03:49, Isha Arkatkar <
>> > > > > isha@datatorrent.com
>> > > > > >>>>>>>>>>> >
>> > > > > >>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> Hey Chinmay,
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>>         I certainly agree on common set of rules for
>> > > > > configuring
>> > > > > >>>>>>>>>>>
>> > > > > >>>>>>>>>>> affinity!
>> > > > > >>>>>>>>>>>> Well
>> > > > > >>>>>>>>>>>> put by a concrete example. :)
>> > > > > >>>>>>>>>>>>         Only thing I would like to point is:
>> affinity of
>> > > > > >>>>>>>>>>>> operators
>> > > > > >>>>>>>>>>>> should
>> > > > > >>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>> cover thread and container locality. Since this is
>> only
>> > > > > >>>>>>>>>>>> applicable
>> > > > > >>>>>>>>>>>> when
>> > > > > >>>>>>>>>>>> operators are connected by stream. So, it makes
>> sense to
>> > > > have
>> > > > > >>>>>>>>>>>> it on
>> > > > > >>>>>>>>>>>> Stream
>> > > > > >>>>>>>>>>>> rather than in common configuration.
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>        And yes, DAG.validate should only check for
>> > > REQUIRED
>> > > > or
>> > > > > >>>>>>>>>>>> STRICT
>> > > > > >>>>>>>>>>>> policy. We
>> > > > > >>>>>>>>>>>> can agree on one of the terminologies STRICT/RELAXED
>> or
>> > > > > >>>>>>>>>>>> REQUIRED/PREFERRED.
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Thanks!
>> > > > > >>>>>>>>>>>> Isha
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> On Fri, Jan 22, 2016 at 8:38 PM, Chinmay Kolhatkar <
>> > > > > >>>>>>>>>>>> chinmay@datatorrent.com>
>> > > > > >>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Hi Isha, Bhupesh,
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> When I suggested singe affinity rule, I was mainly
>> > talking
>> > > > > about
>> > > > > >>>>>>>>>>>> "how
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> to"
>> > > > > >>>>>>>>>>>>> of configuration and not of implementation.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> I see locality is in a way suggesting an affinity of
>> > > > > operators.
>> > > > > >>>>>>>>>>>>> They're
>> > > > > >>>>>>>>>>>>> close terminologies.
>> > > > > >>>>>>>>>>>>> By configuring a locality on stream, we're also, in
>> a
>> > > way,
>> > > > > >>>>>>>>>>>>> defining
>> > > > > >>>>>>>>>>>>> affinity of operators.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Until now, only locality was there and hence was
>> > straight
>> > > > > >>>>>>>>>>>>> forward
>> > > > > >>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>> configuration for user.
>> > > > > >>>>>>>>>>>>> Tomorrow, when anti-affinity configuration comes up,
>> > one
>> > > > > might
>> > > > > >>>>>>>>>>>>> get
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> confused
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> on how to best use both locality & anti-affinity.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Hence suggested to make both (locality/affinity &
>> > > > > >>>>>>>>>>>> anti-affinity) as
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> a
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> part
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> of single configuration.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Suggestion is to have a more commonly adopted
>> > > configuration
>> > > > > >>>>>>>>>>>> which
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> admins
>> > > > > >>>>>>>>>>>>> and developer's are familiar with.
>> > > > > >>>>>>>>>>>>> Again referring to vShere Hypervisor's affinity
>> rules.
>> > I
>> > > > > think
>> > > > > >>>>>>>>>>>>> they
>> > > > > >>>>>>>>>>>>> have
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> a
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> single configuration which does both.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Having said that, here is a quick suggestion on how
>> both
>> > > can
>> > > > > be
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> achieved
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> a single configuration:
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> CATEGORY    TYPE           POLICY
>> ENTITIES
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Affinity             THREAD      REQUIRED
>> O1, O2
>> > > > > >>>>>>>>>>>>> //Meaning
>> > > > > >>>>>>>>>>>>> Operator1 & Operator2 should be thread local
>> > > > > >>>>>>>>>>>>> Affinity             NODE          PREFERRED
>>  O3, O4
>> > > > > >>>>>>>>>>>>> //Meaning
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> O3 &
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> O4 are preferred to be in node local
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> AntiAffinity       NODE          REQUIRED        O1,
>> O4
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> //Meaning
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> O1 &
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> O4 should not be on same node.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> AntiAffinity       RACK          PREFERRED     O2, O4
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> //Meaning
>> > > > > >>>>>>>>>>>>> O2
>> > > > > >>>>>>>>>>>>> &
>> > > > > >>>>>>>>>>>>> O4 are preferred not to be on same rack.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Linux setting affinity of CPUs for threads is
>> another
>> > way
>> > > > of
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> configuration
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> we can take a look at.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Learning from these commonly adopted configuration
>> > > pattern,
>> > > > we
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> should
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> come
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> up with best configuration suitable for distributed
>> > > > > >>>>>>>>>>>>> environment.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Idea here is to not have our own configuration and
>> give
>> > > > > >>>>>>>>>>>> something
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>> the users. Otherwise such an important concept might
>> > > > quickly
>> > > > > >>>>>>>>>>>>> get
>> > > > > >>>>>>>>>>>>> lost.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Regarding the DAG.validate I think we would need to
>> add
>> > > > some
>> > > > > >>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>> stuff
>> > > > > >>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>> take care of anti-affinity.
>> > > > > >>>>>>>>>>>>> Plus, anti-affinity/affinity should be validated at
>> > > > > >>>>>>>>>>>>> DAG.validate
>> > > > > >>>>>>>>>>>>> only
>> > > > > >>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>> the ones which are required.
>> > > > > >>>>>>>>>>>>> For preferred policies, validation in logical plan
>> > might
>> > > > be a
>> > > > > >>>>>>>>>>>>> early
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> check.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Thanks,
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>> Chinmay.
>> > > > > >>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> On Sat, Jan 23, 2016 at 3:15 AM, Isha Arkatkar <
>> > > > > >>>>>>>>>>>>> isha@datatorrent.com>
>> > > > > >>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Hi,
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>        Thanks for inputs! I like the idea of having
>> > > single
>> > > > > set
>> > > > > >>>>>>>>>>>>> of
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> (anti?)affinity rules at dag context level.
>> > > > > >>>>>>>>>>>>>>        There could still be conflicts based on
>> > Locality
>> > > > for
>> > > > > >>>>>>>>>>>>>> Streams
>> > > > > >>>>>>>>>>>>>> or
>> > > > > >>>>>>>>>>>>>> Node
>> > > > > >>>>>>>>>>>>>> locality attribute set for operators. But as
>> Sandeep
>> > > > > >>>>>>>>>>>>>> suggested, I
>> > > > > >>>>>>>>>>>>>> also
>> > > > > >>>>>>>>>>>>>> think Dag.validate should fail in case of
>> > contradicting
>> > > > > >>>>>>>>>>>>>> constraints.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>       We currently do not have 'affinity' among non
>> > > stream
>> > > > > >>>>>>>>>>>>>> operators, as
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> Bhupesh
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> pointed out. It is somewhat achievable by
>> requesting
>> > > node
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> locality
>> > > > > >>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> operators (Assuming node requests work as expected).
>> > But
>> > > > > >>>>>>>>>>>>> should we
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> consider
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> adding affinity specifications support as well
>> along
>> > > with
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> anti-affinity?
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>       Regarding specification of attributes from
>> > > > > dt-site.xml.
>> > > > > >>>>>>>>>>>>>> We
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>> go
>> > > > > >>>>>>>>>>>>> with
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> Json like string or even xml representation for
>> complex
>> > > > > >>>>>>>>>>>>> objects.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> What
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> is
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> our current behavior for setting Java object
>> > properties
>> > > > > >>>>>>>>>>>>>> through
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> XML?
>> > > > > >>>>>>>>>>>>> We
>> > > > > >>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> follow the same for this as well.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>        As for precedence or ability to satisfy
>> > > constraints:
>> > > > > >>>>>>>>>>>>> Right
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> now in
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> normal
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> scenario, if resources are not available for
>> > allocating
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> containers,
>> > > > > >>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> keep
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> sending to request till all are obtained.
>> Likewise, in
>> > > > case
>> > > > > of
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> strict
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> anti-affinity policy, we should keep the
>> application in
>> > > > > >>>>>>>>>>>>> ACCEPTED
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> state
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> till
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> the anti-affinity constraint is satisfied. For
>> relaxed
>> > > > > >>>>>>>>>>>>>> policy, we
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> decide timeout for relaxing the anti-affinity rule.
>> > > Please
>> > > > > note
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> this
>> > > > > >>>>>>>>>>>>>> applies only when we have non-contradicting rules.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> Thanks!
>> > > > > >>>>>>>>>>>>>> Isha
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 5:30 AM, Bhupesh Chawda <
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> bhupesh@datatorrent.com
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> I agree on having a single set of rules for the
>> > affinity
>> > > as
>> > > > > >>>>>>>>>>>>> well
>> > > > > >>>>>>>>>>>>> as
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> anti-affinity of operators / partitions on
>> containers.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> However, I noted the following points:
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         1. AFAIK, we do not support affinity
>> > (locality)
>> > > > in
>> > > > > a
>> > > > > >>>>>>>>>>>>>>> general
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> sense.
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> The
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         affinity is only for a stream, not for
>> *any*
>> > two
>> > > > > >>>>>>>>>>>>> operators.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> So,
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         should also look at the general case and
>> see
>> > > how
>> > > > it
>> > > > > >>>>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>>> be
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> supported,
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> if
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         there are valid use cases.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         2. Coming to anti-affinity, we cannot type
>> > cast
>> > > it
>> > > > > as
>> > > > > >>>>>>>>>>>>>> a
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> type of
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> affinity
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         rule. Saying "two operators must be on the
>> > same
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> container" is
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> very
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         different from saying "these two operators
>> > must
>> > > > not
>> > > > > >>>>>>>>>>>>>>> be on
>> > > > > >>>>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>> same
>> > > > > >>>>>>>>>>>>>>>         container". In this case, the second one
>> is a
>> > > > much
>> > > > > >>>>>>>>>>>>>>> more
>> > > > > >>>>>>>>>>>>>>> relaxed
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> rule
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> as
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         compared to the first one.
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>         3. Once we have this generic set of rules,
>> > there
>> > > > must
>> > > > > >>>>>>>>>>>>>> be a
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         "satisfiability" test run before
>> requesting
>> > > YARN
>> > > > > for
>> > > > > >>>>>>>>>>>>>>> containers.
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> If
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         request is not satisfiable, then there is
>> no
>> > > point
>> > > > > >>>>>>>>>>>>> asking
>> > > > > >>>>>>>>>>>>> YARN
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> allocate
>> > > > > >>>>>>>>>>>>>>>         containers in this manner. In case it is
>> not
>> > > > > >>>>>>>>>>>>>>> satisfiable, we
>> > > > > >>>>>>>>>>>>>>> must
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> also
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> have
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         a default order in which the rules can be
>> > > > "relaxed"
>> > > > > >>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> request
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> be
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> made
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>         satisfiable. For example, some very strict
>> > rules
>> > > > may
>> > > > > be
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> ignored,
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> or
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> made
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>         less constraining ( for example "on the
>> same
>> > > > > >>>>>>>>>>>>> container" =>
>> > > > > >>>>>>>>>>>>> "on
>> > > > > >>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> same
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>         node").
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> Thanks.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> -Bhupesh
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 2:54 PM, Aniruddha
>> Thombare <
>> > > > > >>>>>>>>>>>>>>> aniruddha@datatorrent.com> wrote:
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> +1 On Chinmay's suggestion about having single
>> set of
>> > > > > >>>>>>>>>>>>>>> affinity
>> > > > > >>>>>>>>>>>>>>> rules.
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Thanks,
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Aniruddha
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:57 PM, Sandeep
>> Deshmukh <
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> sandeep@datatorrent.com
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Between 2 operators, if one configures
>> > thread/container
>> > > > > local
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> anti-affinity as well, which one will take
>> affect?
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> The DAG validation step should error out in this
>> > > case.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> +1 on suggestion by  Chinmay to name it  "Affinity
>> > > Rules"
>> > > > > >>>>>>>>>>>>>>> than
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> anti-affinity. We are just extending our container
>> > > > > allocation
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> scheme
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> support containers not to be allocated together.
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> Regards,
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Sandeep
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:43 PM, Chinmay
>> Kolhatkar
>> > <
>> > > > > >>>>>>>>>>>>>>>>> chinmay@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> Hi Isha,
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> Couple of points:
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> 1. About the interface to configuring
>> > anti-affinity,
>> > > as
>> > > > > per
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> suggestion
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> above there are 2 different way to configure
>> > > locality
>> > > > > and
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> anti-affinity:
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> i.e. dag.setAttribute - for anti-affinity  &
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>       dag.addStream(...).setLocality for
>> locality.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> Between 2 operators, if one configures
>> > > thread/container
>> > > > > >>>>>>>>>>>>>>>>>> local
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> anti-affinity as well, which one will take
>> affect?
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> 2. Consider there could be such confusion as
>> above,
>> > > > would
>> > > > > >>>>>>>>>>>>>>>> it
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> make
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> sense
>> > > > > >>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> have a single API which takes care of both
>> > > anti-affinity
>> > > > > and
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> locality.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> This
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> way, one is configurable.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> 3. This point is coming from how VM affinity is
>> > > > configured
>> > > > > >>>>>>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> vSphere.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> The VMs are configured affinity are called as
>> > > > "affinity
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> rules"
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> "anti-affinity rules". Ultimately idea is to
>> > allocate
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> processing
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> nodes.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Via "VM-VM affinity rules", anti-affinity is also
>> > > > > configured.
>> > > > > >>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>> But
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> there
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> is
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> a single set of rule definition for both affinity
>> > > > (similar
>> > > > > to
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> locality
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> our case) and anti-affinity.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Would it be a better approach for configuring
>> > locality
>> > > > > rules
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> anti-affinity rules in a single rule and call
>> it
>> > > > > "affinity
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> rule".
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> Thanks,
>> > > > > >>>>>>>>>>>>>>> Chinmay.
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 12:24 PM, Yogi Devendra <
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> yogidevendra@apache.org
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> @Isha
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> I understand that anti-affinity across
>> application
>> > is
>> > > > not
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> straight-forward.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> It would be OK even if we do not have it in
>> > > iteration
>> > > > > 1.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> But, for attributes syntax; I still think that
>> > Java
>> > > > > object
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> should
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> be
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> avoided as they will be hard to configure from
>> > > > > >>>>>>>>>>>>>>>>> dt-site.xml or
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> other
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> config
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> files.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Other suggestion for this could be JSON
>> > representation
>> > > > of
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> String
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> array:
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> ["O2", "O3"].  (If operator names has some
>> special
>> > > > > >>>>>>>>>>>>>>>>> characters
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> like
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> "
>> > > > > >>>>>>>>>>>>>>>>> or [
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> or , those will be escaped in the JSON
>> > > representation.)
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Not sure if others agree on this; but attribute
>> > syntax
>> > > > > >>>>>>>>>>>>>>>>> should
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> be
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> finalized
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> in iteration 1 to avoid backward compatibility
>> > > issues
>> > > > > >>>>>>>>>>>>>>>>> later.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> ~ Yogi
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> On 22 January 2016 at 00:43, Thomas Weise <
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> Node based requests is the best approach - if
>> it
>> > > works
>> > > > > :-)
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> Blacklisting will require to allocate the
>> containers
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> sequentially.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> It
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> will
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> work, but slow down application startup,
>> especially
>> > > for
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> larger
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> topologies.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> On Thu, Jan 21, 2016 at 10:42 AM, Isha
>> Arkatkar <
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> isha@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> Hi,
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>        Should we consider the node based
>> requests
>> > if
>> > > > it
>> > > > > >>>>>>>>>>>>>>>>>>> works
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> with
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Capacity
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> Scheduler or avoid 2b approach altogether? I
>> > > checked
>> > > > > >>>>>>>>>>>>>>>>>>> that
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> node
>> > > > > >>>>>>>>>>>>>>>>> requests
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> do
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> not work with fair scheduler on CDH cluster.
>> Yarn
>> > > does
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> return
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> any
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> container if hostname is given in the container
>> > > > request.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> I
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> am
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> trying
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> setup a small virtual hortonworks cluster to
>> check
>> > the
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> this
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> behavior
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> on
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> that.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> YARN-2027 <
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/YARN-2027
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> mentioned
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> that
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> container requests are not honored in capacity
>> > > > scheduler
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> too.
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> But I
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> am
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>>>>>>> sure if it is because of distro dependent issue.
>> > > Please
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> share
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> insights.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> @Vlad, Adding support for regular expression
>> > > sounds
>> > > > > >>>>>>>>>>>>>>>>>>> good.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> We
>> > > > > >>>>>>>>>>>>>>>>> could
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> translate to list of operator names internally
>> > based
>> > > > on
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> regex.
>> > > > > >>>>>>>>>>>>>>>>> @Yogi,  I went with a list of strings for
>> attribute
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> because
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> "O2,
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> O3"
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> could
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> be a valid single operator name too :)
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> I am not sure of ways to implement anti-affinity
>> > > across
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> application.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Though
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> something to consider for later iteration.
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> Thanks,
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> Isha
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:59 PM, Thomas
>> Weise <
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> https://issues.apache.org/jira/browse/SLIDER-82
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:56 PM, Thomas Weise
>> <
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> The point was that containers are taken away
>> > from
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> other
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> apps
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> that
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> may
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> have
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> to discard work etc. It's not good style to
>> claim
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> resources
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> use
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> them eventually :-)
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> For this feature it is necessary to look at
>> the
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> scheduler
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> capabilities/semantics and limitations.
>> For
>> > > > > example,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> don't
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> bet
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> exclusively
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> on node requests if the goal is for it to work
>> > with
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> FairScheduler.
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> Also look at Slider, which just recently added
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> support
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> anti-affinity
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> (using node requests). When you run it on the
>> > CDH
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> cluster,
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> it
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> probably
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> won't work...
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 3:19 PM, Pramod
>> Immaneni <
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> pramod@datatorrent.com
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> Once released won't the containers be
>> > available
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> again
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> pool.
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> This
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> would only be optional and not mandatory.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> Thanks
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 2:02 PM, Thomas
>> Weise <
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> How about also supporting a minor
>> variation
>> > of
>> > > it
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> as
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> an
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> option
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> where it greedily gets the total number of
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> containers
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> and
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> discards
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> ones
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> it
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> can't use and repeats the process for the
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> remaining
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> till
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> everything
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> has
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> been allocated.
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> This is problematic as with resource preemption
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> these
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> containers
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> will
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> be
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> potentially taken away from other
>> applications
>> > and
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> then
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> thrown
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> away.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> Also does it make sense to support
>> > anti-cluster
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> affinity?
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> Thanks
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 1:21 PM, Isha
>> > > Arkatkar
>> > > > <
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> isha@datatorrent.com>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> Hi all,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>         We want add support for
>> > Anti-affinity
>> > > in
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Apex
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> allow
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> applications
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> launch specific physical operators on
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> different
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> nodes(APEXCORE-10
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> <
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > https://issues.apache.org/jira/browse/APEXCORE-10
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> ).
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> Want
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> request
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> your
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> suggestions/ideas for the same!
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>        The reasons for using anti-affinity
>> in
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> be:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> ensure
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> reliability, for performance reasons (such as
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> application
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> may
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> not
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> want
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> 2
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> i/o intensive operators to land on the same
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> node
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> improve
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> performance)
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> or
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> for some application specific constraints(for
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> example,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> 2
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> partitions
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> cannot
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> be run on the same node since they use same
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> port
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> number).
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> This
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> is
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> the
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> general rationale for adding Anti-affinity
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> support.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Since, Yarn does not support
>> anti-affinity
>> > yet
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> (YARN-1042
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> <
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > https://issues.apache.org/jira/browse/YARN-1042
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> ),
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> we
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> need
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> to
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> implement
>> > > > > >>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>> the logic in AM. Wanted to get your views on
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> following
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> aspects
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> this
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> implementation:
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> *1. How to specify anti-affinity for physical
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators/partitions
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> in
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> application:*
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>          One way for this is to have an
>> > > > attribute
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> for
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> setting
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> at
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> the logical operator context. And an
>> operator
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> can
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> set
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> this
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> attribute
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> with
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> list of operator names which should not be
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>> collocated.
>> > > > > >>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>           Consider dag with 3 operators:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>           TestOperator o1 =
>> > > > > dag.addOperator("O1",
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
>> > > > > >>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>           TestOperator o2 =
>> > dag.addOperator("O2",
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>           TestOperator o3 =
>> dag.addOperator("O3",
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>       To set anti-affinity for O1 operator:
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>          dag.setAttribute(o1,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> OperatorContext.ANTI_AFFINITY,
>> > > > > >>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>> new
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ArrayList<String>(Arrays.asList("O2",
>> > > "O3")));
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>           This would mean O1 should not
>> be
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> allocated
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> on
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>> nodes
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> containing
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> operators O2 and O3. This applies to all
>> > > > > >>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>> allocated
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> partitions
>> > > > > >>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>> of
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>> O1,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>> O2,
>> > > > > >>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>> O3.
>> > > > > >>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>         Also, if same operator name is
>> part
>> > of
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> list,
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message