apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Isha Arkatkar <i...@datatorrent.com>
Subject Re: Support for Anti-Affinity in Apex
Date Wed, 24 Feb 2016 23:43:17 GMT
Yes that is correct.
If affinity rule is specified for 2 stream connected operators, that is
equivalent to stream locality.

In dag validation phase, we will treat whichever is lower locality bound as
the final stream locality. E.g. if Stream Locality is NODE_LOCAL and
affinity rule between the same 2 operators is set to be CONTAINER_LOCAL.
We'll set locality of that stream to be CONTAINER_LOCAL.

Thanks,
Isha



On Wed, Feb 24, 2016 at 3:37 PM, Pramod Immaneni <pramod@datatorrent.com>
wrote:

> To explain the affinity support below seems to be a super-set that can
> consume the stream locality setting whether that was the original intention
> or not.
>
> On Wed, Feb 24, 2016 at 3:35 PM, Pramod Immaneni <pramod@datatorrent.com>
> wrote:
>
> > Why would we want two different ways of specifying the same thing?
> >
> > On Wed, Feb 24, 2016 at 3:29 PM, Amol Kekre <amol@datatorrent.com>
> wrote:
> >
> >> Stream locality is a stream attribute. Affinity is operator attribute.
> >> Technically they may not be connected. Though operator affinity can be
> >> extended to specify a situation that implicity covers stream locality,
> it
> >> may be better to let user say the two following statements
> independently.
> >>
> >> - This stream is heavy on I/O and I want it to be ..._local // This
> >> statement is independent of future additions of operators to this stream
> >> - I need these logical operators to not be (or be) on the same node
> >>
> >> Do note that container_local or node_local stream localities only relate
> >> to
> >> connected operators.
> >>
> >> Thks
> >> Amol
> >>
> >>
> >> On Wed, Feb 24, 2016 at 1:54 PM, Pramod Immaneni <
> pramod@datatorrent.com>
> >> wrote:
> >>
> >> > Would it make sense to deprecate stream locality in favor of this?
> >> >
> >> > On Tue, Feb 23, 2016 at 6:05 PM, Isha Arkatkar <isha@datatorrent.com>
> >> > wrote:
> >> >
> >> > > Hi Pramod,
> >> > >
> >> > >    We can have a list or operators or regex to specify operators
> list
> >> for
> >> > > affinity rule. Though, internal implementation will translate both
> >> these
> >> > > into list of pairs. Since, having pair of operators helps in
> >> validation
> >> > > phase.
> >> > >
> >> > >  And yes, validation phase will catch conflicting locality or
> affinity
> >> > > rules and throw validation exception.
> >> > >
> >> > >  Thanks for input regarding unifiers. It is probably alright to not
> >> > include
> >> > > it in the affinity settings. Will leave the current implementation
> as
> >> is.
> >> > >
> >> > > Thanks!
> >> > > Isha
> >> > >
> >> > >
> >> > > On Tue, Feb 23, 2016 at 5:56 PM, Pramod Immaneni <
> >> pramod@datatorrent.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > Isha,
> >> > > >
> >> > > > If the implementation would support (eventually) affinity between
> >> more
> >> > > than
> >> > > > 2 logical operators, instead of relying on specifying it as
> multiple
> >> > > rules
> >> > > > why not make provision in the API to specify more than 2 logical
> >> > > operators?
> >> > > > If there is a stream locality rule that conflicts with the
> affinity
> >> > rule
> >> > > > will the validation catch this? With unifiers the common case of
> MxN
> >> > > > locality is already chosen based on downstream operator. In other
> >> cases
> >> > > > such as single downstream or cascade case I am not sure it is all
> >> the
> >> > > > important to have it.
> >> > > >
> >> > > > Thanks
> >> > > >
> >> > > > On Tue, Feb 23, 2016 at 5:37 PM, Isha Arkatkar <
> >> isha@datatorrent.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi all,
> >> > > > >
> >> > > > >     I have opened a review only pull request for handling
> affinity
> >> > > rules
> >> > > > in
> >> > > > > Apex:
> >> > > > >    https://github.com/apache/incubator-apex-core/pull/234
> >> > > > >
> >> > > > >  Just wanted to confirm with everyone, if the APIs for
> specifying
> >> > > > affinity
> >> > > > > rules look alright. Also, wanted to get views on how unifiers
> >> should
> >> > be
> >> > > > > handled in case of operator affinity rules. More details follow:
> >> > > > >
> >> > > > >  I have added a list of affinity rules as an attribute in Dag
> >> > context.
> >> > > > For
> >> > > > > now, have added AffinityRule to be specified for operator pairs.
> >> We
> >> > can
> >> > > > add
> >> > > > > regex support in next iteration.
> >> > > > >
> >> > > > > Here is sample usage:
> >> > > > >
> >> > > > >    List<AffinityRule> rules = new ArrayList<>();
> >> > > > >    // To add node locality between  2 not connected operators
> rand
> >> > and
> >> > > > > console:
> >> > > > >    rules.add(new AffinityRule(Type.AFFINITY, new OperatorPair(
> >> > "rand",
> >> > > > > "console"), Locality.NODE_LOCAL, false));
> >> > > > >
> >> > > > >    // To add anti-affinity between all partitions of rand
> operator
> >> > with
> >> > > > all
> >> > > > > partitions of passThru operator:
> >> > > > >    rules.add(new AffinityRule(Type.ANTI_AFFINITY, new
> >> OperatorPair(
> >> > > > "rand",
> >> > > > > "passThru"), Locality.NODE_LOCAL, false));
> >> > > > >
> >> > > > >    // To add anti-affinity between partitions of passThru
> >> operator,
> >> > > give
> >> > > > > same operator name in pair:
> >> > > > >     rules.add(new AffinityRule(Type.ANTI_AFFINITY, new
> >> OperatorPair(
> >> > > > > "passThru", "passThru"), Locality.NODE_LOCAL, false));
> >> > > > >
> >> > > > >   // Set the rules in dag context
> >> > > > >   dag.setAttribute(DAGContext.AFFINITY_RULES, rules);
> >> > > > >
> >> > > > >
> >> > > > > Please find a sample application using these affinity rules
> here:
> >> > > > >
> >> https://github.com/ishark/Apex-Samples/tree/master/affinity-example
> >> > > > >
> >> > > > >
> >> > > > > The actual implementation of affinity rules heavily depends node
> >> > > specific
> >> > > > > requests, that was already being handled in
> >> > StreamingAppMasterService.
> >> > > To
> >> > > > > handle node requests for cloudera, I have added an override to
> >> > > blacklist
> >> > > > > all other nodes except the ones where container request is to be
> >> > > issued.
> >> > > > >
> >> > > > > There is one open question I wanted to bring up: In case of
> >> > partitioned
> >> > > > > operators, should the unifiers also follow affinity or
> >> anti-affinity
> >> > > > rules?
> >> > > > > Or should they be independent. For now, I have kept them
> >> independent
> >> > > and
> >> > > > > only the actual operators follow affinity rules. Please suggest
> >> what
> >> > > > would
> >> > > > > make more sense from user's specification perspective.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Isha
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Mon, Feb 1, 2016 at 2:12 PM, Isha Arkatkar <
> >> isha@datatorrent.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi folks,
> >> > > > > >
> >> > > > > >    Summarizing the proposal for affinity/anti-affinity rules
> in
> >> > apex
> >> > > as
> >> > > > > > per discussions in this mail thread. Please suggest if I
> missed
> >> > > > > something.
> >> > > > > >
> >> > > > > > *  For configuration:*
> >> > > > > >
> >> > > > > >   - We will have application level affinity/anti-affinity
> rules.
> >> > For
> >> > > > the
> >> > > > > > first iteration, we can support for (anti-)affinity among
> >> operators
> >> > > > > within
> >> > > > > > single application. We can revisit it in the next iteration to
> >> > > support
> >> > > > > > across applications.
> >> > > > > >
> >> > > > > >  -  Each rule will consist of 4 components:
> >> > > > > >    <List of operators>,   AFFINITY/ANTI-AFFINITY,
> >>  STRICT/RELAXED
> >> > > > > policy,
> >> > > > > > CONTAINER/NODE/RACK
> >> > > > > > I have checked that Apex supports container locality between 2
> >> > > > operators
> >> > > > > > that are not connected by Steam. Though did not find an API
> for
> >> > > setting
> >> > > > > it
> >> > > > > > through application. So, affinity rules can cover container
> >> > affinity
> >> > > as
> >> > > > > > well.
> >> > > > > > We will not support THREAD locality in app level affinity or
> >> > > > > > anti-affinity, since it does not apply to unconnected
> operators.
> >> > > > > >
> >> > > > > >  The list of operators can also be specified by regex
> >> > > > > >
> >> > > > > > *For Implementation: *
> >> > > > > >
> >> > > > > >   - I have verified that Node specific requests are honored by
> >> > > > > HortonWorks
> >> > > > > > for both Fair and Capacity scheduler. So we can  go with the
> >> > approach
> >> > > > of
> >> > > > > >  node specific container requests to implement affinity rules.
> >> > > > > >
> >> > > > > >  - However, we may need to fall back on blacklist approach for
> >> CDH
> >> > > > > distro,
> >> > > > > > as node specific requests may not work well. As:
> >> > > > > >          -> With Fair scheduler, node specific requests are
> not
> >> > > honored
> >> > > > > in
> >> > > > > > CDH 4. Application master keeps sending requests to Yarn, but
> no
> >> > > > > containers
> >> > > > > > are allocated.
> >> > > > > >          -> I could not yet verify the same with Fair and
> >> Capacity
> >> > > > > > Scheduler on CDH 5. Ran into some configuration issues when
> >> setting
> >> > > up
> >> > > > > CDH
> >> > > > > > 5.5.1.  Will update the thread with findings.
> >> > > > > >
> >> > > > > >   I will try to build a prototype for affinity rules with
> these
> >> > > things
> >> > > > in
> >> > > > > > mind. Please let me know if anyone would like to collaborate
> on
> >> > this
> >> > > > > > effort! :)
> >> > > > > >
> >> > > > > > Thanks!
> >> > > > > > Isha
> >> > > > > >
> >> > > > > > On Mon, Jan 25, 2016 at 10:23 AM, Vlad Rozov <
> >> > > v.rozov@datatorrent.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> Yes, with ability to refer not only to a host name, but to
> >> > operators
> >> > > > > >> (possibly both in the same and other applications). In
> general
> >> > this
> >> > > > > should
> >> > > > > >> allow to specify both affinity and anti affinity using
> >> > expressions.
> >> > > > For
> >> > > > > >> example LOCALITY_CONTAINER = "{app1.op1 or app1.op2} and
> >> > > {!app2.op1}".
> >> > > > > >>
> >> > > > > >>
> >> > > > > >> On 1/25/16 09:39, Thomas Weise wrote:
> >> > > > > >>
> >> > > > > >>> Agreed. Originally I thought you wanted both operators on
> the
> >> > same
> >> > > > > host.
> >> > > > > >>> To
> >> > > > > >>> pin them to the same container the same idea can be applied,
> >> > > through
> >> > > > a
> >> > > > > >>> "LOCALITY_CONTAINER" attribute.
> >> > > > > >>>
> >> > > > > >>>
> >> > > > > >>> On Mon, Jan 25, 2016 at 9:18 AM, Vlad Rozov <
> >> > > v.rozov@datatorrent.com
> >> > > > >
> >> > > > > >>> wrote:
> >> > > > > >>>
> >> > > > > >>> Works as a workaround and requires two extra ports that
> >> operators
> >> > > > > >>>> designer
> >> > > > > >>>> may not necessarily provide out of the box. We may apply
> the
> >> > same
> >> > > > hack
> >> > > > > >>>> to
> >> > > > > >>>> anti-affinity and request that any two operators with
> >> > > anti-affinity
> >> > > > > rule
> >> > > > > >>>> are connected by non functional stream, but it does not
> look
> >> > like
> >> > > a
> >> > > > > good
> >> > > > > >>>> option :).
> >> > > > > >>>>
> >> > > > > >>>>
> >> > > > > >>>>
> >> > > > > >>>> On 1/25/16 08:52, Thomas Weise wrote:
> >> > > > > >>>>
> >> > > > > >>>> Sandeep suggested to connect them with a stream.
> >> > > > > >>>>>
> >> > > > > >>>>> On Mon, Jan 25, 2016 at 8:49 AM, Vlad Rozov <
> >> > > > v.rozov@datatorrent.com
> >> > > > > >
> >> > > > > >>>>> wrote:
> >> > > > > >>>>>
> >> > > > > >>>>> Sorry, possibly I miss something. X and Z are not directly
> >> > > > connected
> >> > > > > >>>>> by a
> >> > > > > >>>>>
> >> > > > > >>>>>> stream, how can I apply THREAD_LOCAL?
> >> > > > > >>>>>>
> >> > > > > >>>>>>
> >> > > > > >>>>>> On 1/25/16 08:44, Thomas Weise wrote:
> >> > > > > >>>>>>
> >> > > > > >>>>>> Why does THREAD_LOCAL not work?
> >> > > > > >>>>>>
> >> > > > > >>>>>>> On Mon, Jan 25, 2016 at 7:43 AM, Vlad Rozov <
> >> > > > > v.rozov@datatorrent.com
> >> > > > > >>>>>>> >
> >> > > > > >>>>>>> wrote:
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> Neither LOCALITY_HOST or THREAD_LOCAL provides required
> >> > > > > >>>>>>> functionality.
> >> > > > > >>>>>>> The
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> goal is to deploy X and Z to the same container/JVM. X
> >> may be
> >> > > > > >>>>>>>> deployed
> >> > > > > >>>>>>>> on
> >> > > > > >>>>>>>> any node. Z needs to be deployed on the same container
> >> as X,
> >> > > so
> >> > > > X
> >> > > > > >>>>>>>> does
> >> > > > > >>>>>>>> not
> >> > > > > >>>>>>>> have any host affinity, and Z has affinity to X, not to
> >> any
> >> > > > > specific
> >> > > > > >>>>>>>> host.
> >> > > > > >>>>>>>> If database client caches writes, reading inside the
> same
> >> > JVM
> >> > > > will
> >> > > > > >>>>>>>> be
> >> > > > > >>>>>>>> in
> >> > > > > >>>>>>>> process lookup, while deploying Z to any other host/JVM
> >> > leads
> >> > > to
> >> > > > > >>>>>>>> inter
> >> > > > > >>>>>>>> process or inter host lookup.
> >> > > > > >>>>>>>>
> >> > > > > >>>>>>>>
> >> > > > > >>>>>>>> On 1/24/16 21:03, Thomas Weise wrote:
> >> > > > > >>>>>>>>
> >> > > > > >>>>>>>> There are the attributes LOCALITY_HOST and
> LOCALITY_RACK
> >> for
> >> > > an
> >> > > > > >>>>>>>> operator
> >> > > > > >>>>>>>>
> >> > > > > >>>>>>>> to
> >> > > > > >>>>>>>>> achieve precisely what you are describing. The value
> >> would
> >> > be
> >> > > > an
> >> > > > > >>>>>>>>> alias
> >> > > > > >>>>>>>>> that
> >> > > > > >>>>>>>>> can be mapped to physical hosts for allocation. The
> same
> >> > > could
> >> > > > be
> >> > > > > >>>>>>>>> used
> >> > > > > >>>>>>>>> of
> >> > > > > >>>>>>>>> anti-affinity, by assigning different values.
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>> Stream locality is a special case of expressing
> affinity
> >> > for
> >> > > 2
> >> > > > > >>>>>>>>> operators.
> >> > > > > >>>>>>>>> Note that we also need a way to extend this to
> >> partitions
> >> > of
> >> > > > the
> >> > > > > >>>>>>>>> same
> >> > > > > >>>>>>>>> operator.
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>> On Sun, Jan 24, 2016 at 8:52 PM, Vlad Rozov <
> >> > > > > >>>>>>>>> v.rozov@datatorrent.com>
> >> > > > > >>>>>>>>> wrote:
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>> It may be quite extreme use case, but suppose that
> >> > operator X
> >> > > > > >>>>>>>>> writes
> >> > > > > >>>>>>>>> to
> >> > > > > >>>>>>>>> a
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>> distributed in-memory database and operator Z reads
> from
> >> > it.
> >> > > Z
> >> > > > is
> >> > > > > >>>>>>>>> not
> >> > > > > >>>>>>>>>
> >> > > > > >>>>>>>>>> directly connected to X. In such cases it may be
> >> necessary
> >> > > to
> >> > > > > let
> >> > > > > >>>>>>>>>> an
> >> > > > > >>>>>>>>>> application request container affinity (deploying to
> >> the
> >> > > same
> >> > > > > JVM)
> >> > > > > >>>>>>>>>> for
> >> > > > > >>>>>>>>>> operators X and Z as writes may be cached and reads
> >> from
> >> > the
> >> > > > > same
> >> > > > > >>>>>>>>>> JVM
> >> > > > > >>>>>>>>>> will
> >> > > > > >>>>>>>>>> be potentially faster.
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>> Technically the same may be applied to thread local
> on
> >> > NUMA
> >> > > > > >>>>>>>>>> boxes, as
> >> > > > > >>>>>>>>>> NUMA
> >> > > > > >>>>>>>>>> aware deployment of containers is not supported.
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>> Vlad
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>> On 1/23/16 18:01, Yogi Devendra wrote:
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>> @Isha
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>> In my opinion:
> >> > > > > >>>>>>>>>>
> >> > > > > >>>>>>>>>>> THREAD_LOCAL, CONTAINER_LOCAL on stream is a special
> >> case
> >> > > of
> >> > > > > >>>>>>>>>>> generic
> >> > > > > >>>>>>>>>>> rules
> >> > > > > >>>>>>>>>>> for Operator X and Operator Y.
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>> We can say that, THREAD_LOCAL, CONTAINER_LOCAL would
> >> be
> >> > > > > >>>>>>>>>>> applicable
> >> > > > > >>>>>>>>>>> only
> >> > > > > >>>>>>>>>>> if
> >> > > > > >>>>>>>>>>> operator X and Y are connected by stream. But, way
> to
> >> > > express
> >> > > > > >>>>>>>>>>> this
> >> > > > > >>>>>>>>>>> should
> >> > > > > >>>>>>>>>>> be similar to other rules for affinity.
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>> ~ Yogi
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>> On 24 January 2016 at 03:49, Isha Arkatkar <
> >> > > > > isha@datatorrent.com
> >> > > > > >>>>>>>>>>> >
> >> > > > > >>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>> Hey Chinmay,
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>>         I certainly agree on common set of rules for
> >> > > > > configuring
> >> > > > > >>>>>>>>>>>
> >> > > > > >>>>>>>>>>> affinity!
> >> > > > > >>>>>>>>>>>> Well
> >> > > > > >>>>>>>>>>>> put by a concrete example. :)
> >> > > > > >>>>>>>>>>>>         Only thing I would like to point is:
> >> affinity of
> >> > > > > >>>>>>>>>>>> operators
> >> > > > > >>>>>>>>>>>> should
> >> > > > > >>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>> cover thread and container locality. Since this is
> >> only
> >> > > > > >>>>>>>>>>>> applicable
> >> > > > > >>>>>>>>>>>> when
> >> > > > > >>>>>>>>>>>> operators are connected by stream. So, it makes
> >> sense to
> >> > > > have
> >> > > > > >>>>>>>>>>>> it on
> >> > > > > >>>>>>>>>>>> Stream
> >> > > > > >>>>>>>>>>>> rather than in common configuration.
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>        And yes, DAG.validate should only check for
> >> > > REQUIRED
> >> > > > or
> >> > > > > >>>>>>>>>>>> STRICT
> >> > > > > >>>>>>>>>>>> policy. We
> >> > > > > >>>>>>>>>>>> can agree on one of the terminologies
> STRICT/RELAXED
> >> or
> >> > > > > >>>>>>>>>>>> REQUIRED/PREFERRED.
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Thanks!
> >> > > > > >>>>>>>>>>>> Isha
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> On Fri, Jan 22, 2016 at 8:38 PM, Chinmay Kolhatkar
> <
> >> > > > > >>>>>>>>>>>> chinmay@datatorrent.com>
> >> > > > > >>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Hi Isha, Bhupesh,
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> When I suggested singe affinity rule, I was mainly
> >> > talking
> >> > > > > about
> >> > > > > >>>>>>>>>>>> "how
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> to"
> >> > > > > >>>>>>>>>>>>> of configuration and not of implementation.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> I see locality is in a way suggesting an affinity
> of
> >> > > > > operators.
> >> > > > > >>>>>>>>>>>>> They're
> >> > > > > >>>>>>>>>>>>> close terminologies.
> >> > > > > >>>>>>>>>>>>> By configuring a locality on stream, we're also,
> in
> >> a
> >> > > way,
> >> > > > > >>>>>>>>>>>>> defining
> >> > > > > >>>>>>>>>>>>> affinity of operators.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Until now, only locality was there and hence was
> >> > straight
> >> > > > > >>>>>>>>>>>>> forward
> >> > > > > >>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>> configuration for user.
> >> > > > > >>>>>>>>>>>>> Tomorrow, when anti-affinity configuration comes
> up,
> >> > one
> >> > > > > might
> >> > > > > >>>>>>>>>>>>> get
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> confused
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> on how to best use both locality & anti-affinity.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Hence suggested to make both (locality/affinity &
> >> > > > > >>>>>>>>>>>> anti-affinity) as
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> a
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> part
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> of single configuration.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Suggestion is to have a more commonly adopted
> >> > > configuration
> >> > > > > >>>>>>>>>>>> which
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> admins
> >> > > > > >>>>>>>>>>>>> and developer's are familiar with.
> >> > > > > >>>>>>>>>>>>> Again referring to vShere Hypervisor's affinity
> >> rules.
> >> > I
> >> > > > > think
> >> > > > > >>>>>>>>>>>>> they
> >> > > > > >>>>>>>>>>>>> have
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> a
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> single configuration which does both.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Having said that, here is a quick suggestion on how
> >> both
> >> > > can
> >> > > > > be
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> achieved
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> a single configuration:
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> CATEGORY    TYPE           POLICY
> >> ENTITIES
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Affinity             THREAD      REQUIRED
> >> O1, O2
> >> > > > > >>>>>>>>>>>>> //Meaning
> >> > > > > >>>>>>>>>>>>> Operator1 & Operator2 should be thread local
> >> > > > > >>>>>>>>>>>>> Affinity             NODE          PREFERRED
> >>  O3, O4
> >> > > > > >>>>>>>>>>>>> //Meaning
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> O3 &
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> O4 are preferred to be in node local
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> AntiAffinity       NODE          REQUIRED
> O1,
> >> O4
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> //Meaning
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> O1 &
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> O4 should not be on same node.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> AntiAffinity       RACK          PREFERRED     O2,
> O4
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> //Meaning
> >> > > > > >>>>>>>>>>>>> O2
> >> > > > > >>>>>>>>>>>>> &
> >> > > > > >>>>>>>>>>>>> O4 are preferred not to be on same rack.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Linux setting affinity of CPUs for threads is
> >> another
> >> > way
> >> > > > of
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> configuration
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> we can take a look at.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Learning from these commonly adopted configuration
> >> > > pattern,
> >> > > > we
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> should
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> come
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> up with best configuration suitable for
> distributed
> >> > > > > >>>>>>>>>>>>> environment.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Idea here is to not have our own configuration and
> >> give
> >> > > > > >>>>>>>>>>>> something
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>> the users. Otherwise such an important concept
> might
> >> > > > quickly
> >> > > > > >>>>>>>>>>>>> get
> >> > > > > >>>>>>>>>>>>> lost.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Regarding the DAG.validate I think we would need
> to
> >> add
> >> > > > some
> >> > > > > >>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>> stuff
> >> > > > > >>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>> take care of anti-affinity.
> >> > > > > >>>>>>>>>>>>> Plus, anti-affinity/affinity should be validated
> at
> >> > > > > >>>>>>>>>>>>> DAG.validate
> >> > > > > >>>>>>>>>>>>> only
> >> > > > > >>>>>>>>>>>>> for
> >> > > > > >>>>>>>>>>>>> the ones which are required.
> >> > > > > >>>>>>>>>>>>> For preferred policies, validation in logical plan
> >> > might
> >> > > > be a
> >> > > > > >>>>>>>>>>>>> early
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> check.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Thanks,
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>> Chinmay.
> >> > > > > >>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> On Sat, Jan 23, 2016 at 3:15 AM, Isha Arkatkar <
> >> > > > > >>>>>>>>>>>>> isha@datatorrent.com>
> >> > > > > >>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Hi,
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>        Thanks for inputs! I like the idea of
> having
> >> > > single
> >> > > > > set
> >> > > > > >>>>>>>>>>>>> of
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> (anti?)affinity rules at dag context level.
> >> > > > > >>>>>>>>>>>>>>        There could still be conflicts based on
> >> > Locality
> >> > > > for
> >> > > > > >>>>>>>>>>>>>> Streams
> >> > > > > >>>>>>>>>>>>>> or
> >> > > > > >>>>>>>>>>>>>> Node
> >> > > > > >>>>>>>>>>>>>> locality attribute set for operators. But as
> >> Sandeep
> >> > > > > >>>>>>>>>>>>>> suggested, I
> >> > > > > >>>>>>>>>>>>>> also
> >> > > > > >>>>>>>>>>>>>> think Dag.validate should fail in case of
> >> > contradicting
> >> > > > > >>>>>>>>>>>>>> constraints.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>       We currently do not have 'affinity' among
> non
> >> > > stream
> >> > > > > >>>>>>>>>>>>>> operators, as
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> Bhupesh
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> pointed out. It is somewhat achievable by
> >> requesting
> >> > > node
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> locality
> >> > > > > >>>>>>>>>>>>> for
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> operators (Assuming node requests work as
> expected).
> >> > But
> >> > > > > >>>>>>>>>>>>> should we
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> consider
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> adding affinity specifications support as well
> >> along
> >> > > with
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> anti-affinity?
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>       Regarding specification of attributes from
> >> > > > > dt-site.xml.
> >> > > > > >>>>>>>>>>>>>> We
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> can
> >> > > > > >>>>>>>>>>>>> go
> >> > > > > >>>>>>>>>>>>> with
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> Json like string or even xml representation for
> >> complex
> >> > > > > >>>>>>>>>>>>> objects.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> What
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> is
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> our current behavior for setting Java object
> >> > properties
> >> > > > > >>>>>>>>>>>>>> through
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> XML?
> >> > > > > >>>>>>>>>>>>> We
> >> > > > > >>>>>>>>>>>>> can
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> follow the same for this as well.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>        As for precedence or ability to satisfy
> >> > > constraints:
> >> > > > > >>>>>>>>>>>>> Right
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> now in
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> normal
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> scenario, if resources are not available for
> >> > allocating
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> containers,
> >> > > > > >>>>>>>>>>>>> we
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> keep
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> sending to request till all are obtained.
> >> Likewise, in
> >> > > > case
> >> > > > > of
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> strict
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> anti-affinity policy, we should keep the
> >> application in
> >> > > > > >>>>>>>>>>>>> ACCEPTED
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> state
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> till
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> the anti-affinity constraint is satisfied. For
> >> relaxed
> >> > > > > >>>>>>>>>>>>>> policy, we
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> can
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> decide timeout for relaxing the anti-affinity
> rule.
> >> > > Please
> >> > > > > note
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> this
> >> > > > > >>>>>>>>>>>>>> applies only when we have non-contradicting
> rules.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> Thanks!
> >> > > > > >>>>>>>>>>>>>> Isha
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 5:30 AM, Bhupesh Chawda <
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> bhupesh@datatorrent.com
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> I agree on having a single set of rules for the
> >> > affinity
> >> > > as
> >> > > > > >>>>>>>>>>>>> well
> >> > > > > >>>>>>>>>>>>> as
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> anti-affinity of operators / partitions on
> >> containers.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> However, I noted the following points:
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         1. AFAIK, we do not support affinity
> >> > (locality)
> >> > > > in
> >> > > > > a
> >> > > > > >>>>>>>>>>>>>>> general
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> sense.
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> The
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         affinity is only for a stream, not for
> >> *any*
> >> > two
> >> > > > > >>>>>>>>>>>>> operators.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> So,
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> we
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         should also look at the general case and
> >> see
> >> > > how
> >> > > > it
> >> > > > > >>>>>>>>>>>>>>> can
> >> > > > > >>>>>>>>>>>>>>> be
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> supported,
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> if
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         there are valid use cases.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         2. Coming to anti-affinity, we cannot
> type
> >> > cast
> >> > > it
> >> > > > > as
> >> > > > > >>>>>>>>>>>>>> a
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> type of
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> affinity
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         rule. Saying "two operators must be on
> the
> >> > same
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> container" is
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> very
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         different from saying "these two
> operators
> >> > must
> >> > > > not
> >> > > > > >>>>>>>>>>>>>>> be on
> >> > > > > >>>>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>> same
> >> > > > > >>>>>>>>>>>>>>>         container". In this case, the second one
> >> is a
> >> > > > much
> >> > > > > >>>>>>>>>>>>>>> more
> >> > > > > >>>>>>>>>>>>>>> relaxed
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> rule
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> as
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         compared to the first one.
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>         3. Once we have this generic set of rules,
> >> > there
> >> > > > must
> >> > > > > >>>>>>>>>>>>>> be a
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         "satisfiability" test run before
> >> requesting
> >> > > YARN
> >> > > > > for
> >> > > > > >>>>>>>>>>>>>>> containers.
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> If
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         request is not satisfiable, then there is
> >> no
> >> > > point
> >> > > > > >>>>>>>>>>>>> asking
> >> > > > > >>>>>>>>>>>>> YARN
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> allocate
> >> > > > > >>>>>>>>>>>>>>>         containers in this manner. In case it is
> >> not
> >> > > > > >>>>>>>>>>>>>>> satisfiable, we
> >> > > > > >>>>>>>>>>>>>>> must
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> also
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> have
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         a default order in which the rules can be
> >> > > > "relaxed"
> >> > > > > >>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> request
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> be
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> made
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>         satisfiable. For example, some very strict
> >> > rules
> >> > > > may
> >> > > > > be
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> ignored,
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> or
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> made
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>         less constraining ( for example "on the
> >> same
> >> > > > > >>>>>>>>>>>>> container" =>
> >> > > > > >>>>>>>>>>>>> "on
> >> > > > > >>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> same
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>         node").
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> Thanks.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> -Bhupesh
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 2:54 PM, Aniruddha
> >> Thombare <
> >> > > > > >>>>>>>>>>>>>>> aniruddha@datatorrent.com> wrote:
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> +1 On Chinmay's suggestion about having single
> >> set of
> >> > > > > >>>>>>>>>>>>>>> affinity
> >> > > > > >>>>>>>>>>>>>>> rules.
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Thanks,
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Aniruddha
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:57 PM, Sandeep
> >> Deshmukh <
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> sandeep@datatorrent.com
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Between 2 operators, if one configures
> >> > thread/container
> >> > > > > local
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> anti-affinity as well, which one will take
> >> affect?
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> The DAG validation step should error out in
> this
> >> > > case.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> +1 on suggestion by  Chinmay to name it
> "Affinity
> >> > > Rules"
> >> > > > > >>>>>>>>>>>>>>> than
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> anti-affinity. We are just extending our
> container
> >> > > > > allocation
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> scheme
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> support containers not to be allocated
> together.
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> Regards,
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Sandeep
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 1:43 PM, Chinmay
> >> Kolhatkar
> >> > <
> >> > > > > >>>>>>>>>>>>>>>>> chinmay@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> Hi Isha,
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> Couple of points:
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> 1. About the interface to configuring
> >> > anti-affinity,
> >> > > as
> >> > > > > per
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> suggestion
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> above there are 2 different way to configure
> >> > > locality
> >> > > > > and
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> anti-affinity:
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> i.e. dag.setAttribute - for anti-affinity  &
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>       dag.addStream(...).setLocality for
> >> locality.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> Between 2 operators, if one configures
> >> > > thread/container
> >> > > > > >>>>>>>>>>>>>>>>>> local
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> anti-affinity as well, which one will take
> >> affect?
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> 2. Consider there could be such confusion as
> >> above,
> >> > > > would
> >> > > > > >>>>>>>>>>>>>>>> it
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> make
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> sense
> >> > > > > >>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> have a single API which takes care of both
> >> > > anti-affinity
> >> > > > > and
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> locality.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> This
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> way, one is configurable.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> 3. This point is coming from how VM affinity is
> >> > > > configured
> >> > > > > >>>>>>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> vSphere.
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> The VMs are configured affinity are called as
> >> > > > "affinity
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> rules"
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> "anti-affinity rules". Ultimately idea is to
> >> > allocate
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> processing
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> nodes.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Via "VM-VM affinity rules", anti-affinity is
> also
> >> > > > > configured.
> >> > > > > >>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>> But
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> there
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> is
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> a single set of rule definition for both
> affinity
> >> > > > (similar
> >> > > > > to
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> locality
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> our case) and anti-affinity.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> Would it be a better approach for configuring
> >> > locality
> >> > > > > rules
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> anti-affinity rules in a single rule and call
> >> it
> >> > > > > "affinity
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> rule".
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> Thanks,
> >> > > > > >>>>>>>>>>>>>>> Chinmay.
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>> On Fri, Jan 22, 2016 at 12:24 PM, Yogi Devendra
> <
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> yogidevendra@apache.org
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> @Isha
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> I understand that anti-affinity across
> >> application
> >> > is
> >> > > > not
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> straight-forward.
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> It would be OK even if we do not have it in
> >> > > iteration
> >> > > > > 1.
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> But, for attributes syntax; I still think
> that
> >> > Java
> >> > > > > object
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> should
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> be
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> avoided as they will be hard to configure
> from
> >> > > > > >>>>>>>>>>>>>>>>> dt-site.xml or
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> other
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> config
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> files.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> Other suggestion for this could be JSON
> >> > representation
> >> > > > of
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> String
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> array:
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> ["O2", "O3"].  (If operator names has some
> >> special
> >> > > > > >>>>>>>>>>>>>>>>> characters
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> like
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> "
> >> > > > > >>>>>>>>>>>>>>>>> or [
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> or , those will be escaped in the JSON
> >> > > representation.)
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> Not sure if others agree on this; but attribute
> >> > syntax
> >> > > > > >>>>>>>>>>>>>>>>> should
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> be
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> finalized
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> in iteration 1 to avoid backward
> compatibility
> >> > > issues
> >> > > > > >>>>>>>>>>>>>>>>> later.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> ~ Yogi
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> On 22 January 2016 at 00:43, Thomas Weise <
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> Node based requests is the best approach - if
> >> it
> >> > > works
> >> > > > > :-)
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> Blacklisting will require to allocate the
> >> containers
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> sequentially.
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> It
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> will
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> work, but slow down application startup,
> >> especially
> >> > > for
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> larger
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> topologies.
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> On Thu, Jan 21, 2016 at 10:42 AM, Isha
> >> Arkatkar <
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> isha@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> Hi,
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>        Should we consider the node based
> >> requests
> >> > if
> >> > > > it
> >> > > > > >>>>>>>>>>>>>>>>>>> works
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> with
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> Capacity
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> Scheduler or avoid 2b approach altogether?
> I
> >> > > checked
> >> > > > > >>>>>>>>>>>>>>>>>>> that
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> node
> >> > > > > >>>>>>>>>>>>>>>>> requests
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> do
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> not work with fair scheduler on CDH cluster.
> >> Yarn
> >> > > does
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> return
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> any
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> container if hostname is given in the
> container
> >> > > > request.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> I
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> am
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> trying
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> setup a small virtual hortonworks cluster to
> >> check
> >> > the
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> this
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> behavior
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> on
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> that.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> YARN-2027 <
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>
> https://issues.apache.org/jira/browse/YARN-2027
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> mentioned
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> that
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> container requests are not honored in
> capacity
> >> > > > scheduler
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> too.
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> But I
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> am
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>>>>>>> sure if it is because of distro dependent
> issue.
> >> > > Please
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> share
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> insights.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> @Vlad, Adding support for regular
> expression
> >> > > sounds
> >> > > > > >>>>>>>>>>>>>>>>>>> good.
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> We
> >> > > > > >>>>>>>>>>>>>>>>> could
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> translate to list of operator names
> internally
> >> > based
> >> > > > on
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> regex.
> >> > > > > >>>>>>>>>>>>>>>>> @Yogi,  I went with a list of strings for
> >> attribute
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> because
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> "O2,
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> O3"
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> could
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> be a valid single operator name too :)
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> I am not sure of ways to implement
> anti-affinity
> >> > > across
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> application.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> Though
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> something to consider for later iteration.
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> Thanks,
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> Isha
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:59 PM, Thomas
> >> Weise <
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> https://issues.apache.org/jira/browse/SLIDER-82
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 8:56 PM, Thomas
> Weise
> >> <
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> The point was that containers are taken
> away
> >> > from
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> other
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> apps
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> that
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> may
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> have
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> to discard work etc. It's not good style to
> >> claim
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> resources
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> use
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> them eventually :-)
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> For this feature it is necessary to look at
> >> the
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> scheduler
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> capabilities/semantics and limitations.
> >> For
> >> > > > > example,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> don't
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> bet
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> exclusively
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> on node requests if the goal is for it to
> work
> >> > with
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> FairScheduler.
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> Also look at Slider, which just recently
> added
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> support
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> for
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> anti-affinity
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> (using node requests). When you run it on
> the
> >> > CDH
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> cluster,
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> it
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> probably
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> won't work...
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> On Wed, Jan 20, 2016 at 3:19 PM, Pramod
> >> Immaneni <
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> pramod@datatorrent.com
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> Once released won't the containers be
> >> > available
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> again
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> pool.
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> This
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> would only be optional and not mandatory.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> Thanks
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 2:02 PM, Thomas
> >> Weise <
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> thomas@datatorrent.com
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> How about also supporting a minor
> >> variation
> >> > of
> >> > > it
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> as
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> an
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> option
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> where it greedily gets the total number
> of
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> containers
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> and
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> discards
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> ones
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> it
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> can't use and repeats the process for the
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> remaining
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> till
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> everything
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> has
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> been allocated.
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> This is problematic as with resource
> preemption
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> these
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> containers
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> will
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> be
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> potentially taken away from other
> >> applications
> >> > and
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> then
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> thrown
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> away.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> Also does it make sense to support
> >> > anti-cluster
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> affinity?
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> Thanks
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jan 19, 2016 at 1:21 PM, Isha
> >> > > Arkatkar
> >> > > > <
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> isha@datatorrent.com>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> wrote:
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> Hi all,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>         We want add support for
> >> > Anti-affinity
> >> > > in
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> Apex
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> allow
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> applications
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> launch specific physical operators on
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> different
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> nodes(APEXCORE-10
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > https://issues.apache.org/jira/browse/APEXCORE-10
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> ).
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> Want
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> request
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> your
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> suggestions/ideas for the same!
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>        The reasons for using
> anti-affinity
> >> in
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> could
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> be:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> ensure
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> reliability, for performance reasons (such as
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> application
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> may
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> not
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> want
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> 2
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> i/o intensive operators to land on the
> same
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> node
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> improve
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> performance)
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> or
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> for some application specific constraints(for
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> example,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> 2
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> partitions
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> cannot
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> be run on the same node since they use
> same
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> port
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> number).
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> This
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> is
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> the
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> general rationale for adding Anti-affinity
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> support.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> Since, Yarn does not support
> >> anti-affinity
> >> > yet
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> (YARN-1042
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> <
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > https://issues.apache.org/jira/browse/YARN-1042
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> ),
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> we
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> need
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> to
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> implement
> >> > > > > >>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>> the logic in AM. Wanted to get your views on
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> following
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> aspects
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> implementation:
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> *1. How to specify anti-affinity for
> physical
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> operators/partitions
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> in
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> application:*
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>          One way for this is to have an
> >> > > > attribute
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> for
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> setting
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> at
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> the logical operator context. And an
> >> operator
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> can
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> set
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>> this
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> attribute
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> with
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> list of operator names which should not be
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>> collocated.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>           Consider dag with 3 operators:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>           TestOperator o1 =
> >> > > > > dag.addOperator("O1",
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
> >> > > > > >>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>           TestOperator o2 =
> >> > dag.addOperator("O2",
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>           TestOperator o3 =
> >> dag.addOperator("O3",
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> TestOperator());
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>       To set anti-affinity for O1 operator:
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>          dag.setAttribute(o1,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> OperatorContext.ANTI_AFFINITY,
> >> > > > > >>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>> new
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> ArrayList<String>(Arrays.asList("O2",
> >> > > "O3")));
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>           This would mean O1 should not
> >> be
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> allocated
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> on
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>> nodes
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> containing
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> operators O2 and O3. This applies to all
> >> > > > > >>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>> allocated
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> partitions
> >> > > > > >>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>> of
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>> O1,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>> O2,
> >> > > > > >>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>> O3.
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>         Also, if same operator name is
> >> part
> >> > of
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>> anti-affinity
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>> list,
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>>>>>>>>>>>>>>>>>>>>>>>>>
> >> > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message