nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Bean <mark.o.b...@gmail.com>
Subject Re: Cluster using Site-to-Site
Date Mon, 17 Apr 2017 13:21:47 GMT
Joe,

I re-ran the test today, and site-to-site is spreading the files across the
cluster no matter which host I configure in the Remote Process Group
(coordinator or non-coordinator.) It is not always a balanced distribution,
but I'm sure there are several factors considered when the RPG selects the
target. Either way,it is working as expected. My first test must not have
run with sufficient sample set.

Thanks,
Mark


On Sat, Apr 15, 2017 at 6:46 AM, Joe Witt <joe.witt@gmail.com> wrote:

> Mark
>
> You can point the RPG at any node in the other cluster.  Did that not work
> for you?
>
> Thanks
> Joe
>
> On Apr 14, 2017 1:57 PM, "Mark Bean" <mark.o.bean@gmail.com> wrote:
>
> > In 0.x, one would point a Remote Process Group (RPG) at the NCM. The NCM
> > would then take care of load balancing to the corresponding Input Port on
> > the Nodes. In 1.x, the same behavior is true so long as the RPG points to
> > the Cluster Coordinator. (I believe this is true at least.)
> >
> > Here's the rub. Since the Coordinator can be reassigned, the RPG can lose
> > touch with the it. Assume the Coordinator goes down temporarily. Further
> > assume the Coordinator Node returns to the Cluster, but is no longer
> > designated as the Coordinator because a new Coordinator was elected
> during
> > the downtime. The RPG will continue to function (i.e. send FlowFiles),
> but
> > only to the one Node defined by the RPG. Load balancing across the
> Cluster
> > has been lost.
> >
> > Is this an accurate description of what will occur in this scenario? Are
> > there recommendations on how to prevent this? In other words, how does
> one
> > ensure Cluster load balancing is maintained?
> >
> > Thanks,
> > Mark
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message