brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Corbett <>
Subject Re: [DISCUSSION] Determining which Brooklyn node address to advertise
Date Wed, 15 Feb 2017 14:34:53 GMT
Thanks for clarifying. It was this cross-cloud case I was considering.

What is the benefit of the "management-private-<cloud id>" group? It 
seems superfluous to me if "management-private-ingress-<cloud id>" will 
contain the public IPs of the members of the cluster.

On 15/02/2017 14:16, Svetoslav Neykov wrote:
> My response below was based on my thinking about in-cloud access between Brooklyn and
the managed machines.
> When the access is cross-cloud we indeed need _c_ security groups. Each one containing
_n_ records with the public IPs of the HA cluster members. These can be managed by Brooklyn
> To summarise (and get my thinking straight). In each cloud/availability zone we could
>      1. "management-private-<cloud id>" SG assigned on the HA cluster member machines.
No records in the SG.
>      2. "management-private-ingress-<cloud id>" SG with first record allowing all
traffic from "management-private-<cloud id>" (above group) and _n_ more records allowing
all traffic coming from the public IPs of the HA member nodes. This one is assigned to all
managed entities.
> On adding a new member to the cluster:
>    1. Assign "management-private-<cloud id>" to the new machine - this is the only
action that needs to be done by the manager of the cluster
>    2. Go to each cloud and update the "management-private-ingress-<cloud id>" group,
adding the new public IP.
> Svet.
>> On 15.02.2017 г., at 15:40, Svetoslav Neykov <>
>>> You propose that the manager of the Brooklyn HA cluster maintain at least _c_
security groups (one per security group scope - e.g. AWS EC2 region - per cloud).
>> Not quite. It's the number of "clouds HA cluster is deployed to". That's the number
of availability zones - usually a low number 1-2-3.
>>> Each of these groups has _n_ records. When the HA cluster is resized each group
is modified to add or remove a record as appropriate.
>> No, they are empty. Adding a new cluster member is just assigning the SG to the new
machine. Then for each managed entity there's a record in its security group to allow traffic
from the HA security group.
>> For clouds where there are no running HA cluster members Brooklyn will auto-create
the security group and add it to entities' security groups but it will be unused.
>> The manager of the HA cluster can wait for Brooklyn to create the "management" SG
(which will contain the cluster ID in its name which is not known in advance) and then assign
it to the HA cluster members or create one with a predefined name and configure the name in
each HA instance.
>> Svet.
>>> On 15.02.2017 г., at 14:59, Sam Corbett <>
>>> Interesting problem Svet. Your proposal is a neat way of sidestepping the problem
of updating many security groups as the set of HA nodes changes.
>>> Let me rephrase it to see if I understand correctly.
>>> Suppose we have:
>>> * _n_ Brooklyn servers in an HA cluster (that could be across many clouds / regions
within a cloud)
>>> * _c_ clouds that Brooklyn can deploy to
>>> * _m_ instances across those clouds.
>>> We want to avoid the n+1th Brooklyn node requiring _m_ security group updates.
>>> You propose that the manager of the Brooklyn HA cluster maintain at least _c_
security groups (one per security group scope - e.g. AWS EC2 region - per cloud). Each of
these groups has _n_ records. When the HA cluster is resized each group is modified to add
or remove a record as appropriate.
>>> Do I have this correct?
>>> Sam
>>> On 14/02/2017 15:42, Svetoslav Neykov wrote:
>>>> I'm trying to restrict access to the machines managed by Brooklyn using security
groups - tightening jclouds' default behaviour of opening the "inboundPorts" to any source.
>>>> Brooklyn obviously needs to have access to all managed machines. This means
it needs to figure out the address it uses to access each machine and white list it in the
machine's security group.
>>>> This is kind of related to the email thread "[PROPOSAL] Separate management
addresses from the concept of an entity's public address" [1], but in reverse. Instead of
figuring out which machine IP to use I need to do the reverse - which Brooklyn node IP will
access the machine.
>>>> It becomes more complicated when HA is introduced into the mix. Any node
that becomes a master needs to be able to access the machines. This means the security groups
need to be updated in such cases.
>>>> Two questions follow:
>>>>   1. How to determine which IP faces managed machines? There's no one fixed
answer here. Depending on the target cloud and location configuration it varies.
>>>>   2. How to keep the list of IPs from above point in sync, for each of the
members of the HA cluster?
>>>> Don't think we can actually answer q1. That's why the solution I'm thinking
of is:
>>>>   * Always open the external IP to the machines. The external IP is as reported
by "LocalhostExternalIpLoader".
>>>>   * Assign a predefined SG to all machines in the HA cluster - manually/out
of band, since the machines are not managed by Brooklyn. Let Brooklyn know the SG name, defaulting
to "management-<cluster-id>". White list the SG as a source for all managed machines.
This will allow Brooklyn to access managed machines on both the public and private IPs. It
moves the responsibility of assigning the SG to new HA member machines to whoever is managing
the Brooklyn cluster. We could then update the management SG with **all** private IPs in the
HA cluster (need to advertise them in the meta data) or leave it again to the manager of the
>>>> Would be really cool to have HA clusters manage/heal themselves.
>>>> Tangentially related - [2] which IP do we use for the "url" field int he
HA member nodes metadata in REST API (currently empty for the Karaf dist). If it's always
the public IP then it doesn't work for private/VPN instances. It is important for this to
be the right one because:
>>>>   * Users are redirected to the master node
>>>>   * Automated systems need to know which is the current master. On failover
the old master (if still around) will redirect to the new master. Workaround is to keep a
local copy of the HA members and iterate over them until it hits MASTER - but it's still important
that the URLs are accessible.
>>>> Svet
>>>> [1]
>>>> [2]

View raw message