cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12347) Gossip 2.0 - broadcast tree for data dissemination
Date Wed, 31 Aug 2016 13:23:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452232#comment-15452232
] 

Jason Brown commented on CASSANDRA-12347:
-----------------------------------------

[~thobbs] Excellent question, (and sorry for the delay). It's more of the latter point, "Minimize
the amount of redundant messaging, ...". The goal of the broadcast tree is to make sure the
work we do is smart work, and not just sending excess/redundant messages. This is acheived
by using the directionality of the tree (that is, broadcasting from the tree root down to
the leaves) versus a naive flooding of the links (provided by the hyparview active view).
Of course, I doubt that we would use a "naive flooding" approach; however, I'm pretty sure
that once we address questions like:
- when do we stop sending a message (a/k/a cold rumor mongering)?
- how do we detect and recover lost messages (messages not received)?
- how do we not consume a lot of resources (network, cpu, etc)?

We end up with something that looks like a tree.

Thicket nicely awnsers these concerns, and worth pointing out is it specifices an intelligent
branch healing algorithm that allows a node to recover missing messages (and heal the tree).

With regard to the dissemination of the existing 'gossip' states (which I've been calling
'cluster metadata'), I agree that information does not change often. However, I'm not imagining
cluster metadata to be the only information communicated via the new gossip stack. At a minimum,
I've suggested CASSANDRA-12106 use the new gossip to disseminate it's blacklisted partition
data. In addition, I've had conversations in the past with several c* developers where we've
said, "wouldn't it be great if i could just share this data with every node via gossip?".


> Gossip 2.0 - broadcast tree for data dissemination
> --------------------------------------------------
>
>                 Key: CASSANDRA-12347
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12347
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jason Brown
>            Assignee: Jason Brown
>
> Description: A broadcast tree (spanning tree) allows an originating node to efficiently
send out updates to all of the peers in the cluster by constructing a balanced, self-healing
tree based upon the view it gets from the peer sampling service (CASSANDRA-12346). 
> I propose we use an algorithm based on the [Thicket paper|http://www.gsd.inesc-id.pt/%7Ejleitao/pdf/srds10-mario.pdf],
which describes a dynamic, self-healing broadcast tree. When a given node needs to send out
a message, it dynamically builds a tree for each node in the cluster; thus giving us a unique
tree for every node in the cluster (a tree rooted at every cluster node). The trees, of course,
would be reusable until the cluster configurations changes or failures are detected (by the
mechanism described in the paper). Additionally, Thicket includes a mechanism for load-balancing
the trees such that nodes spread out the work amongst themselves.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message