cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5400) Allow multiple ports to gossip from a single IP address
Date Fri, 05 Apr 2013 07:44:16 GMT


Sylvain Lebresne commented on CASSANDRA-5400:

Sorry, I realize I've been a bit carried away. I saw "cql schema definitions" and somehow
assumed we were talking of one of the schema_* table but we're not.  The peers table is a
local one, so there is no problem with backward compatibility as far as rolling upgrade are
concerned, my bad (and so using a composite PK would indeed be an option).

There is compatibility issues to consider however, and that's for clients. The peers table
is the new describe_ring|token_map of CQL3 clients that thus use it. For those, just adding
a new 'port' column (without renaming existing ones) will definitively be easier to handle
that adding a new data type.

But I have more important concerns. Multiple nodes on the same IP means also by necessity
means multiple RPC/native protocol port in the cluster too.

So far, clients assumed that 1) nodes are identified by just their ip address and 2) all nodes
were using the same port for thrift/native protocol. And I'll note that they largely had no
choice about making those assumptions because the (RPC/native protocol) ports where not really
exposed. But this issue breaks those 2 assumptions. Meaning that 1) all existing client drivers
and tools that were identifying node with just the ip addresses will have to be fixed and
2) we'd have to start gossiping the rpc port (CASSANDRA-4501) because otherwise doing node
auto-discovery properly becomes much more harder (and for instance, binary protocol notifications
*assume* that all node use the same rpc port since we don't have CASSANDRA-4501).

I'll also note that none of the describe_* thrift calls include the port currently, and there
is no easy way to return it without breaking the thrift API (of course, shoving the port in
the string currently used to return the ip address would be breaking and thus non acceptable).

So I have to ask, are we absolutely sure that this addition is worth the pretty substantial
headache it will be for most clients drivers and tools? I'm no network expert but can't one
just use multiple IP aliases if he want multiple instances on the same NIC.

To be clear, I absolutely agree that it is lame we haven't identified nodes by ip and port
from day one. But it seems to me too much trouble for little benefit to change it now.

> Allow multiple ports to gossip from a single IP address
> -------------------------------------------------------
>                 Key: CASSANDRA-5400
>                 URL:
>             Project: Cassandra
>          Issue Type: New Feature
>    Affects Versions: 2.0
>            Reporter: Carl Yeksigian
>            Assignee: Carl Yeksigian
>             Fix For: 2.0
>         Attachments: 5400.txt, 5400-v2.txt, 5400-v3.patch, 5400-v4.patch
> If a fat client is running on the same machine as a Cassandra node, the fat client must
be allocated a new IP address. However, since the node is now a part of the gossip, the other
nodes in the ring must be able to talk to it. This means that a local only address (127.0.0.n)
won't actually work for the rest of the ring.
> This also would allow for multiple Cassandra service instances to run on a single machine,
or from a group of machines behind a NAT.
> The change is simple in concept: instead of using an InetAddress, use a different class.
Instead of using an InetSocketAddress, which would still tie us to using InetAddress, I've
added a new class, "CassandraInstanceEndpoint". The serializer allows for reading a serialized
Inet4Address or Inet6Address; also, the message service can still communicate with non-CassandraInstanceEndpoint
aware code.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message