If all the streams are failing it might be due to your VPN tunnel linking the 2 DC or to some port issue, make sur this is ok (Cassandra uses port 7000 for internal communication by default).

I am using this exact same setup and never had streaming issues, excepted during some big repairs. I would say AWS is fine regarding streams, at least to me (I also use EC2Snitch).

What does a nodetool status show ?

Your issue is really weird. If this is only a test (not production) as I think it is, maybe should you go for a 2.1.latest version and see what happen.

2015-07-31 10:54 GMT+02:00 James Vanns <jvanns.ilm@gmail.com>:

 
Why are you using 2.2? In general Cassandra, like most software, can lack stability near the bleeding edge.


I'll be honest, I didn't realise I was using the bleeding edge! I had begun playing around with dsc21 (from the datastax apt repo) but when that failed to install correctly due to dependency issues with the packaging, I switched to dsc22 as this fixed the packaging problem. I *assumed* that dsc22 was an upgraded official release :|
 
error: Error while rebuilding node: Stream failed

This is a relatively common issue with streaming in AWS.


Oh :(
 
Is this a bug or a configuration error perhaps? I know people out there are using
AWS for Cassandra - how are you replicating across regions? Here are two 
values I've tried modifying to no avail;

streaming_socket_timeout_in_ms
phi_convict_threshold

What values did you try? What happened when you did this?

Yes, that would have been helpful I suppose ;) I increased phi_convict_threshold from 10 to 12 and set streaming_socket_timeout_in_ms to 5000. I applied these independently before together. Neither helped
as the same error was reported.

 
As both were referenced in various AWS related Cassandra sources on the web ;)

The amount of data being replicated would be tiny - we're testing a tiny TitanDB
graph of  no more than 100 edges and 100 nodes.

With data that small, you don't really need to use "rebuild," you could just bootstrap the nodes one at a time. Of course bootstrap uses the same streaming as rebuild so that doesn't really help, lol..

Thanks for your help.

 
For issues like this, #cassandra on freenode IRC is often a good forum to get help.

=Rob




--
--
Senior Code Pig
Industrial Light & Magic