cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-5432) Repair Freeze/Gossip Invisibility Issues 1.2.4
Date Sat, 20 Apr 2013 05:31:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637123#comment-13637123
] 

Vijay edited comment on CASSANDRA-5432 at 4/20/13 5:30 AM:
-----------------------------------------------------------

Hi Arya, Thanks and you can call me anytime but it will help others if we keep the discussion
here.

{quote}
Has this always been the case? 
{quote}
As far as i know, yes.

{quote}
 I go to security groups and remove the non SSL on public IP rules that I added in previous
step.
{quote}
I think you should not remove the IP's. Priam opens up ports for the local nodes and also
the remote nodes within the security group (http://goo.gl/l9Q1T). Looks like you shouldn't
do the above because you are now disabling cassandra from restarting the connections.

Also the reason you are seeing all the nodes to be UP in a multi region case event though
they cannot communicate within the DC is because of the issue mentioned in CASSANDRA-3533,
I can almost bet that the read/write requests will be failing in the local DC, If not try
after restarting nodes. :)

Let me know if you still have issues or disagree.

                
      was (Author: vijay2win@yahoo.com):
    Hi Arya, Thanks and you can call me anytime but it will help others if we keep the discussion
here.

{quote}
Has this always been the case? 
{quote}
As far as i know, yes.

{quote}
 I go to security groups and remove the non SSL on public IP rules that I added in previous
step.
{quote}
Priam opens up ports for the local nodes and also the remote nodes within the security group
(http://goo.gl/l9Q1T). Looks like you shouldn't do the above because you are now disabling
cassandra from restarting the connections.

Also the reason you are seeing all the nodes to be UP in a multi region case event though
they cannot communicate within the DC is because of the issue mentioned in CASSANDRA-3533,
I can almost bet that the read/write requests will be failing in the local DC, If not try
after restarting nodes. :)

Let me know if you still have issues or disagree.

                  
> Repair Freeze/Gossip Invisibility Issues 1.2.4
> ----------------------------------------------
>
>                 Key: CASSANDRA-5432
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5432
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.4
>         Environment: Ubuntu 10.04.1 LTS
> C* 1.2.3
> Sun Java 6 u43
> JNA Enabled
> Not using VNodes
>            Reporter: Arya Goudarzi
>            Assignee: Vijay
>            Priority: Critical
>
> Read comment 6. This description summarizes the repair issue only, but I believe there
is a bigger problem going on with networking as described on that comment. 
> Since I have upgraded our sandbox cluster, I am unable to run repair on any node and
I am reaching our gc_grace seconds this weekend. Please help. So far, I have tried the following
suggestions:
> - nodetool scrub
> - offline scrub
> - running repair on each CF separately. Didn't matter. All got stuck the same way.
> The repair command just gets stuck and the machine is idling. Only the following logs
are printed for repair job:
>  INFO [Thread-42214] 2013-04-05 23:30:27,785 StorageService.java (line 2379) Starting
repair command #4, repairing 1 ranges for keyspace cardspring_production
>  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,789 AntiEntropyService.java (line 652)
[repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] new session: will sync /X.X.X.190, /X.X.X.43,
/X.X.X.56 on range (1808575600,42535295865117307932921825930779602032] for keyspace_production.[comma
separated list of CFs]
>  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,790 AntiEntropyService.java (line 858)
[repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] requesting merkle trees for BusinessConnectionIndicesEntries
(to [/X.X.X.43, /X.X.X.56, /X.X.X.190])
>  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,086 AntiEntropyService.java (line 214)
[repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle tree for ColumnFamilyName from
/X.X.X.43
>  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,147 AntiEntropyService.java (line 214)
[repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle tree for ColumnFamilyName from
/X.X.X.56
> Please advise. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message