cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksey Yeschenko (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-5593) Auth.isExistingUser is periodically throwing org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE
Date Fri, 24 May 2013 03:00:20 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aleksey Yeschenko resolved CASSANDRA-5593.
------------------------------------------

    Resolution: Invalid

This is not a good fix. At all.

This behavior is expected. What you should've done instead was to increase RF on system_auth
keyspace (with a regular ALTER KEYSPACE system_auth .. query).
                
> Auth.isExistingUser is periodically throwing org.apache.cassandra.exceptions.UnavailableException:
Cannot achieve consistency level ONE
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-5593
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5593
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.4
>         Environment: Three node cluster
>            Reporter: Justin Geiser
>
> When setting up authentication on a clustered setup we're periodically getting an UnavailableException:
Cannot achieve consistency level ONE whenever one or two of the cluster nodes is down.
> {code}
> java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot
achieve consistency level ONE
>         at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:75)
>         at com.resolve.cassandra.auth.SimpleAuthenticator.setup(SimpleAuthenticator.java:273)
>         at org.apache.cassandra.auth.Auth.setup(Auth.java:139)
>         at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:781)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
>         at org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
>         at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
>         at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
>         at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
> Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency
level ONE
>         at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:250)
>         at org.apache.cassandra.service.ReadCallback.assureSufficientLiveNodes(ReadCallback.java:152)
>         at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:891)
>         at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:829)
>         at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:126)
>         at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:1)
>         at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
>         at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
>         at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:151)
>         at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:71)
>         ... 8 more
> {code}
> Digging into the issue it looks like the problem is the SimpleStrategy.calculateNaturalEndpoints
method is only returning 1 entry, because the replication factor for the system_auth column
family is 1, and if this node happens to be one of the down nodes it gets removed by getLiveNaturalEndpoints
in StorageService.  So by the time it reaches StorageProxy.fetchRows(StorageProxy.java:891)
the endpoints list is empty, even though we have valid nodes running.
> For a quick fix I removed the "endpoints.size() < replicas" check from the while loop
in SimpleStrategy.calculateNaturalEndpoints:
> {code}
>     public List<InetAddress> calculateNaturalEndpoints(Token token, TokenMetadata
metadata)
>     {
>         int replicas = getReplicationFactor();
>         ArrayList<Token> tokens = metadata.sortedTokens();
>         List<InetAddress> endpoints = new ArrayList<InetAddress>(replicas);
>         if (tokens.isEmpty())
>             return endpoints;
>         // Add the token at the index by default
>         Iterator<Token> iter = TokenMetadata.ringIterator(tokens, token, false);
>         while (iter.hasNext())
>         {
>             InetAddress ep = metadata.getEndpoint(iter.next());
>             if (!endpoints.contains(ep))
>             {
>                 endpoints.add(ep);
>             }
>         }
>         return endpoints;
>     }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message