cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Blake Eggleston (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6246) EPaxos
Date Wed, 12 Nov 2014 02:19:35 GMT


Blake Eggleston commented on CASSANDRA-6246:

Thanks Sankalp. Since my last post, I've been cleaning things up and improving the tests.
Sorry for the delay pushing it up.

I also found a problem in the execution phase that was slowing things down. Epaxos is now
40% faster than the existing implementation in uncontended workloads, and 20x faster in contended

Here are the performance numbers:

bq. 1) In the DependencyManger, we might want to keep the last executed instance otherwise
we won't know if the next one depends on the previous one or we have missed any in between.

 Instances only become eligible for eviction when they’ve been both executed and acknowledged.
An executed instance will be a dependency of at least one additional instance before being
evicted from the manager.

2) You might want to create java packages and move files there. For example in repair code, where we keep all the Request Responses. We can do the
same for verb handler, etc. 
3) We should add the new verbs to DatabaseDescriptor.getTimout(). Otherwise they will use
the default timeout. I fixed this for current paxos implementation in CASSANDRA-7752
4) PreacceptResponse.failure can also accept missingInstances in the constructor. You can
make it final and not volatile. 

I'll look into these

bq. 5) ExecutionSorter.getOrder(). Here if condition uncommitted.size() == 0 is always true.
Also loadedScc is empty as we don't insert into it. 

ids are being put into uncommitted in the addInstance method, so it won’t always equal 0,
good catch on the loadedScc though. I’ll get that fixed.

bq. 6) In, Instance toExecute = state.loadInstance(toExecuteId); should
be within the try as we are holding a lock. 

fixed in the cleaned up code

bq. 7) EpaxosState.commitCallbacks could be a multi map. 

agreed, I'll update

8) In, successors, noop and fastPathPossible are not used. We can also get rid
of Instance.applyRemote() method.
14) ParticipantInfo.endpoints will not be required once we remove the Epaxos.getSuccessors()

successors and noop will be used in the prepare and execute phases respectively, fastPathImpossible
should be removed through. 

bq. 9) PreacceptCallback.ballot need not be an instance variable as we set completed=true
after we set it. 

agreed, I'll update

10) PreacceptResponse.missingInstance is not required as it can be calculated on the leader
in the PreacceptCallback. 
11) EpaxosState.accept(). We can filter out the skipPlaceholderPredicate when we calculated
missingInstances in PreacceptCallback.getAcceptDecision()

Missing instances are sent both ways. When a node responds to a preaccept message, if it believes
the leader is missing an instance, it will include it in it's response. Once the leader has
received all the responses, if it thinks any of the replicas are missing instances, it will
send them along.

12) PreacceptCallback.getAcceptDecision() We don't need to calculate missingIds if accept
is going to be false in AcceptDecision. 
13) ParticipantInfo.remoteEndpoints. Here we are not doing any isAlive check and just sending
messages to all remote endpoints. 

I'll fix

bq. 15) Accept is send to live local endpoints and to all remote endpoints. In AcceptCallback,
I think we should count response from only local endpoints 

fixed in cleaned up code

bq. 16) When we execute the instance in ExecuteTask, what if we crash after executing the
instance but before recording it.

Saving the best for last I see :)
The existing implementation has this problem as well. Cassandra doesn't have a way to mutate
multiple keyspaces with a single commit log entry (that I've found). We could collect the
mutations from the actual cas write, the dependency manager update, and the instances update
and hold off on applying them until the very end, but that only makes the problem less likely.

Speaking of which, the default of not waiting for an fsync before considering a write successful
is a more serious problem for paxos/epaxos, since a paxos node forgetting it's state can cause

I'll give this and your timeline consistency question some more thought.

> EPaxos
> ------
>                 Key: CASSANDRA-6246
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Blake Eggleston
>            Priority: Minor
> One reason we haven't optimized our Paxos implementation with Multi-paxos is that Multi-paxos
requires leader election and hence, a period of unavailability when the leader dies.
> EPaxos is a Paxos variant that requires (1) less messages than multi-paxos, (2) is particularly
useful across multiple datacenters, and (3) allows any node to act as coordinator:
> However, there is substantial additional complexity involved if we choose to implement

This message was sent by Atlassian JIRA

View raw message