Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Content-Type: multipart/alternative;
 boundary="===============5901222595430278892=="
MIME-Version: 1.0
Subject: Re: Review Request 19862: Design document for review on cross-cluster
 replication
From: "Josh Elser" <josh.elser@gmail.com>
To: "accumulo" <dev@accumulo.apache.org>, "Josh Elser" <josh.elser@gmail.com>
Date: Wed, 02 Apr 2014 15:15:25 -0000
Message-ID: <20140402151525.2272.33994@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Josh Elser" <noreply@reviews.apache.org>
References: <20140401015820.28608.95878@reviews.apache.org>
In-Reply-To: <20140401015820.28608.95878@reviews.apache.org>
Reply-To: "Josh Elser" <josh.elser@gmail.com>

--===============5901222595430278892==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19862/#review39260
-----------------------------------------------------------


docs/src/main/resources/design/ACCUMULO-378-design.mdtext
<https://reviews.apache.org/r/19862/#comment71626>

    Thinking about this from a total ordering standpoint. Say we're replicating to two slaves, and we have three rfiles to replicate (1, 2 and 3) to those two slaves.
    
    We replicate rfile1 to both, but then the link to slave2 goes down. We can still replicate rfile2 and then rfile3 to slave1, while we try to send rfile2 to slave2.
    
    What, if instead of the link being down, we happen to communicate to an angry server inside of slave2 which never completes the transfer. We don't want to transfer rfile3 to attempt to better preserve global ordering.
    
    This can be restated as "we only want to replicate one 'file' to a slave at a time" so that we preserve the original semantics of the replication "queue" (table). The problem is that this could drastically slow down replication when the link between master and slave cannot be saturated by one replication task at a time.
    
    This isn't anything that we can reliably guarantee now (without conditional mutations), right? Is it worth trying to tackle? The one clear change I want to make is that we do want to put the identifier for the slave in with the replication record rather than defer determination of where a record should be replicated.


- Josh Elser


On April 1, 2014, 1:58 a.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19862/
> -----------------------------------------------------------
> 
> (Updated April 1, 2014, 1:58 a.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> Re-posting a version of the design doc that I own. Contains grammatical fixes from round one, with a few extra clarifications. New content should be posted here, but I'll maintain the old review as discussion progresses.
> 
> 
> Diffs
> -----
> 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/19862/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


--===============5901222595430278892==--