accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser" <josh.el...@gmail.com>
Subject Re: Review Request 19790: ACCUMULO-378 Design document
Date Sun, 30 Mar 2014 23:25:51 GMT


> On March 28, 2014, 6:06 p.m., Mike Drob wrote:
> > I did not see any considerations for dealing with limited WAN capacity. If data
ingest is local to the master, then it is feasible that the ingest bandwidth is much higher
than can be efficiently acheived between e.g. Oregon and Virginia. We would also probably
need some sequence function so that while clients reading from the slave might see stale information,
they are not at risk of seeing Key combinations that have never existed.
> 
> kturner wrote:
>     Are you concerned about data being written in a different order on the slave?  Or
mutations being broken up?  Or something else?

I'm pretty sure I called out provisions in the design to support the link being offline. The
only necessary detail that I didn't call out was that we need ordering across WALs (make sure
WAL1 comes before WAL2) and bulk imports (rfile1 is replicated before rfile2).

Thinking some more, I'd have to look into the recovery cases on the master cluster. Data is
written to a table by tserver1 using wal1, tserver1 crashes and is replayed on tserver2 which
is using wal2. Do we need to do anything to make sure that wal1 is replicated before wal2
begins replication? I believe so because after tserver2 takes ownership of that tablets from
tserver1, it might have new mutations that must be replayed to ensure proper ordering. Is
this what you were thinking of?


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19790/#review38922
-----------------------------------------------------------


On March 28, 2014, 5:54 p.m., kturner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19790/
> -----------------------------------------------------------
> 
> (Updated March 28, 2014, 5:54 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-378
>     https://issues.apache.org/jira/browse/ACCUMULO-378
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> ACCUMULO-378 Design document.  Posting for review here, not meant for commit.  Final
version of document should be posted on issue.
> 
> 
> Diffs
> -----
> 
>   docs/src/main/resources/design/ACCUMULO-378-design.mdtext PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/19790/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> kturner
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message