accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ke...@deenlo.com
Subject Re: Review Request 27198: ACCUMULO-3236 introducing cloneInto feature
Date Mon, 22 Dec 2014 23:34:19 GMT


> On Oct. 30, 2014, 4:06 p.m., kturner wrote:
> > server/master/src/main/java/org/apache/accumulo/master/tableOps/CloneIntoTable.java,
line 84
> > <https://reviews.apache.org/r/27198/diff/2/?file=741888#file741888line84>
> >
> >     Are the tables offline?  If not, then the splits could change during this operation.
> 
> John Vines wrote:
>     They are not, but this isn't an issue. The locks prevent a merge and the bulk import
code we're utilizing handles tablets splitting mid-operation.

What about the following situation?  Seems like this has the possibility to reintroduce deleted
data (data deleted before clone into starts).

1. Src table has splits {B, C, R}
2. Insert Row=N col=cf1,cq1 val=4 into Src tablet (C,R]
3. Insert Row=I col=cf1,cq1 val=6 into Src tablet (C,R]
4. Tablet Src (C,R] flushes creating file F8
5. Delete Row=I col=cf1,cq1 in Src tablet (C,R]
6. Tablet Src (C,R] flushes creating file F9
7. Clone into operation starts (Dest has splits {B, C, R})
8. Clone into checks that Src is subset of Dest
9. Src tablet (C,R] splits into (C,K] and (K,R].  Tablet (C,K] has files F9,F8.  Tablet (K,R]
has file F8.
10. Src tablet (C,K] does full compaction dropping delete marker for Row I
11. CloneInto imports only file F8 into Dest tablet (C,R]


- kturner


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27198/#review59201
-----------------------------------------------------------


On Dec. 22, 2014, 7:59 p.m., John Vines wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27198/
> -----------------------------------------------------------
> 
> (Updated Dec. 22, 2014, 7:59 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-3236
>     https://issues.apache.org/jira/browse/ACCUMULO-3236
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> Includes all code to support feature, including thrift changes
> Includes minor code cleanup to TableLocator and items in the Bulk path to remove signature
items that are unused (arguments & exceptions)
> Includes renaming of some bulk import functions to clarify their purpose (because they're
now multi-purpose)
> 
> Patch is based on 1.6, but we can choose to make it target only 1.7 if we choose (this
conversation should be taken up on jira, not in RB)
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java 97f538d

>   core/src/main/java/org/apache/accumulo/core/client/impl/RootTabletLocator.java 97d476b

>   core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java 07df1bd

>   core/src/main/java/org/apache/accumulo/core/client/impl/TabletLocator.java e396d82

>   core/src/main/java/org/apache/accumulo/core/client/impl/TabletLocatorImpl.java c550f15

>   core/src/main/java/org/apache/accumulo/core/client/impl/TimeoutTabletLocator.java bcbe561

>   core/src/main/java/org/apache/accumulo/core/client/impl/thrift/TableOperation.java
7716823 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockTableOperationsImpl.java
de19137 
>   core/src/main/java/org/apache/accumulo/core/client/mock/impl/MockTabletLocator.java
35f160f 
>   core/src/main/java/org/apache/accumulo/core/master/thrift/FateOperation.java f65f552

>   core/src/main/java/org/apache/accumulo/core/tabletserver/thrift/TabletClientService.java
2ba7674 
>   core/src/main/thrift/client.thrift 38a8076 
>   core/src/main/thrift/master.thrift 38e9227 
>   core/src/main/thrift/tabletserver.thrift 25e0b10 
>   core/src/test/java/org/apache/accumulo/core/client/admin/TableOperationsHelperTest.java
1d91574 
>   core/src/test/java/org/apache/accumulo/core/client/impl/TableOperationsHelperTest.java
02838ed 
>   server/base/src/main/java/org/apache/accumulo/server/client/BulkImporter.java 4cc13a9

>   server/base/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java
fe17a62 
>   server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java
258080c 
>   server/base/src/test/java/org/apache/accumulo/server/client/BulkImporterTest.java 3680341

>   server/master/src/main/java/org/apache/accumulo/master/FateServiceHandler.java 5818da3

>   server/master/src/main/java/org/apache/accumulo/master/tableOps/CloneIntoTable.java
PRE-CREATION 
>   server/tserver/src/main/java/org/apache/accumulo/tserver/Tablet.java 9a07a4a 
>   server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java 3f594cc

>   test/src/main/java/org/apache/accumulo/test/performance/thrift/NullTserver.java 0591b19

>   test/src/test/java/org/apache/accumulo/test/functional/CloneIntoIT.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/27198/diff/
> 
> 
> Testing
> -------
> 
> Includes CloneIntoIT, which exercises all permutations of the flags. Existing BulkIT
still functions as intended for validation of no feature loss in refactoring exiting code
for multi-purposing.
> 
> 
> Thanks,
> 
> John Vines
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message