accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2801) define tablet syncs walog for each tablet in a batch
Date Tue, 10 Jun 2014 20:29:03 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14026951#comment-14026951
] 

Jonathan Park commented on ACCUMULO-2801:
-----------------------------------------

[~keith_turner] what are your thoughts on not calling sync for define tablet and instead relying
on the sync for a data write to ensure that it exists?

It will make it possible for there to be a metadata table entry for the WAL without there
being an associated DEFINE_TABLET in the WAL which I think recovery will currently ignore
(looking at 1.5.1). It might change our recovery semantics (I'm not fully familiar with what
our current guarantees are) in the case of log rollovers/defines. 

> define tablet syncs walog for each tablet in a batch
> ----------------------------------------------------
>
>                 Key: ACCUMULO-2801
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2801
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.5.0, 1.5.1, 1.6.0
>            Reporter: Keith Turner
>
> When the batch writer sends a batch of mutations for N tablets that were not currently
using a walog, then define tablet will be called for each tablet.  Define tablet will sync
the walog.   In hadoop 2 hsync is used, which is much slower than hadoop1 sync calls.  If
hsync takes 50ms and there are 100 tablets, then this operation would take 5 secs.  The calls
to define tablet do not occur frequently, just when walogs switch or tablets are loaded so
the cost will be amortized.  Ideally there could be one walog sync call for all of the tablets
in a batch of mutations.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message