hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12988) [Replication]Parallel apply edits on row-level
Date Wed, 13 May 2015 05:29:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541365#comment-14541365
] 

Lars Hofhansl commented on HBASE-12988:
---------------------------------------

Looking into this. This is a bit more intricate than expected. An Entry in the WAL is a WALKey/WALEdit
pair. The WALKey (among other stuff has table and region name), the WALEdit has the cells,
which in turn have the row. A WALEdit can contain Cells for many rows.

So to group by row we would need to pull WALEdits apart. I'd have to think through the implication
to convince myself that that's OK. On top of that after the row-grouping we'd now have to
build _new_ WALEdits, since they'd have a different set of Cells. Ugh...

Grouping entire WALEdits on the other hand is easier, but those I can only group by table,
and - as discussed above - that might not be effective in many scenarios.

I can easily do a size based breakup and then rely on the fact that we keep deletes around
for a little bit, but that is brittle... I do have a test patch for this.


> [Replication]Parallel apply edits on row-level
> ----------------------------------------------
>
>                 Key: HBASE-12988
>                 URL: https://issues.apache.org/jira/browse/HBASE-12988
>             Project: HBase
>          Issue Type: Improvement
>          Components: Replication
>            Reporter: hongyu bi
>            Assignee: hongyu bi
>
> we can apply  edits to slave cluster in parallel on table-level to speed up replication
.
> update : per conversation blow , it's better to apply edits on row-level in parallel



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message