hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeffrey Zhong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10201) Port 'Make flush decisions per column family' to trunk
Date Fri, 12 Dec 2014 01:04:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243528#comment-14243528
] 

Jeffrey Zhong commented on HBASE-10201:
---------------------------------------

[~saint.ack@gmail.com] 
{quote}
Are you referring to the following: "Will this mean we drop edits because region thinks its
sequenceid is higher than it should be?"
{quote}
Yes, as of today during replay edits in both modes, we drop WAL edits whose seqId less than
relating store Seq Ids. There some edge cases(like a new PUT, region move to a different RS,
DELETE on the new PUT, major compaction, move back to the original RS and the RS crashes)
we have to know the hFile seqId accurately otherwise the PUT may be restored after recovery.


We need to pass flushed seqIds per store to master so that we can optimize recovery process
but doesn't impact correctness. 

> Port 'Make flush decisions per column family' to trunk
> ------------------------------------------------------
>
>                 Key: HBASE-10201
>                 URL: https://issues.apache.org/jira/browse/HBASE-10201
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: Ted Yu
>            Assignee: zhangduo
>             Fix For: 1.0.0, 2.0.0
>
>         Attachments: 3149-trunk-v1.txt, HBASE-10201-0.98.patch, HBASE-10201-0.98_1.patch,
HBASE-10201-0.98_2.patch, HBASE-10201-0.99.patch, HBASE-10201.patch, HBASE-10201_1.patch,
HBASE-10201_10.patch, HBASE-10201_11.patch, HBASE-10201_12.patch, HBASE-10201_13.patch, HBASE-10201_13.patch,
HBASE-10201_14.patch, HBASE-10201_15.patch, HBASE-10201_16.patch, HBASE-10201_17.patch, HBASE-10201_18.patch,
HBASE-10201_2.patch, HBASE-10201_3.patch, HBASE-10201_4.patch, HBASE-10201_5.patch, HBASE-10201_6.patch,
HBASE-10201_7.patch, HBASE-10201_8.patch, HBASE-10201_9.patch, compactions.png, count.png,
io.png, memstore.png
>
>
> Currently the flush decision is made using the aggregate size of all column families.
When large and small column families co-exist, this causes many small flushes of the smaller
CF. We need to make per-CF flush decisions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message