hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jimmy Xiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7006) [MTTR] Study distributed log splitting to see how we can make it faster
Date Mon, 15 Apr 2013 18:26:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631981#comment-13631981
] 

Jimmy Xiang commented on HBASE-7006:
------------------------------------

I read the proposal and have some questions. At first, it sounds we trade disk io to network
io, which should have better performance.  As to the memstore flush write saving after recovered.edits
have been replayed, the proposal needs to do the same, right?  You just write them to another
WAL file, isn't it true?

Suppose a region server failed again in the middle, does a split worker need to split the
WAL again? This means a WAL may be read/split multiple times?

In the attached performance testing, do we have a breakdown on how many time it spends on
reading the log file, writing to the recovered edits file?  How did you measure the log splitting
time?
                
> [MTTR] Study distributed log splitting to see how we can make it faster
> -----------------------------------------------------------------------
>
>                 Key: HBASE-7006
>                 URL: https://issues.apache.org/jira/browse/HBASE-7006
>             Project: HBase
>          Issue Type: Bug
>          Components: MTTR
>            Reporter: stack
>            Assignee: Jeffrey Zhong
>            Priority: Critical
>             Fix For: 0.95.1
>
>         Attachments: LogSplitting Comparison.pdf, ProposaltoimprovelogsplittingprocessregardingtoHBASE-7006.pdf
>
>
> Just saw interesting issue where a cluster went down  hard and 30 nodes had 1700 WALs
to replay.  Replay took almost an hour.  It looks like it could run faster that much of the
time is spent zk'ing and nn'ing.
> Putting in 0.96 so it gets a look at least.  Can always punt.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message