hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1008) [performance] The replay of logs on server crash takes way too long
Date Mon, 18 May 2009 18:23:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710435#action_12710435
] 

Jim Kellerman commented on HBASE-1008:
--------------------------------------

I'm not a big fan of having to read all the logs into memory.

My suggestion would be for each unique region in the HLog(s), create a blocking queue and
a thread that will
dequeue entries and write them directly to the log file. Then you have one thread doing the
reading and multiple
threads writing, and the memory footprint is reduced significantly.

Make sense?

> [performance] The replay of logs on server crash takes way too long
> -------------------------------------------------------------------
>
>                 Key: HBASE-1008
>                 URL: https://issues.apache.org/jira/browse/HBASE-1008
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: stack
>            Assignee: Jean-Daniel Cryans
>            Priority: Blocker
>             Fix For: 0.20.0, 0.19.3
>
>         Attachments: 1008-v2.patch, hbase-1008-3.patch, hbase-1008-v4-0.19.patch, hbase-1008-v4.patch
>
>
> Watching recovery from a crash on streamy.com where there were 1048 logs and repay is
running at rate of about 20 seconds each.  Meantime these regions are not online.  This is
way too long to wait on recovery for a live site.  Marking critical.  Performance related
so priority and in 0.20.0.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message