hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1025) Reconstruction log playback has no bounds on memory used
Date Tue, 22 Jun 2010 23:12:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881441#action_12881441

HBase Review Board commented on HBASE-1025:

Message from: "Nicolas" <nspiegelberg@facebook.com>

This is an automatically generated e-mail. To reply, visit:

couple questions after reviewing.  didn't look at previous reviews first, so sorry if I duplicated


    don't you still want to keep around code to read oldlogfile.log & just remove write
path?  We're not changing Log file format between 0.20=>0.21, so a customer should be able
to cleanly upgrade.


    technically, it's -1 if no outstanding log edits exist.  you store the max sequence ID
even if you skip all the edits.


    is there a use case for putting HDFS in safe mode, then running HBase with hbase.skip.errors
do see the state of the cluster?  If so, fs.delete + fs.rename will both assert when this
is played on cluster restart.  Maybe you want to catch both and print errors?


    do you want to update the currentEditSeqId even if it's from the wrong family?  just making


    do we want the option to store this HLog for post-mortem in this case?  we're talking
about CF-level, so this couldn't happen because of region splitting, right?


    would it make more sense to have the interval be in seconds instead of count, then have
the update give the edit count?  Or is the difference in restoring large edits (~50k) versus
small ones inconsequential?

- Nicolas

> Reconstruction log playback has no bounds on memory used
> --------------------------------------------------------
>                 Key: HBASE-1025
>                 URL: https://issues.apache.org/jira/browse/HBASE-1025
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.21.0
>         Attachments: 1025-v2.txt, 1025-v3.txt, 1025-v5.patch, 1025-v8.txt, 1025.txt
> Makes a TreeMap and just keeps adding edits without regard for size of edits applied;
could cause OOME (I've not seen a definitive case though have seen an OOME around time of
a reconstructionlog replay -- perhaps this the straw that broke the fleas antlers?)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message