hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1025) Reconstruction log playback has no bounds on memory used
Date Thu, 24 Jun 2010 07:40:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882079#action_12882079

HBase Review Board commented on HBASE-1025:

Message from: "Nicolas" <nspiegelberg@facebook.com>

bq.  On 2010-06-22 16:07:23, Nicolas wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java, line 2002
bq.  > <http://review.hbase.org/r/179/diff/2/?file=1358#file1358line2002>
bq.  >
bq.  >     do we want the option to store this HLog for post-mortem in this case?  we're
talking about CF-level, so this couldn't happen because of region splitting, right?
bq.  stack wrote:
bq.      This condition should never happen.  Only reason it might would be if schema was
edited between log creation and new deploy.   It'd be cumbersome adding a keep log at this
stage of the processing.  Should I open an issue?

I definitely want to minimize code complexity for rare cases, but I've applied too many patches
that delete the phrase 'should never happen' that it makes me want to have at least a half-baked
plan to keep around the data for later analysis. Doing something like Log.error and then placing
the entire log in a 'should_never_happen' directory would be fine.  Maybe I'm being overly-paranoid?

- Nicolas

This is an automatically generated e-mail. To reply, visit:

> Reconstruction log playback has no bounds on memory used
> --------------------------------------------------------
>                 Key: HBASE-1025
>                 URL: https://issues.apache.org/jira/browse/HBASE-1025
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: stack
>             Fix For: 0.21.0
>         Attachments: 1025-v2.txt, 1025-v3.txt, 1025-v5.patch, 1025-v8.txt, 1025.txt
> Makes a TreeMap and just keeps adding edits without regard for size of edits applied;
could cause OOME (I've not seen a definitive case though have seen an OOME around time of
a reconstructionlog replay -- perhaps this the straw that broke the fleas antlers?)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message