hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14987) HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
Date Thu, 17 Dec 2015 17:16:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062360#comment-15062360
] 

Ted Yu commented on HBASE-14987:
--------------------------------

The compaction marker is used by region replica feature.
We should not skip replaying it.

During interval discussion a 3rd approach was brought up: replay the recovered edits from
hbck.
There may be several recovered edits which need to be replayed. In this approach, hbck needs
to perform the replay in parallel.
There may be more marker(s) added in the future, this approach introduces tight coupling between
hbck and WAL handling.

A 4th approach might be not throwing WrongRegionException for the compaction marker in WAL.
Assumption is that hbck (or admin) may move WAL from region A to region B.

> HBaseFsck#mergeRegionDirs() needs to handle compaction marker in recovered edits
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-14987
>                 URL: https://issues.apache.org/jira/browse/HBASE-14987
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Ted Yu
>            Assignee: Stephen Yuan Jiang
>
> One customer encountered the following error when replaying recovered edits, leading
to region open failure:
> {code}
> region=table1,d6b-2282-9223370590058224807-U-9856557-        EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.,
starting to roll back the global memstore   size.
> org.apache.hadoop.hbase.regionserver.WrongRegionException: Compaction marker from WAL
table_name: "table1"
> encoded_region_name: "d389c70fde9ec07971d0cfd20ef8f575"
> ...
> region_name: "table1,d6b-2282-9223370590058224807-U-9856557-EJ452727-16313786400171,1449089609367.d389c70fde9ec07971d0cfd20ef8f575."
>  targetted for region d389c70fde9ec07971d0cfd20ef8f575 does not match this region: {ENCODED
=> fa8a526f2578eb3630bb08a4b1648f5d, NAME => 'table1,d6b-2282-                     
  9223370590058224807-U-9856557-EJ452727-16313786400171,1449616291799.fa8a526f2578eb3630bb08a4b1648f5d.',
STARTKEY => 'd6b-2282-9223370590058224807-U-9856557-EJ452727-             16313786400171',
ENDKEY => 'd76-2553-9223370588576178807-U-7416904-EK875822-17662180600000'}
>   at org.apache.hadoop.hbase.regionserver.HRegion.checkTargetRegion(HRegion.java:4592)
>   at org.apache.hadoop.hbase.regionserver.HRegion.replayWALCompactionMarker(HRegion.java:3831)
>   at org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:3747)
>   at org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3601)
>   at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:911)
>   at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:789)
>   at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:762)
>   at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5774)
>   at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5744)
> {code}
> This was likely caused by the following action of hbck:
> {code}
> 15/12/08 18:11:34 INFO util.HBaseFsck: [hbasefsck-pool1-t37] Moving files from hdfs://Zealand/hbase/data/default/table1/d389c70fde9ec07971d0cfd20ef8f575/recovered.edits
into     containing region hdfs://Zealand/hbase/data/default/table1/fa8a526f2578eb3630bb08a4b1648f5d/recovered.edits
> {code}
> The recovered.edits for d389c70fde9ec07971d0cfd20ef8f575 contained compaction marker
which couldn't be replayed against fa8a526f2578eb3630bb08a4b1648f5d



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message