hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenrongwei (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15125) HBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary
Date Thu, 21 Jan 2016 06:29:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110145#comment-15110145
] 

chenrongwei commented on HBASE-15125:
-------------------------------------

Maybe we should have a look at the debug log which produced by the unit test case added newly.
 It appears like below,

First,after call the function of adoptHdfsOrphan,it produced below logs as we expected.  
  

2016-01-21 13:59:26,151 INFO  [Time-limited test] util.HBaseFsck(932): Creating new region
: {ENCODED => 5dc51a10d64abdd23e8c7612b6514b23, NAME => 'tableHDFSRegioninfoMissing,B0,1453355965454.5dc51a10d64abdd23e8c7612b6514b23.',
STARTKEY => 'B0', ENDKEY => 'B5\x00'}

Second,After it done fix hole,then will do check again,and there is no error more ( actually,the
case will be passed),
      // fix hole
      doFsck(conf, true);

      // check that hole fixed
      assertNoErrors(doFsck(conf, false));

The most important we got the correct region infos,like below,

:	[ { meta => tableHDFSRegioninfoMissing,,1453355931907.b34f3bdf3accfac4813ba28f0a5fbd01.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/b34f3bdf3accfac4813ba28f0a5fbd01,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,,1453355931907.b34f3bdf3accfac4813ba28f0a5fbd01.,
replicaId => 0 }, A]

A:	[ { meta => tableHDFSRegioninfoMissing,A,1453355931907.b99b6ad5f486529a1fa5966015818958.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/b99b6ad5f486529a1fa5966015818958,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,A,1453355931907.b99b6ad5f486529a1fa5966015818958.,
replicaId => 0 }, B]

B:	[ { meta => tableHDFSRegioninfoMissing,B,1453355991639.bc127d54f96bb44017e380a2c92fea88.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/bc127d54f96bb44017e380a2c92fea88,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,B,1453355991639.bc127d54f96bb44017e380a2c92fea88.,
replicaId => 0 }, B0]

B0:	[ { meta => tableHDFSRegioninfoMissing,B0,1453355965454.5dc51a10d64abdd23e8c7612b6514b23.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/5dc51a10d64abdd23e8c7612b6514b23,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,B0,1453355965454.5dc51a10d64abdd23e8c7612b6514b23.,
replicaId => 0 }, B5\x00]

 //abviously,it will add a new region by just fix meta only.I think it should be the special
handling.


B5\x00:	[ { meta => tableHDFSRegioninfoMissing,B5\x00,1453355991680.c4a9dc6a2f8be610d593b2ac53f2c498.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/c4a9dc6a2f8be610d593b2ac53f2c498,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,B5\x00,1453355991680.c4a9dc6a2f8be610d593b2ac53f2c498.,
replicaId => 0 }, C]  

C:	[ { meta => tableHDFSRegioninfoMissing,C,1453355931907.580fd11d812fb477653bc5ebae7e2cc4.,
hdfs => hdfs://localhost:58128/user/chenrongwei/test-data/95cddd59-558b-47af-b699-9e4a3fb9649a/data/default/tableHDFSRegioninfoMissing/580fd11d812fb477653bc5ebae7e2cc4,
deployed => 10.0.39.138,58139,1453355763481;tableHDFSRegioninfoMissing,C,1453355931907.580fd11d812fb477653bc5ebae7e2cc4.,
replicaId => 0 }, ]

Finnally, I think if the last region corrupt,it will create two new regions to replace it.
one is [the min key in last region's hfile,the max key in last region's hfile+byte[1]), another
one is [the max key in last region's hfile+byte[1],null).It's no different with other corrupt
region.

> HBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-15125
>                 URL: https://issues.apache.org/jira/browse/HBASE-15125
>             Project: HBase
>          Issue Type: Bug
>          Components: hbck
>    Affects Versions: 2.0.0
>            Reporter: chenrongwei
>            Assignee: chenrongwei
>         Attachments: HBASE-15125-V001.patch, HBASE-15125-v002.patch, HBASE-15125-v003.patch,
HBASE-15125-v004.patch
>
>
> There is a bug in HBaseFsck's adoptHdfsOrphan function.At the last of this function will
create a region,which want to cover all the orphan regions.But the end key of this new region
was set incorrectly.Correct region's boundary should be [startKey,endKey),but this function
create a region with boundary of [startKey,endKey],this bug will leads to scan operation omit
some data.
> I think we should create the region like bellow,
>     // create new region on hdfs. move data into place.
>     HRegionInfo hri = new HRegionInfo(template.getTableName(), orphanRegionRange.getFirst(),
>         Bytes.add(orphanRegionRange.getSecond(), new byte[1]));



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message