hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13938) Deletes done during the region merge transaction may get eclipsed
Date Tue, 23 Jun 2015 02:40:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597031#comment-14597031
] 

Hadoop QA commented on HBASE-13938:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12741172/hbase-13938_master.patch
  against master branch at commit d51a184051d968dc3bdc00b1c9257c0a9e5ff8a6.
  ATTACHMENT ID: 12741172

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified
tests.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.1 2.5.2 2.6.0)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of
protoc compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

                {color:red}-1 checkstyle{color}.  The applied patch generated 1944 checkstyle
errors (more than the master's current 1942 errors).

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
100:
    +      regionServer.compactSplitThread.requestRegionsMerge(regionA, regionB, forcible,
masterSystemTime);
+      MetaTableAccessor.mergeRegions(connection, merged, parentA, parentB, serverName0, 3,
HConstants.LATEST_TIMESTAMP);

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/14513//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/14513//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/14513//artifact/patchprocess/checkstyle-aggregate.html

                Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14513//console

This message is automatically generated.

> Deletes done during the region merge transaction may get eclipsed
> -----------------------------------------------------------------
>
>                 Key: HBASE-13938
>                 URL: https://issues.apache.org/jira/browse/HBASE-13938
>             Project: HBase
>          Issue Type: Bug
>          Components: master, regionserver
>            Reporter: Devaraj Das
>            Assignee: Enis Soztutar
>             Fix For: 0.98.14, 1.2.0, 1.1.1, 1.3.0
>
>         Attachments: 13938-branch-1.1.txt, hbase-13938_master.patch, hbase-13938_v2-branch-1.1.patch,
hbase-13938_v3-0.98.patch, hbase-13938_v3-0.98.patch, hbase-13938_v3-branch-1.0.patch, hbase-13938_v3-branch-1.1.patch,
hbase-13938_v3-branch-1.2.patch, hbase-13938_v3-branch-1.patch
>
>
> Was looking at an issue from our internal testing. It seems the Deletes of the region
rows from the meta done during the merge transaction could be eclipsed by the Put of a region
row that might have happened moments before.
> The master logs this for the merge:
> {noformat}
> 2015-06-18 13:13:46,018 INFO  [AM.ZK.Worker-pool2-t12] master.AssignmentManager: Handled
MERGED event; merged=IntegrationTestIngest,a666665c,1434633226681.0927319db6bf5e128e3bec2a420819aa.,
region_a=IntegrationTestIngest,a666665c,1434631353820.8b911862d7705ac808b8d132d0154c16., region_b=IntegrationTestIngest,acccccc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec.,
on ddas-2-5.openstacklocal,16020,1434632778438
> {noformat}
> One of the regions that got merged got Opened a few seconds back:
> {noformat}
> 2015-06-18 13:13:46,591 INFO  [RS_OPEN_REGION-ddas-2-5:16020-1] regionserver.HRegion:
Onlined 1bdaf759862f45d133ef77fdbda21aec; next sequenceid=182988
> {noformat}
> The above would have done a Put in the meta.
> Looking at the raw scan of the meta, for the new merged region, the creation timestamp
is 1434633226101:
> {noformat}
>  IntegrationTestIngest,a666665c,1434633226681.0927319db6bf5e128e3bec2a420819aa. column=info:regioninfo,
timestamp=1434633226101, value={ENCODED => 0927319db6bf5e128e3bec2a420819aa, NAME =>
'IntegrationTestIngest,a666665c,1434633226681.0927319db6bf5e128e3bec2a420819aa.', STARTKEY
=> 'a666665c', ENDKEY => 'b3333328'}
> {noformat}
> Looking at the raw scan of the meta, the timestamp for the region open of the already
merged region is 1434633226600. This is a little after the merge transaction's timestamp.
> {noformat}
> IntegrationTestIngest,acccccc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. column=info:seqnumDuringOpen,
timestamp=1434633226600, value=\x00\x00\x00\x00\x00\x02\xCA\xCC
>  IntegrationTestIngest,acccccc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. column=info:server,
timestamp=1434633226600, value=ddas-2-5.openstacklocal:16020
>  IntegrationTestIngest,acccccc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. column=info:serverstartcode,
timestamp=1434633226600, value=1434632778438
> {noformat}
> We need to fix it so that the merge region transaction also takes the master's timestamp.
Similar to HBASE-13875.
> When this happens, clients start to see a row in the meta with an empty HRegionInfo (this
is because the Put done during the region open only updates the location information but not
the HRI, and the HRI deleted during the merge transaction "remains deleted").



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message