hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
Date Fri, 08 Jul 2016 05:51:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367257#comment-15367257
] 

stack commented on HBASE-16176:
-------------------------------

This last patch, 003, includes fix for the parent scan problem. Here is the commit message:

{code}
 TimeRangeTracker as point of contention when many threads reading a StoreFile
 Fixes HBASE-16074 ITBLL fails, reports lost big or tiny families broken
 scanning because of a side effect of a clean up in  HBASE-15650 to make
 TimeRange construction consistent exposed a latent issue in
 TimeRange#compare. See HBASE-16074 for more detail.
Also change HFile Writer constructor so we pass in the TimeRangeTracker, if one,
on construction rather than set later (the flag and reference were not volatile
so could have made for issues in concurrent case). And make sure the construction
of a TimeRange from a TimeRangeTracer on open of an HFile Reader never makes a
bad minimum value, one that would preclude us reading any values from a file
(set min to 0)
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
 Call through to next constructor (if minStamp was 0, we'd skip setting
 allTime=true). Add asserts that timestamps are not < 0 cos it messes
 us up if they are (we already were checking for < 0 on construction but
 assert passed in timestamps are not < 0).
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
 Add constructor override that takes a TimeRangeTracker (set when flushing
 but not when compacting)
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
 Add override creating an HFile in tmp that takes a TimeRangeTracker
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
 Add override for HFile Writer that takes a TimeRangeTracker Take it on
 construction instead of having it passed by a setter later (flags and
 reference set by the setter were not volatile... could have been prob
 in concurrent case)
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
 Log WARN if bad initial TimeRange value (and then 'fix' it)
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
 A few tests to prove serialization works as expected and that we'll get a bad min if not
constructed properly.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
 Handle OLDEST_TIMESTAMP explictly. Don't expect TimeRange to do it.
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
 Refactor from junit3 to junit4 and add test for this weird case.
{code}

> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention
when many threads reading a StoreFile
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-16176
>                 URL: https://issues.apache.org/jira/browse/HBASE-16176
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: stack
>            Assignee: stack
>             Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
>         Attachments: HBASE-16176.branch-1.3.001.patch, HBASE-16176.branch-1.3.002.patch,
HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650 "Remove
TimeRangeTracker as point of contention when many threads reading a StoreFile". Lets get them
in. Here are the changes:
> {code}
>   6      Change HFile Writer constructor so we pass in the TimeRangeTracker, if one,
>   7      on construction rather than set later (the flag and reference were not
>   8      volatile so could have made for issues in concurrent case) 2. Make sure the
>   9      construction of a TimeRange from a TimeRangeTracer on open of an HFile Reader
>  10      never makes a bad minimum value, one that would preclude us reading any
>  11      values from a file (add a log and set min to 0)
>  12     M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
>  13      Call through to next constructor (if minStamp was 0, we'd skip setting allTime=true)
>  14     M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
>  15      Add constructor override that takes a TimeRangeTracker (set when flushing but
>  16      not when compacting)
>  17     M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
>  18      Add override creating an HFile in tmp that takes a TimeRangeTracker
>  19     M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
>  20      Add override for HFile Writer that takes a TimeRangeTracker
>  21      Take it on construction instead of having it passed by a setter later (flags
>  22      and reference set by the setter were not volatile... could have been prob
>  23      in concurrent case)
>  24     M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
>  25      Log WARN if bad initial TimeRange value (and then 'fix' it)
>  26     M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
>  27      A few tests to prove serialization works as expected and that we'll get a bad
min if
>  28      not constructed properly.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message