hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15968) New behavior of versions considering mvcc and ts rather than ts only
Date Thu, 20 Jul 2017 11:42:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094566#comment-16094566
] 

Hadoop QA commented on HBASE-15968:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 17s{color} | {color:blue}
Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue}  0m  0s{color} | {color:blue}
rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue}  0m  0s{color} | {color:blue}
Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  0s{color}
| {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m  0s{color}
| {color:green} The patch appears to include 15 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 31s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 36s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 24s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 22s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 43s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  6s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 58s{color} |
{color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 18s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 27s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 14s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 14s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 20s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 45s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 31m 20s{color}
| {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5
2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 48s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 53s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 39s{color} | {color:green}
hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 38s{color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  1m 28s{color} | {color:red}
hbase-shell in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 26s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 53s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.TestBlockCacheReporting |
|   | hadoop.hbase.client.TestShell |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Issue | HBASE-15968 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12878140/HBASE-15968.v06.patch
|
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  rubocop  ruby_lint  |
| uname | Linux 832de49dbdf4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 01db60d |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7732/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7732/artifact/patchprocess/patch-unit-hbase-shell.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/7732/testReport/ |
| modules | C: hbase-client hbase-server hbase-shell U: . |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7732/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> New behavior of versions considering mvcc and ts rather than ts only
> --------------------------------------------------------------------
>
>                 Key: HBASE-15968
>                 URL: https://issues.apache.org/jira/browse/HBASE-15968
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Phil Yang
>            Assignee: Phil Yang
>             Fix For: 3.0.0
>
>         Attachments: HBASE-15968.v05.patch, HBASE-15968.v06.patch, HBASE-15968-v1.patch,
HBASE-15968-v2.patch, HBASE-15968-v3.patch, HBASE-15968-v4.patch
>
>
> In HBase book, we have a section in Versions called "Current Limitations" see http://hbase.apache.org/book.html#_current_limitations
> {quote}
> 28.3. Current Limitations
> 28.3.1. Deletes mask Puts
> Deletes mask puts, even puts that happened after the delete was entered. See HBASE-2256.
Remember that a delete writes a tombstone, which only disappears after then next major compaction
has run. Suppose you do a delete of everything ⇐ T. After this you do a new put with a timestamp
⇐ T. This put, even if it happened after the delete, will be masked by the delete tombstone.
Performing the put will not fail, but when you do a get you will notice the put did have no
effect. It will start working again after the major compaction has run. These issues should
not be a problem if you use always-increasing versions for new puts to a row. But they can
occur even if you do not care about time: just do delete and put immediately after each other,
and there is some chance they happen within the same millisecond.
> 28.3.2. Major compactions change query results
> …​create three cell versions at t1, t2 and t3, with a maximum-versions setting of
2. So when getting all versions, only the values at t2 and t3 will be returned. But if you
delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction
has run, such behavior will not be the case anymore…​ (See Garbage Collection in Bending
time in HBase.)
> {quote}
> These limitations result from the current implementation on multi-versions: we only consider
timestamp, no matter when it comes; we will not remove old version immediately if there are
enough number of new versions. 
> So we can get a stronger semantics of versions by two guarantees:
> 1, Delete will not mask Put that comes after it.
> 2, If a version is masked by enough number of higher versions (VERSIONS in cf's conf),
it will never be seen any more.
> Some examples for understanding:
> (delete t<=3 means use Delete.addColumns to delete all versions whose ts is not greater
than 3, and delete t3 means use Delete.addColumn to delete the version whose ts=3)
> case 1: put t2 -> put t3 -> delete t<=3 -> put t1, and we will get t1 because
the put is after delete.
> case 2: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3, and we will always
get t2 no matter if there is a major compaction, because t1 is masked when we put t3 so t1
will never be seen.
> case 3: maxversion=2, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3,
and we will get nothing.
> case 4: maxversion=3, put t1 -> put t2 -> put t3 -> delete t2 -> delete t3,
and we will get t1 because it is not masked.
> case 5: maxversion=2, put t1 -> put t2 -> put t3 -> delete t3 -> put t1,
and we can get t3+t1 because when we put t1 at second time it is the 2nd latest version and
it can be read.
> case 6:maxversion=2, put t3->put t2->put t1, and we will get t3+t2 just like what
we can get now, ts is still the key of versions.
> Different VERSIONS may result in different results even the size of result is smaller
than VERSIONS(see case 3 and 4).  So Get/Scan.setMaxVersions will be handled at end after
we read correct data according to CF's  VERSIONS setting.
> The semantics is different from the current HBase, and we may need more logic to support
the new semantic, so it is configurable and default is disabled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message