hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17924) Consider sorting the row order when processing multi() ops before taking rowlocks
Date Thu, 20 Apr 2017 14:17:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976766#comment-15976766
] 

Hadoop QA commented on HBASE-17924:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 6s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 26s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 1s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 25s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 14s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 30s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 57m 32s {color}
| {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5
2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m 2s {color} | {color:red}
hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s {color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 173m 26s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 34s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 257m 24s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  Dead store to mutationActionMap in org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(ClientProtos$RegionActionResult$Builder,
Region, OperationQuota, List, CellScanner)  At RSRpcServices.java:org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(ClientProtos$RegionActionResult$Builder,
Region, OperationQuota, List, CellScanner)  At RSRpcServices.java:[line 886] |
|  |  org.apache.hadoop.hbase.wal.WALSplitter$MutationReplay defines equals and uses Object.hashCode()
 At WALSplitter.java:Object.hashCode()  At WALSplitter.java:[lines 2305-2308] |
| Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshot |
|   | hadoop.hbase.snapshot.TestMobExportSnapshot |
|   | hadoop.hbase.snapshot.TestMobSecureExportSnapshot |
| Timed out junit tests | org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster |
|   | org.apache.hadoop.hbase.master.TestWarmupRegion |
|   | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12864224/HBASE-17924.v3.patch
|
| JIRA Issue | HBASE-17924 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux 00c155f969ac 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b1ef8dd |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/6512/artifact/patchprocess/new-findbugs-hbase-server.html
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6512/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-HBASE-Build/6512/artifact/patchprocess/patch-unit-hbase-server.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6512/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6512/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Consider sorting the row order when processing multi() ops before taking rowlocks
> ---------------------------------------------------------------------------------
>
>                 Key: HBASE-17924
>                 URL: https://issues.apache.org/jira/browse/HBASE-17924
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0, 1.1.8
>            Reporter: Andrew Purtell
>            Assignee: Allan Yang
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17924.patch, HBASE-17924.v0.patch, HBASE-17924.v2.patch, HBASE-17924.v3.patch
>
>
> When processing a batch mutation, we take row locks in whatever order the mutations were
added to the multi op by the client.
>  
> {noformat}
> RSRpcServices#multi -> RSRpcServices#mutateRows -> HRegion#mutateRow -> HRegion#mutateRowsWithLocks
-> HRegion#processRowsWithLocks
> {noformat}
> Or
> {noformat}
> RSRpcServices#multi -> RSRpcServices#doNonAtomicRegionMutation ->
>       HRegion#get 
>     | HRegion#append 
>     | HRegion#increment 
>     | HRegionServer#doBatchOp -> HRegion#batchMutate -> HRegion#doMiniBatchMutation
> {noformat}
>  
> multi() is fed by client APIs that accept a RowMutations object containing actions for
multiple rows. The container for ops inside RowMutations is an ArrayList, which doesn't change
the ordering of objects added to it. The protobuf implementation of the messages for multi
ops do not reorder the list of actions. When processing multi ops we iterate over the actions
in the order rehydrated from protobuf.
> We should discuss sorting the order of ops by row key when processing multi() ops before
taking row locks. Does this make lock ordering more predictable for server side operations?
Yes, but potentially surprising for the client, right? Is there any legitimate reason we should
take locks out of row key sorted order because the client has structured the request as such?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message