hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18027) HBaseInterClusterReplicationEndpoint should respect RPC size limits when batching edits
Date Thu, 11 May 2017 04:05:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005853#comment-16005853
] 

Hadoop QA commented on HBASE-18027:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 2 new or modified test files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 58s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 54s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 6s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color}
| {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s {color} | {color:red}
The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 30m 11s {color}
| {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5
2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 58s {color} | {color:red}
hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s {color} |
{color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 12s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 146m 2s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  Should org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$ReplicationEntryList
be a _static_ inner class?  At HBaseInterClusterReplicationEndpoint.java:inner class?  At
HBaseInterClusterReplicationEndpoint.java:[lines 198-216] |
| Failed junit tests | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL
|
|   | hadoop.hbase.replication.TestReplicationEndpoint |
|   | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleAsyncWAL |
| Timed out junit tests | org.apache.hadoop.hbase.master.procedure.TestTableDescriptorModificationFromClient
|
|   | org.apache.hadoop.hbase.TestAcidGuarantees |
|   | org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache |
|   | org.apache.hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12867479/HBASE-18027.patch
|
| JIRA Issue | HBASE-18027 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux 0699854560b0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c5cc81d |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| whitespace | https://builds.apache.org/job/PreCommit-HBASE-Build/6759/artifact/patchprocess/whitespace-eol.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/6759/artifact/patchprocess/new-findbugs-hbase-server.html
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/6759/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-HBASE-Build/6759/artifact/patchprocess/patch-unit-hbase-server.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/6759/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/6759/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> HBaseInterClusterReplicationEndpoint should respect RPC size limits when batching edits
> ---------------------------------------------------------------------------------------
>
>                 Key: HBASE-18027
>                 URL: https://issues.apache.org/jira/browse/HBASE-18027
>             Project: HBase
>          Issue Type: Bug
>          Components: Replication
>    Affects Versions: 2.0.0, 1.4.0, 1.3.1
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 1.4.0, 1.3.2
>
>         Attachments: HBASE-18027.patch
>
>
> In HBaseInterClusterReplicationEndpoint#replicate we try to replicate in batches. We
create N lists. N is the minimum of configured replicator threads, number of 100-waledit batches,
or number of current sinks. Every pending entry in the replication context is then placed
in order by hash of encoded region name into one of these N lists. Each of the N lists is
then sent all at once in one replication RPC. We do not test if the sum of data in each N
list will exceed RPC size limits. This code presumes each individual edit is reasonably small.
Not checking for aggregate size while assembling the lists into RPCs is an oversight and can
lead to replication failure when that assumption is violated.
> We can fix this by generating as many replication RPC calls as we need to drain a list,
keeping each RPC under limit, instead of assuming the whole list will fit in one.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message