hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19024) provide a configurable option to hsync WAL edits to the disk for better durability
Date Mon, 27 Nov 2017 20:24:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16267423#comment-16267423
] 

Hadoop QA commented on HBASE-19024:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 54s{color} | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue}  0m  7s{color} | {color:blue}
Shelldocs was not available. {color} |
| {color:blue}0{color} | {color:blue} xml {color} | {color:blue}  0m  0s{color} | {color:blue}
jrunscript was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  0s{color}
| {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  0s{color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  8s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 13s{color} | {color:red}
root in branch-1.2 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  7s{color} | {color:red}
root in branch-1.2 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 10s{color} | {color:red}
hbase-server in branch-1.2 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 13s{color}
| {color:green} branch-1.2 passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red}  0m 10s{color} | {color:red}
branch has 10 errors when building our shaded downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m  6s{color} | {color:red}
hbase-server in branch-1.2 failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  8s{color} | {color:red}
root in branch-1.2 failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  6s{color} | {color:red}
hbase-server in branch-1.2 failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  6s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m  9s{color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  8s{color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m  7s{color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m  8s{color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m  7s{color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 11s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green}  0m  0s{color}
| {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red}  0m 10s{color} | {color:red}
patch has 10 errors when building our shaded downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 17s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 25s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 34s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 42s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 50s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  0m 57s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m  5s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 14s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 22s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 30s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 38s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 46s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.7.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red}  1m 53s{color} | {color:red}
The patch causes 10 errors with Hadoop v2.7.3. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Skipped patched modules with no Java source: . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m  5s{color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  6s{color} | {color:red}
root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  6s{color} | {color:red}
hbase-server in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  0m  7s{color} | {color:red}
root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m  9s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 40s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:f29f170 |
| JIRA Issue | HBASE-19024 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12899481/HBASE-19024.branch-1.2.004.patch
|
| Optional Tests |  asflicense  shellcheck  shelldocs  javac  javadoc  unit  shadedjars  hadoopcheck
 xml  compile  findbugs  hbaseanti  checkstyle  |
| uname | Linux 15fb8ae9b9a3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-1.2 / ea9d171 |
| maven | version: Apache Maven 3.0.5 |
| Default Java | 1.7.0_151 |
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-mvninstall-root.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-compile-root.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-compile-hbase-server.txt
|
| shellcheck | v0.4.6 |
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-findbugs-hbase-server.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-javadoc-root.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/branch-javadoc-hbase-server.txt
|
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-mvninstall-root.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-compile-root.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-compile-hbase-server.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-compile-root.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-compile-hbase-server.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-findbugs-hbase-server.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-javadoc-root.txt
|
| javadoc | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-javadoc-hbase-server.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/artifact/patchprocess/patch-unit-root.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/testReport/ |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/10045/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



> provide a configurable option to hsync WAL edits to the disk for better durability
> ----------------------------------------------------------------------------------
>
>                 Key: HBASE-19024
>                 URL: https://issues.apache.org/jira/browse/HBASE-19024
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>         Environment: 
>            Reporter: Vikas Vishwakarma
>            Assignee: Harshal Jain
>         Attachments: HBASE-19024.branch-1.2.001.patch, HBASE-19024.branch-1.2.002.patch,
HBASE-19024.branch-1.2.003.patch, HBASE-19024.branch-1.2.004.patch, branch-1.branch-1.patch,
branch-1.v1.branch-1.patch, master.patch, master.v2.patch, master.v3.patch, master.v5.patch,
master.v5.patch, master.v6.patch, master.v9.patch
>
>
> At present we do not have an option to hsync WAL edits to the disk for better durability.
In our local tests we see 10-15% latency impact of using hsync instead of hflush which is
not very high.  
> We should have a configurable option to hysnc WAL edits instead of just sync/hflush which
will call the corresponding API on the hadoop side. Currently HBase handles both SYNC_WAL
and FSYNC_WAL as the same calling FSDataOutputStream sync/hflush on the hadoop side. This
can be modified to let FSYNC_WAL call hsync on the hadoop side instead of sync/hflush. We
can keep the default value to sync as the current behavior and hsync can be enabled based
on explicit configuration.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message