hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singhi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
Date Sat, 21 Nov 2015 13:16:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020457#comment-15020457
] 

Ashish Singhi commented on HBASE-13153:
---------------------------------------

Total check style warnings found in patch by checkstyle_report.py are:
{noformat}
$ dev-support/checkstyle_report.py trunkCheckstyle.xml patchCheckstyle.xml 
hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java	85	87
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
7	11
hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationStateZKBase.java
3	4
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java	232	234
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALReplicationCallable.java
0	2
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicationCallable.java
0	4
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/DefaultSourceFSConfigurationProvider.java
0	1
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/SourceFSConfigurationProvider.java
0	2
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
2	3
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationHFileCleaner.java
0	6
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java
1	2
{noformat}
but Hadoop QA something has something else to say!

Out of all those I have not fixed below the one's:

ZKUtil.java (2)
{noformat}
<error line="1926" severity="error" message="'ReplicationStateZKBase' have incorrect indentation
level 8, expected level should be 10." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
<error line="1927" severity="error" message="'ReplicationStateZKBase' have incorrect indentation
level 8, expected level should be 10." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

ReplicationQueuesZKImpl.java (1)
{noformat}
<error line="456" severity="error" message="'HConstants' have incorrect indentation level
8, expected level should be 10." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

ReplicationStateZKBase.java (1)
{noformat}
<error line="81" severity="error" message="'ZOOKEEPER_ZNODE_REPLICATION_HFILE_REFS_DEFAULT'
have incorrect indentation level 6, expected level should be 8." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

RSRpcServices.java (2)
{noformat}
<error line="1804" severity="error" message="'request' have incorrect indentation level
10, expected level should be 12." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
<error line="1805" severity="error" message="'request' have incorrect indentation level
10, expected level should be 12." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

WALReplicationCallable.java (1)
{noformat}
<error line="22" severity="error" message="Wrong order for 'org.apache.hadoop.hbase.TableName'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
{noformat}

HFileReplicationCallable.java (4)
{noformat}
<error line="32" severity="error" message="Wrong order for 'org.apache.commons.logging.Log'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
<error line="55" severity="error" message="Wrong order for 'com.google.common.util.concurrent.ThreadFactoryBuilder'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
<error line="355" severity="error" message="'e' have incorrect indentation level 12, expected
level should be 14." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
<error line="363" severity="error" message="'e1' have incorrect indentation level 14, expected
level should be 16." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

DefaultSourceFSConfigurationProvider.java (1)
{noformat}
<error line="19" severity="error" message="Wrong order for 'org.apache.commons.logging.Log'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
{noformat}

SourceFSConfigurationProvider.java (1)
{noformat}
<error line="15" severity="error" message="Wrong order for 'org.apache.hadoop.conf.Configuration'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
{noformat}

ReplicationHFileCleaner.java (5)
{noformat}
<error line="18" severity="error" message="Wrong order for 'org.apache.commons.logging.Log'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
<error line="34" severity="error" message="Wrong order for 'com.google.common.base.Predicate'
import." source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/>
<error line="120" severity="error" message="'HConstants' have incorrect indentation level
6, expected level should be 8." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
<error line="121" severity="error" message="'HConstants' have incorrect indentation level
6, expected level should be 8." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
<error line="122" severity="error" message="'HConstants' have incorrect indentation level
6, expected level should be 8." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

TableCfWALEntryFilter.java (1)
{noformat}
<error line="86" severity="error" message="'cell' have incorrect indentation level 12,
expected level should be 14." source="com.puppycrawl.tools.checkstyle.checks.indentation.IndentationCheck"/>
{noformat}

These belongs to two categories,
1. IndentationCheck: The code formatter which we [use|https://github.com/apache/hbase/blob/master/dev-support/hbase_eclipse_formatter.xml]
adds only two spaces for the next line which is continuation of previous line.
2.  ImportOrderCheck: I am not sure what to fix here! I am using eclipse IDE and it by default
organizes the imports in that way as mentioned [here|http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Freference%2Fpreferences%2Fjava%2Fcodestyle%2Fref-preferences-organize-imports.htm]

[~appy] can you throw some light, what can I do here ?

> Bulk Loaded HFile Replication
> -----------------------------
>
>                 Key: HBASE-13153
>                 URL: https://issues.apache.org/jira/browse/HBASE-13153
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: sunhaitao
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0
>
>         Attachments: HBASE-13153-v1.patch, HBASE-13153-v10.patch, HBASE-13153-v11.patch,
HBASE-13153-v12.patch, HBASE-13153-v13.patch, HBASE-13153-v14.patch, HBASE-13153-v15.patch,
HBASE-13153-v16.patch, HBASE-13153-v17.patch, HBASE-13153-v2.patch, HBASE-13153-v3.patch,
HBASE-13153-v4.patch, HBASE-13153-v5.patch, HBASE-13153-v6.patch, HBASE-13153-v7.patch, HBASE-13153-v8.patch,
HBASE-13153-v9.patch, HBASE-13153.patch, HBase Bulk Load Replication-v1-1.pdf, HBase Bulk
Load Replication-v2.pdf, HBase Bulk Load Replication-v3.pdf, HBase Bulk Load Replication.pdf,
HDFS_HA_Solution.PNG
>
>
> Currently we plan to use HBase Replication feature to deal with disaster tolerance scenario.But
we encounter an issue that we will use bulkload very frequently,because bulkload bypass write
path, and will not generate WAL, so the data will not be replicated to backup cluster. It's
inappropriate to bukload twice both on active cluster and backup cluster. So i advise do some
modification to bulkload feature to enable bukload to both active cluster and backup cluster



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message