hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12405) WAL accounting by Store
Date Tue, 10 Mar 2015 06:53:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354418#comment-14354418
] 

Hudson commented on HBASE-12405:
--------------------------------

FAILURE: Integrated in HBase-TRUNK #6231 (See [https://builds.apache.org/job/HBase-TRUNK/6231/])
HBASE-12405 WAL accounting by Store (zhangduo: rev 9b53a1c214f87bd0a38b12e8d308e7e3bde6de84)
* hbase-protocol/src/main/protobuf/ClusterStatus.proto
* hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestGetLastFlushedSequenceId.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LastSequenceId.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/RegionLoad.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestPerColumnFamilyFlush.java
* hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
* hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/SplitLogWorkerCoordination.java
* hbase-protocol/src/main/protobuf/RegionServerStatus.proto
* hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZKSplitLogManagerCoordination.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* hbase-protocol/src/main/protobuf/ZooKeeper.proto


> WAL accounting by Store
> -----------------------
>
>                 Key: HBASE-12405
>                 URL: https://issues.apache.org/jira/browse/HBASE-12405
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>    Affects Versions: 2.0.0, 1.1.0
>            Reporter: zhangduo
>            Assignee: zhangduo
>             Fix For: 2.0.0, 1.1.0
>
>         Attachments: HBASE-12405.patch, HBASE-12405_1.patch, HBASE-12405_2.patch, HBASE-12405_3.patch,
HBASE-12405_4.patch, HBASE-12405_5.patch
>
>
> HBASE-10201 has made flush decisions per Store, but has not done enough work on HLog,
so there are two problems:
> 1. We record minSeqId both in HRegion and FSHLog, which is a duplication.
> 2. There maybe holes in WAL accounting.
>     For example, assume family A with sequence id 1 and 3, family B with seqId 2. If
we flush family A, we can only record that WAL before sequence id 1 can be removed safely.
If we do a replay at this point, sequence id 3 will also be replayed which is unnecessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message