Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C05A0200B49 for ; Tue, 19 Jul 2016 09:34:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BD5F0160A76; Tue, 19 Jul 2016 07:34:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 14807160A89 for ; Tue, 19 Jul 2016 09:34:21 +0200 (CEST) Received: (qmail 4902 invoked by uid 500); 19 Jul 2016 07:34:21 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 4865 invoked by uid 99); 19 Jul 2016 07:34:21 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Jul 2016 07:34:21 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id F2F212C0278 for ; Tue, 19 Jul 2016 07:34:20 +0000 (UTC) Date: Tue, 19 Jul 2016 07:34:20 +0000 (UTC) From: "Phil Yang (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-9465) Push entries to peer clusters serially MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 19 Jul 2016 07:34:22 -0000 [ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-9465: ----------------------------- Attachment: HBASE-9465.pdf HBASE-9465.v2.patch Upload v2 patch and pdf design doc > Push entries to peer clusters serially > -------------------------------------- > > Key: HBASE-9465 > URL: https://issues.apache.org/jira/browse/HBASE-9465 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication > Reporter: Honghua Feng > Assignee: Phil Yang > Attachments: HBASE-9465-v1.patch, HBASE-9465.pdf, HBASE-9465.v2.patch > > > When region-move or RS failure occurs in master cluster, the hlog entries that are not pushed before region-move or RS-failure will be pushed by original RS(for region move) or another RS which takes over the remained hlog of dead RS(for RS failure), and the new entries for the same region(s) will be pushed by the RS which now serves the region(s), but they push the hlog entries of a same region concurrently without coordination. > This treatment can possibly lead to data inconsistency between master and peer clusters: > 1. there are put and then delete written to master cluster > 2. due to region-move / RS-failure, they are pushed by different replication-source threads to peer cluster > 3. if delete is pushed to peer cluster before put, and flush and major-compact occurs in peer cluster before put is pushed to peer cluster, the delete is collected and the put remains in peer cluster > In this scenario, the put remains in peer cluster, but in master cluster the put is masked by the delete, hence data inconsistency between master and peer clusters -- This message was sent by Atlassian JIRA (v6.3.4#6332)