Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 16 Sep 2015 20:09:49 +0000 (UTC)
From: "Lars Hofhansl (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12860915.1441090816000.359125.1442434189277@Atlassian.JIRA>
In-Reply-To: <JIRA.12860915.1441090816000@Atlassian.JIRA>
References: <JIRA.12860915.1441090816000@Atlassian.JIRA>
 <JIRA.12860915.1441090816226@arcas>
Subject: [jira] [Commented] (HBASE-14352) Replication is terribly slow with
 WAL compression
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791052#comment-14791052 ] 

Lars Hofhansl commented on HBASE-14352:
---------------------------------------

I took a look at the code some weeks back. The problem immediately jumps out... At the source we constantly reset the read position into the current WAL. With compression it means we have start from a point where the compression dictionary is written. That is very expensive.

We have to do that in order to be sure we'll see the edits in the current block being written.
So I don't see immediately a way out of it. Perhaps we simply tail until we reach the end of a file. And that case we'll try one more time with a reset, and only declare the WAL done when that is done.

> Replication is terribly slow with WAL compression
> -------------------------------------------------
>
>                 Key: HBASE-14352
>                 URL: https://issues.apache.org/jira/browse/HBASE-14352
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.13
>            Reporter: Abhishek Singh Chouhan
>         Attachments: age_of_last_shipped.png, size_of_log_queue.png
>
>
> For the same load, replication with WAL compression enabled is almost 6x slower than with compression turned off. Age of last shipped operation is also correspondingly much higher when compression is turned on. 
> By observing Size of log queue we can see that it is taking too much time for the queue to clear up.
> Attaching corresponding graphs.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)