hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Georgi Chalakov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-14520) Block compaction for WASB
Date Sun, 11 Jun 2017 04:15:18 GMT
Georgi Chalakov created HADOOP-14520:
----------------------------------------

             Summary: Block compaction for WASB
                 Key: HADOOP-14520
                 URL: https://issues.apache.org/jira/browse/HADOOP-14520
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs/azure
    Affects Versions: 3.0.0-alpha3
            Reporter: Georgi Chalakov
            Assignee: Georgi Chalakov


Block Compaction for WASB allows uploading new blocks for every hflush/hsync call. When the
number of blocks is above a predefined, configurable value, next hflush/hsync triggers the
block compaction process. Block compaction replaces a sequence of blocks with one block. From
all the sequences with total length less than 4M, compaction chooses the longest one. It is
a greedy algorithm that preserve all potential candidates for the next round. Block Compaction
for WASB increases data durability and allows using block blobs instead of page blobs. By
default, block compaction is disabled. Similar to the configuration for page blobs, the client
needs to specify HDFS folders where block compaction over block blobs is enabled. 





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message