hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Georgi Chalakov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14520) Block compaction for WASB
Date Mon, 12 Jun 2017 22:06:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Georgi Chalakov updated HADOOP-14520:
    Status: In Progress  (was: Patch Available)

> Block compaction for WASB
> -------------------------
>                 Key: HADOOP-14520
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14520
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/azure
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Georgi Chalakov
>            Assignee: Georgi Chalakov
>         Attachments: HADOOP-14520-01.patch, HADOOP-14520-01-test.txt
> Block Compaction for WASB allows uploading new blocks for every hflush/hsync call. When
the number of blocks is above a predefined, configurable value, next hflush/hsync triggers
the block compaction process. Block compaction replaces a sequence of blocks with one block.
From all the sequences with total length less than 4M, compaction chooses the longest one.
It is a greedy algorithm that preserve all potential candidates for the next round. Block
Compaction for WASB increases data durability and allows using block blobs instead of page
blobs. By default, block compaction is disabled. Similar to the configuration for page blobs,
the client needs to specify HDFS folders where block compaction over block blobs is enabled.

> Results for HADOOP-14520-01.patch
> Tests run: 704, Failures: 0, Errors: 0, Skipped: 119

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message