hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6659) Create a Block List
Date Mon, 11 Aug 2014 15:18:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092871#comment-14092871

Daryn Sharp commented on HDFS-6659:

I'd suggest trying to create a subclass of {{ChunkedArrayList}} that implements the custom
hole tracking you need.

> Create a Block List
> -------------------
>                 Key: HDFS-6659
>                 URL: https://issues.apache.org/jira/browse/HDFS-6659
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 2.4.1
>            Reporter: Amir Langer
>            Assignee: Amir Langer
>         Attachments: 0001-BlockList-a-list-of-Blocks-that-saves-memory-by-mana.patch
> BlockList - An efficient array based list that can extend its capacity with two main
> 1. Gaps (result of remove operations) are managed internally without the need for extra
memory - We create a linked list of gaps by using the array index as references + An int to
the head of the gaps list. In every insert operation, we first use any available gap before
extending the array.
> 2. Array extension is done by chaining different arrays, not by allocating a larger array
and copying all its data across. This is a lot less heavy in terms of latency for that particular
call. It also avoids having large amount of contiguous heap space and so behaves nicer with
garbage collection.

This message was sent by Atlassian JIRA

View raw message