hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15264) Implement a fan out HDFS OutputStream
Date Wed, 17 Feb 2016 03:07:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149763#comment-15149763

Duo Zhang commented on HBASE-15264:

There are several advantages

1. The fan out mechanism. This will reduce the latency.
2. The asynchronous WAL could also run in the same EventLoop, we could just call write and
flush inside the EventLoop thread, so generally we only have one thread to do all the things.
{{FSHLog}} have one disruptor consumer thread and five sync threads default.
3. Fail-fast when connection to datanode error. The WAL implementation could open new writer
ASAP. The recovery mentioned above is only used to make closing a little faster, does not
intend to recover the writer.
4. We may benefit from the {{ByteBufAllocator}} of netty.

And the stream is designed for implementing WAL, so the idea itself has no obvious downsides.
And the current implementation, too many reflections maybe a problem...


> Implement a fan out HDFS OutputStream
> -------------------------------------
>                 Key: HBASE-15264
>                 URL: https://issues.apache.org/jira/browse/HBASE-15264
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>         Attachments: HBASE-15264-v1.patch, HBASE-15264-v2.patch, HBASE-15264.patch

This message was sent by Atlassian JIRA

View raw message