hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Bo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout
Date Fri, 13 Feb 2015 03:57:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319537#comment-14319537

Li Bo commented on HDFS-7729:

At first my idea is using a block writer to receive bytes and write to remote datanode,  a
high level class does encode/decode work and manages multiple writers(HDFS-7653,7679,7546).
But this will generate a lot new code(some are duplicated with existing classes). Zhe's design
has the advantage of making the smallest change to current code. If we think the logic of
DFSOutputStream is complex and is not appropriate to add more logic such as striping layout,
I think it's a good idea to split it into more classes.

> Add logic to DFSOutputStream to support writing a file in striping layout 
> --------------------------------------------------------------------------
>                 Key: HDFS-7729
>                 URL: https://issues.apache.org/jira/browse/HDFS-7729
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>         Attachments: Codec-tmp.patch, HDFS-7729-001.patch, HDFS-7729-002.patch, HDFS-7729-003.patch,
HDFS-7729-004.patch, HDFS-7729-005.patch, HDFS-7729-006.patch, HDFS-7729-007.patch, HDFS-7729-008.patch
> If client wants to directly write a file striping layout, we need to add some logic to
DFSOutputStream.  DFSOutputStream needs multiple DataStreamers to write each cell of a stripe
to a remote datanode. 

This message was sent by Atlassian JIRA

View raw message