hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Bo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files
Date Thu, 02 Apr 2015 08:44:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392385#comment-14392385

Li Bo commented on HDFS-7889:

Thanks for Vinayakumar’s careful review.
bq. I didn't understand this part. Why catching InterruptedException?
This is because {{byteArrayManager.newByteArray(cellSize)}} will throw InterruptedException.
I think we don’t need to get cell buffers from {{byteArrayManager}}. Will fix it in new

bq. I think below part of the code never needed. Because BufferOverflowException is thrown
in addToCellBuffer itself.
Right. We need to check before {{ addToCellBuffer ()}}. Will fix it in new patch.

bq.clearCellBuffers(int index) not required, at least as of now.
{{clearCellBuffers(int index)}} makes sure that the cell buffers are reset. If we delete it,
we have to check if the buffers are reset after encoding.

bq. getPaddingBytes
Because we will not pad zero-bytes to incomplete stripe, this function will be deleted later.

bq. StripedDataStreamer constructor
Will remove unrelated comments in new patch.

bq. the position of newStreamForCreate
Both looks good. Adding checks in {{DFSOutputStream#newStreamForCreate}} has the advantage
of less changes to current code. Will discuss with Zhe about it.

> Subclass DFSOutputStream to support writing striping layout files
> -----------------------------------------------------------------
>                 Key: HDFS-7889
>                 URL: https://issues.apache.org/jira/browse/HDFS-7889
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>         Attachments: HDFS-7889-001.patch, HDFS-7889-002.patch, HDFS-7889-003.patch, HDFS-7889-004.patch,
HDFS-7889-005.patch, HDFS-7889-006.patch
> After HDFS-7888, we can subclass  {{DFSOutputStream}} to support writing striping layout

This message was sent by Atlassian JIRA

View raw message