hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files
Date Wed, 08 Apr 2015 00:05:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484383#comment-14484383
] 

Zhe Zhang commented on HDFS-7889:
---------------------------------

bq. in encode(), just copy the references of buffer in order to call raw coder’s encode
function
I see. We don't need to worry about it then.

bq. Different streamers share the same ExtendedBlock, so I think we can use any of them.
Well some streamers could fall behind streamers\[0\]. But I agree the {{ExtendedBlock}} at
streamers\[0\] should be the final block of the file.

bq. Not very clear about your meaning. What hook? DFSStripedOutputStream is created in previous
sentence os = fs.create(TestPath);
Basically, why do we need to do the following? {{fs.create(TestPath)}} should give a striped
output stream if the file is in an erasure coding zone. 
{code}
      DFSStripedOutputStream dos = (DFSStripedOutputStream) Whitebox.getInternalState(os,
          "wrappedStream");
{code}
You can use {{fs.getClient().createErasureCodingZone("/");}} to make all files erasure coded.

bq. Could we set the limit to 90 or 100 chars? 80 chars seem a little short, and there’re
many lines with more than 80 chars(<100) in current code.
As far as I was told 80 characters is the standard. In test codes I've seen some long lines
but rarely in main codes. 

> Subclass DFSOutputStream to support writing striping layout files
> -----------------------------------------------------------------
>
>                 Key: HDFS-7889
>                 URL: https://issues.apache.org/jira/browse/HDFS-7889
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>         Attachments: HDFS-7889-001.patch, HDFS-7889-002.patch, HDFS-7889-003.patch, HDFS-7889-004.patch,
HDFS-7889-005.patch, HDFS-7889-006.patch, HDFS-7889-007.patch, HDFS-7889-008.patch, HDFS-7889-009.patch
>
>
> After HDFS-7888, we can subclass  {{DFSOutputStream}} to support writing striping layout
files. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message