hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files
Date Tue, 14 Apr 2015 17:59:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494495#comment-14494495
] 

Zhe Zhang commented on HDFS-7889:
---------------------------------

As we go deeper in testing, I've found the following issues and questions about {{StripedDataStreamer#locateFollowingBlock}}:
# {{hasCommittedBlock}} is initially {{false}}. But once becoming {{true}}, it will never
be {{false}} again. What's the purpose of this flag?
# Why are we always polling the first located block, instead of the i_th?
{code}
        for (int i = 1; i < HdfsConstants.NUM_DATA_BLOCKS; i++) {
          try {
            LocatedBlock finishedLocatedBlock = stripedBlocks.get(0).poll(30,
                TimeUnit.SECONDS);
{code}
# Why do we need the above loop at all? Shouldn't we always commit {{block.getNumBytes() *
NUM_DATA_BLOCKS}}?

Let's clarify these questions here and see if we need to do a follow-on for this logic.

> Subclass DFSOutputStream to support writing striping layout files
> -----------------------------------------------------------------
>
>                 Key: HDFS-7889
>                 URL: https://issues.apache.org/jira/browse/HDFS-7889
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Li Bo
>            Assignee: Li Bo
>             Fix For: HDFS-7285
>
>         Attachments: HDFS-7889-001.patch, HDFS-7889-002.patch, HDFS-7889-003.patch, HDFS-7889-004.patch,
HDFS-7889-005.patch, HDFS-7889-006.patch, HDFS-7889-007.patch, HDFS-7889-008.patch, HDFS-7889-009.patch,
HDFS-7889-010.patch, HDFS-7889-011.patch, HDFS-7889-012.patch, HDFS-7889-013.patch, HDFS-7889-014.patch
>
>
> After HDFS-7888, we can subclass  {{DFSOutputStream}} to support writing striping layout
files. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message