hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11667) Block Storage:Handling flushing of incomplete block id buffers during shutdown
Date Wed, 19 Apr 2017 18:04:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15975190#comment-15975190
] 

Chen Liang commented on HDFS-11667:
-----------------------------------

Thanks [~msingh] for working on this! Looks good to me overall, only a few question, to make
sure we are on the same page:

In {{AsyncBlockWriter.java}}:

1. It appears to me that the newly added method
{code}
public void start() throws IOException {
  ...
}
{code}
is to make sure the log dir does exist before we try to access it in {{writeBlockBufferToFile}}.
What will happen if this directory does exist, and there are files already, under this directory
also (etc. from some unclean previous shutdown)? Are we then running into the case where both
the new dirty log file and old one exist?

2. In {{writeBlockBufferToFile}}, there is:
{code}
    // If there is nothing written to blockId buffer,
    // then skip flushing of blockId buffer
    if (blockBuffer.position() == 0) {
      return;
    }
{code}
I'm wondering, in what scenario will we get such an empty blockBuffer?

> Block Storage:Handling flushing of incomplete block id buffers during shutdown
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-11667
>                 URL: https://issues.apache.org/jira/browse/HDFS-11667
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11667-HDFS-7240.001.patch, HDFS-11667-HDFS-7240.002.patch
>
>
> Currently, whenever the cache shutdown not, zero blocks are written to DirtyLog. This
change will ensure that only required number of blocks are written to the DirtyLog.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message