hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11667) Block Storage:Handling flushing of incomplete block id buffers during shutdown
Date Thu, 20 Apr 2017 19:57:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977349#comment-15977349
] 

Chen Liang edited comment on HDFS-11667 at 4/20/17 7:56 PM:
------------------------------------------------------------

Hi Mukul,

Thanks for the reply! #2 makes sense to me, an interesting scenario.

For #1, I got the point of
bq. This is done to make sure that if the directory doesn't exist then they should be created
when CblockCache is starting
But since the code is
{code}if (!logDir.exists() && !logDir.mkdirs()) {code}
If the directory already exists, nothings happens here. So I was wondering, when the directory
already exists, *and it is not empty, with probably some old dirtylog files in it already*,
will there be any potential issue? say, will it overwrite the old dirtylog files by chance?
I'm under the impression that it should not, partly given we have code that requeues dirtylog
files on restart. In this case, we might want to avoid touching old dirtylogs for this particular
JIRA. (although I haven't verified that these are the same directory or something, please
correct me if I made any statement wrong here.)


was (Author: vagarychen):
Hi Mukul,

Thanks for the reply! #2 makes sense to me, an interesting scenario.

For #1, I got the point of
bq. This is done to make sure that if the directory doesn't exist then they should be created
when CblockCache is starting
But since the code is
{code}if (!logDir.exists() && !logDir.mkdirs()) {code}
If the directory already exists, nothings happens here. So I was wondering, when the directory
already exists, *and it is not empty, with probably some old dirtylog files in it already*,
will there be any potential issue? say, will it pick up the old dirtylog files? I'm under
the impression that it should not, given we already have code that requeues dirtylog files
on restart. In this case, we might want to avoid picking up old dirtylogs for this particular
JIRA. (although I haven't verified that these are the same directory or something, please
correct me if I made any statement wrong here.)

> Block Storage:Handling flushing of incomplete block id buffers during shutdown
> ------------------------------------------------------------------------------
>
>                 Key: HDFS-11667
>                 URL: https://issues.apache.org/jira/browse/HDFS-11667
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>         Attachments: HDFS-11667-HDFS-7240.001.patch, HDFS-11667-HDFS-7240.002.patch
>
>
> Currently, whenever the cache shutdown not, zero blocks are written to DirtyLog. This
change will ensure that only required number of blocks are written to the DirtyLog.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message