hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11714) Newly added NN storage directory won't get initialized and cause space exhaustion
Date Fri, 28 Apr 2017 17:42:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989212#comment-15989212
] 

Daryn Sharp commented on HDFS-11714:
------------------------------------

In FSImage, please correct the inventive word: "tirggered". :)  It would be nice if initNewDirs
was encapsulated in FSImage since it's not something the servlet should need to know about
but I'm not familiar enough with the code to know what will break.  Up to you.

And we need the branch-2 patch.

> Newly added NN storage directory won't get initialized and cause space exhaustion
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-11714
>                 URL: https://issues.apache.org/jira/browse/HDFS-11714
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.3
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Critical
>         Attachments: HDFS-11714.trunk.patch
>
>
> When an empty namenode storage directory is detected on normal NN startup, it may not
be fully initialized. The new directory is still part of "in-service" NNStrage and when a
checkpoint image is uploaded, a copy will also be written there.  However, the retention manager
won't be able to purge old files since it is lacking a VERSION file.  This causes fsimages
to pile up in the directory.  With a big name space, the disk will be filled in the order
of days or weeks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message