hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kanaka kumar avvaru (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8626) Reserved RBW space is not released if creation of RBW File fails
Date Thu, 18 Jun 2015 17:30:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592149#comment-14592149
] 

kanaka kumar avvaru commented on HDFS-8626:
-------------------------------------------

Thanks [~kihwal], In that scenario we had only one disk path configured for data node.
Also for information, get availability on separate Java code  test  and df command gives available
space from disk as 100+ GB but data node jmx report shown as 3 GB  after few hours  also.

In order to confirm the leak , I have ensured  through datanode heap dump analysis that reservedRBW
variable in FSVolumeImpl is holding the number equalent to missing 100+ GB.

> Reserved RBW space is not released if creation of RBW File fails
> ----------------------------------------------------------------
>
>                 Key: HDFS-8626
>                 URL: https://issues.apache.org/jira/browse/HDFS-8626
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: kanaka kumar avvaru
>            Assignee: kanaka kumar avvaru
>            Priority: Blocker
>         Attachments: HDFS-8626-01.patch
>
>
> The DataNode reserves disk space for a full block when creating an RBW block and will
release the space when the block is finalized (introduced in HDFS-6898) 
> But if the RBW file creation fails, the reserved space is not released back. 
> In a scenario, when the data node disk is full it causes no space left {{IOException}}.
Eventually even if the disk got cleaned,  the reserved space is not release until the  Data
Node is restarted



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message