hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6604) Disk space leak with shortcircuit
Date Fri, 27 Jun 2014 22:53:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe updated HDFS-6604:
---------------------------------------

    Attachment: HDFS-6604.001.patch

The CacheCleaner thread is supposed to time out replicas that haven't been used for a while
(5 minutes by default).  It looks like this thread has not been timing out the non-mmapped
entries as it should, due to a typo.  This patch should fix it.

We probably should also have the DN notify the client when a block that the client is reading
is unlinked.  It could do this via the existing shared memory segment code, in the same way
we handle uncaching now.

> Disk space leak with shortcircuit
> ---------------------------------
>
>                 Key: HDFS-6604
>                 URL: https://issues.apache.org/jira/browse/HDFS-6604
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.4.0
>         Environment: Centos 6.5 and distribution Hortonworks Data Platform v2.1
>            Reporter: Giuseppe Reina
>            Priority: Critical
>         Attachments: HDFS-6604.001.patch
>
>
> When HDFS shortcircuit is enabled, the file descriptors of the deleted HDFS blocks are
kept open until the cache is full. This prevents the operating system to free the space on
disk. 
> More details on the [mailing list thread|http://mail-archives.apache.org/mod_mbox/hbase-user/201406.mbox/%3CCAPjB-CA3RV=SLhUHwUE5CV3PC4+rFfz10-TkYdBfS9RT2DeKPw@mail.gmail.com%3E]



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message