hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12533) staging directories are not deleted after secure bulk load
Date Thu, 20 Nov 2014 19:29:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219836#comment-14219836
] 

Enis Soztutar commented on HBASE-12533:
---------------------------------------

So, in the patch instead of calling the bulk load endpoint for prepare and cleanup for every
region, we are just calling them for only the first region in the table if I understand the
patch correctly. It seems strange that we are relying on the client to send the clean up request
rather than keeping lease and do the server side clean up, since if the client goes away,
the staging dir will still be left there forever. We should fix that in another issue perhaps.


Can you add a comment when we are calling the coprocessors to say that we are only calling
the first region. Other than that +1. 

> staging directories are not deleted after secure bulk load
> ----------------------------------------------------------
>
>                 Key: HBASE-12533
>                 URL: https://issues.apache.org/jira/browse/HBASE-12533
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.98.6
>         Environment: CDH5.2 + Kerberos
>            Reporter: Andrejs Dubovskis
>            Assignee: Jeffrey Zhong
>         Attachments: HBASE-12533.patch
>
>
> We using secure bulk load heavily in our environment. And it was working with no problem
during some time. But last week I found that clients hangs while calling *doBulkLoad*
> After some investigation I found that HDFS keeps more than 1,000,000 directories in /tmp/hbase-staging
directory.
> When directory's content was purged the load process runs successfully.
> According the [hbase book|http://hbase.apache.org/book/ch08s03.html#hbase.secure.bulkload]

> {code}
> HBase manages creation and deletion of this directory.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message