hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12487) Explicitly flush the file name in sweep job
Date Mon, 17 Nov 2014 11:00:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214525#comment-14214525

ramkrishna.s.vasudevan commented on HBASE-12487:

    public void hflush() throws IOException {
      if (out != null) {
Hope you are calling this hflush and expect any IOException thrown from here to be propogated

> Explicitly flush the file name in sweep job
> -------------------------------------------
>                 Key: HBASE-12487
>                 URL: https://issues.apache.org/jira/browse/HBASE-12487
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>    Affects Versions: hbase-11339
>            Reporter: Jingcheng Du
>            Assignee: Jingcheng Du
>             Fix For: hbase-11339
>         Attachments: HBASE-12487.diff
> Currently in the implementation of sweeper, we need to remove the mob files that are
not referenced by hbase any more.
> # List the names of all the existing files and write them to a seq file.
> # In each reducer, write the visited mob file names to a seq file.
> # After the mapreduce is done, remove the files that are existent in step1 but not in
step2 (those are the unused/unreferenced files).
> Currently the flush of the writer depends on the IOUtils.closeStream(writer), if this
close operation fails silently, the file names won't be written to seq files, some files that
are still referenced by hbase will be archived after the mapreduce is finished.
> We should explicitly invoke write.hflush() to flush out the user buffer.

This message was sent by Atlassian JIRA

View raw message