hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingcheng Du (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-12487) Explicitly flush the file name in sweep job
Date Sat, 15 Nov 2014 06:07:34 GMT
Jingcheng Du created HBASE-12487:
------------------------------------

             Summary: Explicitly flush the file name in sweep job
                 Key: HBASE-12487
                 URL: https://issues.apache.org/jira/browse/HBASE-12487
             Project: HBase
          Issue Type: Sub-task
    Affects Versions: hbase-11339
            Reporter: Jingcheng Du
            Assignee: Jingcheng Du
             Fix For: hbase-11339


Currently in the implementation of sweeper, we need to remove the mob files that are not referenced
by hbase any more.
# List the names of all the existing files and write them to a seq file.
# In each reducer, write the visited mob file names to a seq file.
# After the mapreduce is done, remove the files that are existent in step1 but not in step2
(those are the unused/unreferenced files).

Currently the flush of the writer depends on the IOUtils.closeStream(writer), if this close
operation fails silently, the file names won't be written to seq files, some files that are
still referenced by hbase will be archived after the mapreduce is finished.
We should explicitly invoke write.hflush() to flush out the user buffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message