hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Klimontovich (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HBASE-2280) HFileOutputFormat writes output to "unsafe" directory
Date Wed, 03 Mar 2010 13:22:27 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vladimir Klimontovich resolved HBASE-2280.
------------------------------------------

    Resolution: Duplicate

Duplicate of HBASE-2063

> HFileOutputFormat writes output to "unsafe" directory
> -----------------------------------------------------
>
>                 Key: HBASE-2280
>                 URL: https://issues.apache.org/jira/browse/HBASE-2280
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3
>            Reporter: Vladimir Klimontovich
>            Priority: Critical
>             Fix For: 0.20.5, 0.21.0, 0.22.0
>
>
> HFileOutputFormat writes data direct to output folder. It's incorrect as failed (or killed,
or interrupted) reducers leaves inconsistent files in output folder.
> The convinient way to ouput data from OutputFormat is to use "working directory". The
content of this directory is being moved to output directory at the end of reducer process
if only reducer succeeded (this process is called "output commit" or "atomic commit").
> If means that instead of
>  final Path outputdir = FileOutputFormat.getOutputPath(context);
> hbase should use
>  final Path outputdir = FileOutputFormat.getWorkOutputPath(context);

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message