hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Marc Spaggiari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13985) Add configuration to skip validating HFile format when bulk loading millions of HFiles
Date Mon, 29 Jun 2015 08:09:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605249#comment-14605249
] 

Jean-Marc Spaggiari commented on HBASE-13985:
---------------------------------------------

Can we print a warning when this parameter is used? Like:
"You are skipping HFiles validation, it might cause this or that issue of files are not correct.
If you fail to read data from your table after using this option, consider removing the files
and push again without the option"? 

It's just a very bad example, but just something to say it's dangerous?

> Add configuration to skip validating HFile format when bulk loading millions of HFiles
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-13985
>                 URL: https://issues.apache.org/jira/browse/HBASE-13985
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.98.13
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>            Priority: Minor
>              Labels: regionserver
>             Fix For: 0.98.14
>
>         Attachments: HBASE-13985-v2.patch, HBASE-13985.patch
>
>
> When bulk loading millions of HFile into one HTable, checking HFile format is the most
time-consuming phase. Maybe we could use a parallel mechanism to increase the speed, but when
it comes to millions of HFiles, it may still cost dozens of minutes. So I think it's necessary
to add an option for advanced user to bulkload without checking HFile format at all. 
> Of course, the default value of this option should be true.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message