hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13588) ExportSnapshot should checksum data transferred
Date Tue, 05 May 2015 00:43:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527648#comment-14527648

Matteo Bertozzi commented on HBASE-13588:

{quote}I see a similar filechecksum compare prior to exporting snapshot without any checks
on file system. {quote}
correct, we rewrite the file if the checksums are different. which is not ideal but at least
you end up with the correct result.

{quote}I'm proposing making checksum compare optional with default <false>{quote}
that sounds good, we need the default false to avoid breaking existing scripts running export

{quote}And just logging a warn if those are different when enabled. what do you think?{quote}
not sure about this one, an exception will be fine with me since the default is false. if
you are asking for the checksum post copy you probably want a failure. but I don't have a
strong opinion on this. maybe another flag? "fail.if.checksum.failed" and in case false just
log. just to provide the user both options? again, no strong opinion. your call.

> ExportSnapshot should checksum data transferred
> -----------------------------------------------
>                 Key: HBASE-13588
>                 URL: https://issues.apache.org/jira/browse/HBASE-13588
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.0.1, 0.98.13
>            Reporter: Mubeen Afzal
>            Assignee: Mubeen Afzal
>         Attachments: HBASE-13588.patch
> We sent quite a bit of data using ExportSnapshot and noticed some corruption.  We use
checksums to verify if we have transferred a particular file but never do any verification
after the file has been transferred.  I propose that we do a checksum upon completing file
transfer and if the checksums do not match up, I propose we fail by throwing an IOException.

This message was sent by Atlassian JIRA

View raw message