hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13588) ExportSnapshot should checksum data transferred
Date Sun, 03 May 2015 14:09:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525850#comment-14525850

Ted Yu commented on HBASE-13588:

We have FSHDFSUtils#isSameHdfs().
Maybe a similar utility method can be added without the last check in FSHDFSUtils#isSameHdfs()
which is specific to DistributedFileSystem.

Checksums can be compared for FileSystem's of the same type.

> ExportSnapshot should checksum data transferred
> -----------------------------------------------
>                 Key: HBASE-13588
>                 URL: https://issues.apache.org/jira/browse/HBASE-13588
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.0.1, 0.98.13
>            Reporter: Mubeen Afzal
>            Assignee: Mubeen Afzal
>         Attachments: HBASE-13588.patch
> We sent quite a bit of data using ExportSnapshot and noticed some corruption.  We use
checksums to verify if we have transferred a particular file but never do any verification
after the file has been transferred.  I propose that we do a checksum upon completing file
transfer and if the checksums do not match up, I propose we fail by throwing an IOException.

This message was sent by Atlassian JIRA

View raw message