hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13588) ExportSnapshot should checksum data transferred
Date Sun, 03 May 2015 06:02:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525681#comment-14525681
] 

Matteo Bertozzi commented on HBASE-13588:
-----------------------------------------

the problem with this patch, as you may see from the test failure, 
is that if we release it with the checksum on, it will always fail when exporting to a different
filesystem
e.g. hdfs to local (as in the test), hdfs to s3, hdfs to an hdfs with a different checksum
conf and so on. each fs implementation/configuration will have a different checksum. 


> ExportSnapshot should checksum data transferred
> -----------------------------------------------
>
>                 Key: HBASE-13588
>                 URL: https://issues.apache.org/jira/browse/HBASE-13588
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.0.1, 0.98.13
>            Reporter: Mubeen Afzal
>            Assignee: Mubeen Afzal
>         Attachments: HBASE-13588.patch
>
>
> We sent quite a bit of data using ExportSnapshot and noticed some corruption.  We use
checksums to verify if we have transferred a particular file but never do any verification
after the file has been transferred.  I propose that we do a checksum upon completing file
transfer and if the checksums do not match up, I propose we fail by throwing an IOException.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message