hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-928) make checksums optional per FileSystem
Date Mon, 12 Feb 2007 23:32:06 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Hairong Kuang updated HADOOP-928:

    Attachment: checksum.patch

This patch includes two major changes to hadoop file systems:
1. separating the raw FileSystem interface from the ChecksumFileSystem interface.
2. separating checksum generation & checksum checking from FSDataOutputStream & FSDataInputStream

Please review and give me feedback on the changes.

I simply removed all the raw methods from the FileSystem interface. Should I keep them and
mark them as deprecated for the backward compatibility purpose?

> make checksums optional per FileSystem
> --------------------------------------
>                 Key: HADOOP-928
>                 URL: https://issues.apache.org/jira/browse/HADOOP-928
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Doug Cutting
>         Assigned To: Hairong Kuang
>         Attachments: checksum.patch
> Checksumming is currently built into the base FileSystem class.  It should instead be
optional, with each FileSystem implementation electing whether to use the Hadoop-provided
checksum system, or to disable it, or to implement its own custom checksum system.
> To implement this, a ChecksumFileSystem implementation can be provided that wraps another
FileSystem implementation, implementing checksums as in Hadoop's current mandatory implementation
(i.e., as a separate crc file per file that's elided from directory listings).  The 'raw'
FileSystem methods would be removed.  FSDataInputStream and FSDataOutputStream would be made

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message