hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3515) Protocol changes to allow appending to the last partial crc chunk of a file
Date Mon, 09 Jun 2008 22:23:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603712#action_12603712
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3515:
------------------------------------------------

I think we could have a third option.

3. Client know the original file size.  It can add padding for the first chunk, so that crc
boundary will be aligned with the original block.  Then, the primary datanode only has to
compute the crc for the first chunk but not the afterward.

For example, suppose the original file size is 3123 bytes, block size is 2000 bytes and crc
chunk size is 500 bytes.  The last block original has 1123 bytes.  The client will add 123
bytes prefix padding for the first chunk.

> Protocol changes to allow appending to the last partial  crc chunk of a file
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-3515
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3515
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> To support "appending" to an existing file, we need the ability to append data to the
last partial crc chunk of the file. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message