hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3514) Reduce seeks during shuffle, by inline crcs
Date Fri, 01 Aug 2008 17:54:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619084#action_12619084
] 

Raghu Angadi commented on HADOOP-3514:
--------------------------------------

My nit : 

{{ChecksumInputStream}} and {{ChecksumOutputStream}} are in hadoop.io package seem to imply
they are more general purpose checksum streams. But these don't seem so.. these are utilities
for dealing with another stream that has 'checksum per record'. I would recommend 'Record'
some where in the name of these classes or moving them to MR. 

> Reduce seeks during shuffle, by inline crcs
> -------------------------------------------
>
>                 Key: HADOOP-3514
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3514
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.18.0
>            Reporter: Devaraj Das
>            Assignee: Jothi Padmanabhan
>             Fix For: 0.19.0
>
>         Attachments: hadoop-3514-v1.patch, hadoop-3514-v2.patch, hadoop-3514.patch
>
>
> The number of seeks can be reduced by half in the iFile if we move the crc into the iFile
rather than having a separate file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message