hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6166) Improve PureJavaCrc32
Date Wed, 19 Aug 2009 17:29:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745143#action_12745143
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-6166:
------------------------------------------------

> Because the '8' variants shift to one byte at a time if the input is less than 8 bytes,
they perform worse than the old PureJavaCrc32 at the 4 byte to 7 byte level. Is this important?
It would be useful to know how often the crc code is called on small byte chunks. We can get
this to near PureJavaCrc32 speeds for 4 byte sizes if we add a four byte at a time block to
8_8d.

I tried this before but it did not help.  BTW, the while-loop in the middle should be an if-statement
since we have len <= 7.

> Improve PureJavaCrc32
> ---------------------
>
>                 Key: HADOOP-6166
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6166
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt, c6166_20090722_benchmark_64VM.txt,
c6166_20090727.patch, c6166_20090728.patch, c6166_20090810.patch, c6166_20090811.patch, graph.r,
graph.r, Rplots-laptop.pdf, Rplots-nehalem32.pdf, Rplots-nehalem64.pdf, Rplots.pdf, Rplots.pdf,
Rplots.pdf
>
>
> Got some ideas to improve CRC32 calculation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message