hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10865) Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases
Date Wed, 10 Feb 2016 05:04:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140314#comment-15140314
] 

Masatake Iwasaki commented on HADOOP-10865:
-------------------------------------------

The patch still applies and I agree to add the benchmark. Just a few comments.

{noformat}
Running org.apache.hadoop.util.TestDataChecksum
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 10.174 sec <<< FAILURE!
- in org.apache.hadoop.util.TestDataChecksum
testCrc32(org.apache.hadoop.util.TestDataChecksum)  Time elapsed: 0.11 sec  <<< FAILURE!
java.lang.AssertionError: NativeCrc32 is not available
        at org.apache.hadoop.util.Crc32PerformanceTest.<init>(Crc32PerformanceTest.java:122)
        at org.apache.hadoop.util.TestDataChecksum.testCrc32(TestDataChecksum.java:203)
{noformat}

The {{TestDataChecksum#testCrc32}} case should be skipped in non-native profile by calling
{{GenericTestUtils#assumeInNativeProfile}}.


{code}
  public void testCrc32() throws Exception {
    new Crc32PerformanceTest(8, 3, true).run();
    new Crc32PerformanceTest(8, 3, false).run();
  }
{code}

Do we need the both? Running only {{Crc32PerformanceTest(8, 3, true)}} seems to be ok. It
makes test time shorter.


> Add a Crc32 chunked verification benchmark for both directly and non-directly buffer
cases
> ------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-10865
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10865
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>            Priority: Minor
>              Labels: BB2015-05-TBR
>         Attachments: HADOOP-10865.002.patch, c10865_20140717.patch
>
>
> Currently, it is not easy to compare Crc32 chunked verification implementations.  Let's
add a benchmark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message