hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8648) libhadoop: native CRC32 validation crashes when io.bytes.per.checksum=1
Date Wed, 05 Sep 2012 20:57:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe updated HADOOP-8648:
-----------------------------------------

    Attachment: HADOOP-8648.005.patch

* add test_bulk_crc32 execution to the native profile, not the kerberos profile.  (Good catch,
Eli.)
                
> libhadoop:  native CRC32 validation crashes when io.bytes.per.checksum=1
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-8648
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8648
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HADOOP-8648.001.patch, HADOOP-8648.002.patch, HADOOP-8648.003.patch,
HADOOP-8648.004.patch, HADOOP-8648.005.patch
>
>
> The native CRC32 code, found in {{pipelined_crc32c}}, crashes when chunksize is set to
1.
> {code}
> 12:27:14,886  INFO NativeCodeLoader:50 - Loaded the native-hadoop library
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007fa00ee5a340, pid=24100, tid=140326058854144
> #
> # JRE version: 6.0_29-b11
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # C  [libhadoop.so.1.0.0+0x8340]  pipelined_crc32c+0xa0
> #
> # An error report file with more information is saved as:
> # /h/hs_err_pid24100.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> Aborted
> {code}
> The Java CRC code works fine in this case.
> Choosing blocksize=1 is a __very__ odd choice.  It means that we're storing a 4-byte
checksum for every byte. 
> {code}
> -rw-r--r--  1 cmccabe users  49398 Aug  3 11:33 blk_4702510289566780538
> -rw-r--r--  1 cmccabe users 197599 Aug  3 11:33 blk_4702510289566780538_1199.meta
> {code}
> However, obviously crashing is never the right thing to do.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message