hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6561) Byte array native checksumming on client side
Date Thu, 14 Aug 2014 00:48:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096361#comment-14096361
] 

Colin Patrick McCabe commented on HDFS-6561:
--------------------------------------------

Great idea.

The {{hdfs-6561-just-hadoop-changes.txt}} patch needs to be rebased... it didn't apply cleanly
for me against trunk.

{code}
JNIEXPORT void JNICALL Java_org_apache_hadoop_util_NativeCrc32_nativeComputeChunkedSums
  (JNIEnv *env, jclass clazz,
    jint bytes_per_checksum, jint j_crc_type,
    jobject j_sums, jint sums_offset,
    jobject j_data, jint data_offset, jint data_len,
    jstring j_filename, jlong base_pos, jboolean verify)
{code}

Later, you use an if(likely) on the verify boolean.  Rather than do this, why not just have
a utility function that both nativeComputeChunkedSumsByteArray and nativeVerifyChunkedSums
call?

{code}
-#include <stdint.h>
+#include <stdbool.h>
{code}

Please, no.  There are a lot of older C compilers floating around out there that will choke
on this.  Plus we still need {{stdint.h}}, since we're using {{uint32_t}}, etc. etc.  I don't
think the C99 _Bool stuff adds a lot of type safety anyway, since any non-struct type can
implicitly be converted to a bool, and a bool can be used as in int in many contexts.

> Byte array native checksumming on client side
> ---------------------------------------------
>
>                 Key: HDFS-6561
>                 URL: https://issues.apache.org/jira/browse/HDFS-6561
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, hdfs-client, performance
>            Reporter: James Thomas
>            Assignee: James Thomas
>         Attachments: HDFS-6561.2.patch, HDFS-6561.3.patch, HDFS-6561.patch, hdfs-6561-just-hadoop-changes.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message