hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3110) libhdfs implementation of direct read API
Date Tue, 03 Apr 2012 23:24:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245878#comment-13245878

Todd Lipcon commented on HDFS-3110:

Sorry, a few things I missed when I looked at the earlier rev:

- we no longer need to expose the hdfsReadDirect API in hdfs.h, given it's automatically handled.
Instead we should probably rename to {{readDirect}} and make it static in hdfs.c. If you need
it for the test case, then I think we should just add it as an extern declaration in the test,
and leave it non-static -- but not advertise it in hdfs.h. We don't want users to think they
have to use it explicitly.

+    // Bit fields for  hdfsFile_internal flags
nit: extra space


Looks like handleReadResult returns -1 when the underlying code calls an exception. Then we
have the following code:
+    if (noReadBytes != 0) {
+      (*env)->GetByteArrayRegion(env, jbRarray, 0, noReadBytes, buffer);
won't that end up calling {{GetByteArrayRegion}} with a negative length? Seems suspicious.

> libhdfs implementation of direct read API
> -----------------------------------------
>                 Key: HDFS-3110
>                 URL: https://issues.apache.org/jira/browse/HDFS-3110
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: libhdfs
>            Reporter: Henry Robinson
>            Assignee: Henry Robinson
>             Fix For: 0.24.0
>         Attachments: HDFS-3110.0.patch, HDFS-3110.1.patch, HDFS-3110.2.patch, HDFS-3110.3.patch
> Once HDFS-2834 gets committed, we can add support for the new API to libhdfs, which leads
to significant performance increases when reading local data from C.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message