hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3110) libhdfs implementation of direct read API
Date Tue, 03 Apr 2012 21:36:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245773#comment-13245773

Todd Lipcon commented on HDFS-3110:

uld be declared {{static}}
- I think your new patch was actually a delta vs the old patch, instead of a completely new
one vs trunk. We need a new one for QA & commit
- When NewDirectByteBuffer returns NULL with no errno set, I think it's better to set {{errno
= ENOMEM;}} in an {{else}} clause -- just a little easier to read.

- The new flag HDFS_SUPPORTS_DIRECT_READ is only used internally, so not sure it belongs in
the public header hdfs.h (this is what users include, right?). Also, I think it would be better
named something like {{HDFS_FILE_SUPPORTS_DIRECT_READ}} since it refers to a specific stream
rather than the entire FS.

- Rather than declaring it as a {{const}} I think it's better to use an enum or #define, since
consts are a C++ thing and this code is mostly straight C. Also, I think it's better to define
it as (1 << 0) to indicate that this is going to be in a bitfield.

- Please add a comment above the definition of the new flag referring to hdfsFile_internal.flags,
so we know where the flags end up.

- the new {{flags}} field should be unsigned -- {{uint32_t}} probably

- in the new test, why are you hardcoding {{localhost:20300}}? I'd think using {{default}}
as before is the right choice, since it will pick up whatever is {{fs.default.name}} in your
{{core-site.xml}} on the classpath. That way this same test can be run against local FS or
against DFS

> libhdfs implementation of direct read API
> -----------------------------------------
>                 Key: HDFS-3110
>                 URL: https://issues.apache.org/jira/browse/HDFS-3110
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: libhdfs
>            Reporter: Henry Robinson
>            Assignee: Henry Robinson
>             Fix For: 0.24.0
>         Attachments: HDFS-3110.0.patch, HDFS-3110.1.patch, HDFS-3110.2.patch
> Once HDFS-2834 gets committed, we can add support for the new API to libhdfs, which leads
to significant performance increases when reading local data from C.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message