hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Clampffer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects
Date Mon, 09 Nov 2015 21:47:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997424#comment-14997424

James Clampffer commented on HDFS-9144:

There's a few things I like about this:
-std::function for callbacks makes it a lot easier to figure out what the API expects.
-Polymorphism makes things a lot more flexible than templates (and I expect the performance
impact to be negligible).
-Separating stateful and stateless components makes the interfaces clear and I think will
reduce the chances of introducing bugs that assume things won't change in certain places.
-Merging FileSystem/InputStream and HadoopFileSystem/FileHandle is a major improvement for

Waiting on this is starting to block or at least complicate other work so I'd like it to get
in soon, it seems like a solid improvement to me.


> Refactor libhdfs into stateful/ephemeral objects
> ------------------------------------------------
>                 Key: HDFS-9144
>                 URL: https://issues.apache.org/jira/browse/HDFS-9144
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: HDFS-8707
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
>         Attachments: HDFS-9144.HDFS-8707.001.patch, HDFS-9144.HDFS-8707.002.patch
> In discussion for other efforts, we decided that we should separate several concerns:
> * A posix-like FileSystem/FileHandle object (stream-based, positional reads)
> * An ephemeral ReadOperation object that holds the state for reads-in-progress, which
> * An immutable FileInfo object which holds the block map and file size (and other metadata
about the file that we assume will not change over the life of the file)

This message was sent by Atlassian JIRA

View raw message