hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Hansen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects
Date Thu, 22 Oct 2015 14:00:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969191#comment-14969191

Bob Hansen commented on HDFS-9144:

Thanks for the feedback; [~wheat9]: you are correct, the RpcEngine is owned by the NNConnection.

By "familiar", I mean an easy path for those who are used to POSIX file semantics: open a
file, sequential reads, seeks, close a file.  We will, of course, also support parallel asynchronous
positional reads, but I think we want a nice clean interface for POSIX-y semantics.

Let's break the work down into smaller pieces.  I started with abstracting out the NN connection
in the attached patch.  I will work on abstracting the DN connection next.

> Refactor libhdfs into stateful/ephemeral objects
> ------------------------------------------------
>                 Key: HDFS-9144
>                 URL: https://issues.apache.org/jira/browse/HDFS-9144
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: HDFS-8707
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
>         Attachments: HDFS-9144.HDFS-8707.001.patch
> In discussion for other efforts, we decided that we should separate several concerns:
> * A posix-like FileSystem/FileHandle object (stream-based, positional reads)
> * An ephemeral ReadOperation object that holds the state for reads-in-progress, which
> * An immutable FileInfo object which holds the block map and file size (and other metadata
about the file that we assume will not change over the life of the file)

This message was sent by Atlassian JIRA

View raw message