hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9144) Refactor libhdfs into stateful/ephemeral objects
Date Fri, 06 Nov 2015 21:28:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994458#comment-14994458
] 

ASF GitHub Bot commented on HDFS-9144:
--------------------------------------

GitHub user bobhansen opened a pull request:

    https://github.com/apache/hadoop/pull/43

    HDFS-9144: libhdfs++ refactoring

    Code changes for HDFS-9144 as described in the JIRA.  Removing some templates and traits
and restructuring the code for more modularity.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/bobhansen/hadoop HDFS-9144-merge

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hadoop/pull/43.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #43
    
----
commit 1fb1ea527c9b5321e6da6c2543859db2ec3eaf7c
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T11:58:41Z

    Refactored NameNodeConnection

commit c6cf5175b9c21561bdcbd22be27f50e22a1d3ebd
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T12:01:36Z

    Removed fs_ from InputStream

commit 8b8190d334224d8acec9a4bef97d5e0226c1045a
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T13:05:53Z

    Moved GetBlockInfo to NN connection

commit 108b54f3079ed21149a59b9222d6d9832ee05d79
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T13:20:56Z

    Moved GetBlockLocations to std::function

commit 6d112a17048bcec437701b422209641e56f6196e
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T13:48:02Z

    Added comments

commit e57b0ed02e29781f347499f0f3546659870aabab
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-22T13:52:39Z

    Stripped whitespace

commit c9c82125e8c0b742ee3a70d6fdbdedca180cdd4f
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-27T16:07:33Z

    Renamed NameNodeConnection to NameNodeOperations

commit 01499b6027ec771ebf04d4723899ee976b2a6044
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-27T23:26:26Z

    Renamed input_stream and asio_continuation

commit 02c67837fe832e45286a675f1a27fa29e1b80a9a
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-27T23:30:44Z

    Renamed CreatePipeline to Connect

commit 5d28d02e1752be74975647f8dc656776ab9e2cbf
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-27T23:58:18Z

    Rename async_connect to async_request

commit 9d98bf41091c923103cbeeadb5459c3119b50584
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T13:01:38Z

    Renamed read_some to read_packet

commit 6ced4a97e297ce0e833db8dbd4b38c91c966d71c
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T13:15:50Z

    Renamed async_request to async_request_block

commit f05a771e578969b9b281de4e0c97887f98b0f2cf
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T13:19:09Z

    Renamed BlockReader::request to request_block

commit fcf1585bf67f84ef8c0acc72660d2ad250005e3b
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T19:12:39Z

    Moved to file_info

commit a3fd975285b25a3eae448e5ac46d0118a14d6610
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T19:16:20Z

    Made file_info pointers const

commit 366f488b8e8364eba3f1966b931216d2bf404ae1
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-28T21:37:46Z

    Refactored DataNodeConnection, etc.

commit 418799feb8d12181d9e5bd6b6aa94333bb21e126
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T13:53:46Z

    Added shared_ptr to DN_Connection

commit f043e154a261e9ff64f1ead450e3a256ecd023a2
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T15:31:28Z

    Moved DNConnection into trait

commit aea859ff34a6768c7df29ec25f1abd2b92835b9e
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T15:32:12Z

    Trimmed whitespace

commit 55d7b5dcd92b0fd9d0011e97d8f47e78c3316205
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T17:23:30Z

    Re-enabled IS tests

commit 142efabbda38852b431d94096d6cef69f5c96393
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T17:31:05Z

    Cleaned up some tests

commit 4bc0f448fe52a762a242428a1331272c9fee3247
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T21:53:57Z

    Working on less templates

commit dd16d4fa9f08f55f9d4140219471f002eca5a8ed
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-29T23:28:01Z

    Compiles!

commit 2b14efa8277c66a3e9e0fb67af925501757d39f8
Author: Bob Hansen <bob@hp.com>
Date:   2015-10-30T20:46:52Z

    Fixed DNconnection signature

commit 8d143e789a98431f8cd2cb08db37a0a05f4d9c77
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-02T16:35:54Z

    Fixed segfault in ReadData

commit b6f5454e626c1caa1b76398c9edf220fc1252be9
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-02T18:36:15Z

    Removed BlockReader callback templates

commit 3b5d712b454f5b817c22909bac2f3477a64624fe
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-02T18:52:16Z

    Removed last templates from BlockReader

commit d9b9241f12a957226df7ccacad07d8e1a0d98cca
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-02T20:56:43Z

    Moved entirely over to BlockReader w/out templates

commit 5de0bce35fb52b7a688d3fc4ad02748106fca38e
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-02T21:06:25Z

    Removed unnecessary impls

commit d5baa8784643bdfed454c8a4ba0edb102d73f40a
Author: Bob Hansen <bob@hp.com>
Date:   2015-11-03T15:00:50Z

    Moved DN to its own file

----


> Refactor libhdfs into stateful/ephemeral objects
> ------------------------------------------------
>
>                 Key: HDFS-9144
>                 URL: https://issues.apache.org/jira/browse/HDFS-9144
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: HDFS-8707
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
>         Attachments: HDFS-9144.HDFS-8707.001.patch, HDFS-9144.HDFS-8707.002.patch
>
>
> In discussion for other efforts, we decided that we should separate several concerns:
> * A posix-like FileSystem/FileHandle object (stream-based, positional reads)
> * An ephemeral ReadOperation object that holds the state for reads-in-progress, which
consumes
> * An immutable FileInfo object which holds the block map and file size (and other metadata
about the file that we assume will not change over the life of the file)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message