hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaowei Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10796) libhdfs++: rationalize ioservice interactions
Date Fri, 09 Dec 2016 21:20:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736377#comment-15736377

Xiaowei Zhu commented on HDFS-10796:

Things look good to me. The only concern is the failures with multiply threads running. I
think one way to verify that this patch doesn't introduce anything new is to only set number
o threads larger than 1 and run the tests. Then we can compare w/ and w/o your patch, if the
failures look all the same, and if there is anything new.

> libhdfs++: rationalize ioservice interactions
> ---------------------------------------------
>                 Key: HDFS-10796
>                 URL: https://issues.apache.org/jira/browse/HDFS-10796
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Bob Hansen
>            Assignee: James Clampffer
>         Attachments: HDFS-10796.HDFS-8707.000.patch
> Firstly, we should be pulling the number of threads from options.io_threads (which should
default to std::thread::hardware_concurrency()).  The library should pass all tests always
with io_threads set to 1 or to <a very high number>
> Secondly, we should have _a_ constructor where the consumer doesn't need to manage the
IOService explicitly, and the FileSystemImpl should create its own internally.
> Since the FileSystem is defined as being for a particular user/identity, there is a valid
use case for the consumer to be constructing many FileSystem instances to represent many authenticated
users in the same process, but want to share resources (notably have a single io_service shared
amongst them all).  In this case, the consumer would want to own the IOService and pass the
same instance to multiple FileSystem instances.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message