hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-6311) Add support for unix domain sockets to JNI libs
Date Tue, 16 Oct 2012 22:39:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477419#comment-13477419
] 

Colin Patrick McCabe commented on HADOOP-6311:
----------------------------------------------

Thanks for the comments.

With respect to security: there is always a possibility for a client to open a socket with
the same name as the server would have used.  This is similar to the problem with TCP/IP sockets
of a malicious program grabbing the port before the DataNode could get it (or after the DataNode
has died.)

I guess this is a problem that actually is worse with the abstract socket namespace.  With
path-based sockets, you can set up the path so that the permissions of the path itself prevent
this attack.  However, with the abstract socket namespace, there's no way to prevent another
process from grabbing the port first.

I agree that there are downsides to the short-circuit approach.  I was very careful to maintain
the ability for the server to decline to offer short-circuit local reads in my patch set.
 This is obviously important for our future flexibility.  It might be advisable to allow this
on a file-by-file basis as well.

I don't think that on-disk format changes are that big of a deal for the short-circuit pathway.
 We tell old clients they can't use short-circuit reads on those files, and fix new clients
to understand the new format.

We should definitely have a way for short-circuit clients to report statistics, disk errors,
etc. to the DataNode.  However, let's not gate this change on features like that.  They can
easily be added as features later and aren't really related to the core issue of fixing local
reads + security.  I think I'll open a separate JIRA for that.

TCP optimizations are pretty cool, but not when you run on RHEL6, as many folks do :)  Maybe
we should open a separate JIRA to investigate things like TCP fast open, changing TCP kernel
options, etc. might be used with Hadoop in the future.  There are also certain performance
improvements we could do in the read and write paths on the DataNode, but again, that's out
of scope for this JIRA, I think.
                
> Add support for unix domain sockets to JNI libs
> -----------------------------------------------
>
>                 Key: HADOOP-6311
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6311
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: native
>    Affects Versions: 0.20.0
>            Reporter: Todd Lipcon
>            Assignee: Colin Patrick McCabe
>         Attachments: 6311-trunk-inprogress.txt, design.txt, HADOOP-6311.014.patch, HADOOP-6311.016.patch,
HADOOP-6311.018.patch, HADOOP-6311.020b.patch, HADOOP-6311.020.patch, HADOOP-6311.021.patch,
HADOOP-6311.022.patch, HADOOP-6311-0.patch, HADOOP-6311-1.patch, hadoop-6311.txt
>
>
> For HDFS-347 we need to use unix domain sockets. This JIRA is to include a library in
common which adds a o.a.h.net.unix package based on the code from Android (apache 2 license)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message