hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7597) Clients seeking over webhdfs may crash the NN
Date Fri, 09 Jan 2015 20:01:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271789#comment-14271789

Daryn Sharp commented on HDFS-7597:

The problem stems from the delegation token identifier generating a distinct UGI every time
getUser() is called.  The simple solution is caching the token identifier to ugi mapping.
 The webhdfs servlet  extracts the token identifier's UGI so it can wrap the operation in
a doAs.  Caching will result in the same token identifier (from multiple connections) using
a single UGI context on the client-side which then utilizes the possibly cached RPC connection.

Caching will also benefit the server-side RPC layer.  Multiple token-based connections will
share a common UGI, thus reducing the garbage generated esp. for short-lived connections.

This change should ideally be common, but I'm uncomfortable with whether other non-NN hadoop
services (ex. yarn) are doing server-side manipulation of the ugi.  I'll post a patch early
next week after a POC clears QA.

> Clients seeking over webhdfs may crash the NN
> ---------------------------------------------
>                 Key: HDFS-7597
>                 URL: https://issues.apache.org/jira/browse/HDFS-7597
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.0.0-alpha
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
> Webhdfs seeks involve closing the current connection, and reissuing a new open request
with the new offset.  The RPC layer caches connections so the DN keeps a lingering connection
open to the NN.  Connection caching is in part based on UGI.  Although the client used the
same token for the new offset request, the UGI is different which forces the DN to open another
unnecessary connection to the NN.
> A job that performs many seeks will easily crash the NN due to fd exhaustion.

This message was sent by Atlassian JIRA

View raw message