hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7597) Clients seeking over webhdfs may crash the NN
Date Fri, 09 Jan 2015 22:18:34 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HDFS-7597:
--------------------------------
    Hadoop Flags: Reviewed

I suppose another possibility could be a read-write lock, where the write half of the lock
pair is only held for the {{put}} call.  I think that's an optional thing though considering
how small the race condition risk is here.

+1 for the current patch pending Jenkins.  Thanks again, Daryn.

> Clients seeking over webhdfs may crash the NN
> ---------------------------------------------
>
>                 Key: HDFS-7597
>                 URL: https://issues.apache.org/jira/browse/HDFS-7597
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.0.0-alpha
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-7597.patch
>
>
> Webhdfs seeks involve closing the current connection, and reissuing a new open request
with the new offset.  The RPC layer caches connections so the DN keeps a lingering connection
open to the NN.  Connection caching is in part based on UGI.  Although the client used the
same token for the new offset request, the UGI is different which forces the DN to open another
unnecessary connection to the NN.
> A job that performs many seeks will easily crash the NN due to fd exhaustion.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message