Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 9 Jan 2015 19:48:35 +0000 (UTC)
From: "Daryn Sharp (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12766514.1420832858000.50335.1420832915001@Atlassian.JIRA>
In-Reply-To: <JIRA.12766514.1420832858000@Atlassian.JIRA>
References: <JIRA.12766514.1420832858000@Atlassian.JIRA>
 <JIRA.12766514.1420832858614@arcas>
Subject: [jira] [Created] (HDFS-7597) Clients seeking over webhdfs may crash
 the NN
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Daryn Sharp created HDFS-7597:
---------------------------------

             Summary: Clients seeking over webhdfs may crash the NN
                 Key: HDFS-7597
                 URL: https://issues.apache.org/jira/browse/HDFS-7597
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: webhdfs
    Affects Versions: 2.0.0-alpha
            Reporter: Daryn Sharp
            Assignee: Daryn Sharp
            Priority: Critical


Webhdfs seeks involve closing the current connection, and reissuing a new open request with the new offset.  The RPC layer caches connections so the DN keeps a lingering connection open to the NN.  Connection caching is in part based on UGI.  Although the client used the same token for the new offset request, the UGI is different which forces the DN to open another unnecessary connection to the NN.

A job that performs many seeks will easily crash the NN due to fd exhaustion.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)