Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 739FF119D3 for ; Tue, 9 Sep 2014 16:07:29 +0000 (UTC) Received: (qmail 82522 invoked by uid 500); 9 Sep 2014 16:07:29 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 82475 invoked by uid 500); 9 Sep 2014 16:07:29 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 82463 invoked by uid 99); 9 Sep 2014 16:07:29 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Sep 2014 16:07:29 +0000 Date: Tue, 9 Sep 2014 16:07:29 +0000 (UTC) From: "Daryn Sharp (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127156#comment-14127156 ] Daryn Sharp commented on HDFS-6840: ----------------------------------- In addition to Jason's comment, I'm mildly concerned with the tests assuming and hardcoding the ordering based on seed. Presumably the jdk could change how the seeding works at anytime which would cause test failures. Note that a few months ago I saw a jdk bug about how java's randomness isn't very random at all so it's possible the ordering could change in the near future. > Clients are always sent to the same datanode when read is off rack > ------------------------------------------------------------------ > > Key: HDFS-6840 > URL: https://issues.apache.org/jira/browse/HDFS-6840 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 2.5.0 > Reporter: Jason Lowe > Assignee: Andrew Wang > Priority: Critical > Attachments: hdfs-6840.001.patch, hdfs-6840.002.patch > > > After HDFS-6268 the sorting order of block locations is deterministic for a given block and locality level (e.g.: local, rack. off-rack), so off-rack clients all see the same datanode for the same block. This leads to very poor behavior in distributed cache localization and other scenarios where many clients all want the same block data at approximately the same time. The one datanode is crushed by the load while the other replicas only handle local and rack-local requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)