Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1A98F119D7 for ; Wed, 2 Jul 2014 22:03:26 +0000 (UTC) Received: (qmail 99899 invoked by uid 500); 2 Jul 2014 22:03:25 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 99855 invoked by uid 500); 2 Jul 2014 22:03:25 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 99842 invoked by uid 99); 2 Jul 2014 22:03:25 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jul 2014 22:03:25 +0000 Date: Wed, 2 Jul 2014 22:03:25 +0000 (UTC) From: "Vladimir Rodionov (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050777#comment-14050777 ] Vladimir Rodionov commented on HBASE-7336: ------------------------------------------ I was not right, Lars. *DFSInputStream* overrides positional read - no locks. But there is something else ... There is no much sense in allowing one random scanner run in a stream mode as since, there is no guarantee that next call to read HFile block from the "lucky" scanner will use the same streaming API and pre-cached data will still be valid. Some other scanner might dump this data before. Correct? You may try all *pread*'s, for all scanners and compare performance. I bet it will be close to what we have right now. > HFileBlock.readAtOffset does not work well with multiple threads > ---------------------------------------------------------------- > > Key: HBASE-7336 > URL: https://issues.apache.org/jira/browse/HBASE-7336 > Project: HBase > Issue Type: Sub-task > Components: Performance > Reporter: Lars Hofhansl > Assignee: Lars Hofhansl > Priority: Critical > Fix For: 0.94.4, 0.95.0 > > Attachments: 7336-0.94.txt, 7336-0.96.txt > > > HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. > It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)