Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 43E52200D54 for ; Fri, 8 Dec 2017 20:23:04 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 42763160C1F; Fri, 8 Dec 2017 19:23:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 88DA6160BFD for ; Fri, 8 Dec 2017 20:23:03 +0100 (CET) Received: (qmail 51281 invoked by uid 500); 8 Dec 2017 19:23:02 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 51268 invoked by uid 99); 8 Dec 2017 19:23:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Dec 2017 19:23:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 133211A0618 for ; Fri, 8 Dec 2017 19:23:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id IVIzgFedobdx for ; Fri, 8 Dec 2017 19:23:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 12EC35F3CC for ; Fri, 8 Dec 2017 19:23:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 926B8E0162 for ; Fri, 8 Dec 2017 19:23:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 49B51273D3 for ; Fri, 8 Dec 2017 19:23:00 +0000 (UTC) Date: Fri, 8 Dec 2017 19:23:00 +0000 (UTC) From: "Zach York (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-19435) Reopen Files for ClosedChannelException in BucketCache MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 08 Dec 2017 19:23:04 -0000 [ https://issues.apache.org/jira/browse/HBASE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284076#comment-16284076 ] Zach York commented on HBASE-19435: ----------------------------------- [~stack] Unfortunately I'm still working on the reproduction from my side and don't have the helpful logs :( [~ram_krish] Where in the code does it evict blocks in a compaction (just for my own learning, I understand it's not as important here). I'm thinking that your proposition might be correct. This is a major compaction after heavy ingestion with a few key cache configs that cause heavy cache usage (prefetch on open, cache on write, cache on read, etc). > Reopen Files for ClosedChannelException in BucketCache > ------------------------------------------------------ > > Key: HBASE-19435 > URL: https://issues.apache.org/jira/browse/HBASE-19435 > Project: HBase > Issue Type: Bug > Components: BucketCache > Affects Versions: 2.0.0, 1.3.1 > Reporter: Zach York > Assignee: Zach York > Fix For: 1.4.1, 2.0.0-beta-1 > > Attachments: HBASE-19435.branch-1.001.patch, HBASE-19435.master.001.patch, HBASE-19435.master.002.patch, HBASE-19435.master.003.patch, HBASE-19435.master.004.patch, HBASE-19435.master.005.patch, HBASE-19435.master.006.patch, HBASE-19435.master.007.patch, HBASE-19435.master.007.patch > > > When using the FileIOEngine for BucketCache, the cache will be disabled if the connection is interrupted or closed. HBase will then get ClosedChannelExceptions trying to access the file. After 60s, the RS will disable the cache. This causes severe read performance degradation for workloads that rely on this cache. FileIOEngine never tries to reopen the connection. This JIRA is to reopen files when the BucketCache encounters a ClosedChannelException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)