hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zach York (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19435) Reopen Files for ClosedChannelException in BucketCache
Date Thu, 07 Dec 2017 22:42:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282637#comment-16282637
] 

Zach York commented on HBASE-19435:
-----------------------------------

[~stack] Yes we have seen instances of this in production. We are currently evaluating the
solution. I can prove that it works for this particular Exception (the unit test proves that),
but as we haven't completely root caused this, there might be an addendum on what Exceptions
are handled.

bq. nit: if we fail the close here:
bq. LOG.warn("Failed to close FileChannel", e);
bq. ... will we be able to open the file subsequently (at least there'll be a hanging fd).

Since this is only used in tests, I don't see this as a risk. What's the worst that will happen?
We have a resource leak that will be cleaned up when the tests finish.

> Reopen Files for ClosedChannelException in BucketCache
> ------------------------------------------------------
>
>                 Key: HBASE-19435
>                 URL: https://issues.apache.org/jira/browse/HBASE-19435
>             Project: HBase
>          Issue Type: Bug
>          Components: BucketCache
>    Affects Versions: 2.0.0, 1.3.1
>            Reporter: Zach York
>            Assignee: Zach York
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-19435.branch-1.001.patch, HBASE-19435.master.001.patch, HBASE-19435.master.002.patch,
HBASE-19435.master.003.patch, HBASE-19435.master.004.patch, HBASE-19435.master.005.patch,
HBASE-19435.master.006.patch, HBASE-19435.master.007.patch
>
>
> When using the FileIOEngine for BucketCache, the cache will be disabled if the connection
is interrupted or closed. HBase will then get ClosedChannelExceptions trying to access the
file. After 60s, the RS will disable the cache. This causes severe read performance degradation
for workloads that rely on this cache. FileIOEngine never tries to reopen the connection.
This JIRA is to reopen files when the BucketCache encounters a ClosedChannelException.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message