hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2751) Consider closing StoreFiles sometimes
Date Mon, 18 Oct 2010 21:02:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922258#action_12922258

ryan rawson commented on HBASE-2751:

We need to move to a DFSClient that doesn't keep sockets open when a file is opened.  We should
only have 1 socket open against the current Block's datanode, and we open and close them for
pread() calls as they happen.  

jbooth's hdfs patch should probably be mentioned and revived at some point.

> Consider closing StoreFiles sometimes
> -------------------------------------
>                 Key: HBASE-2751
>                 URL: https://issues.apache.org/jira/browse/HBASE-2751
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jean-Daniel Cryans
>            Priority: Minor
> Having a lot of regions per region server could be considered harmless if most of them
aren't used, but that's not really true at the moment. We keep all files opened all the time
(except for rolled HLogs). I'm thinking of 2 solutions
>  # Lazy open the store files, or at least close them down after we read the file info.
Or we could do this for every file except the most recent one.
>  # Close files when they're not in use. We need some heuristic to determine when is the
best moment to declare that a file can be closed. 
> Both solutions go hand in hand, and I think it would be a huge gain in order to lower
the ulimit and xceivers-related issues.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message