hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13991) Hierarchical Layout for Humongous Tables
Date Tue, 30 Jun 2015 05:46:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14607544#comment-14607544

stack commented on HBASE-13991:

[~benlau] Thank you for writing up design and for the helpful experiments. 

"Simply put, HDFS was not designed to handle the existence of millions of files/directories
in a single directory. "


How about just making a four level bucket of all HFiles. From  http://static.googleusercontent.com/media/research.google.com/en/us/archive/bigtable-osdi06.pdf,
"Each tablet’s SSTables are registered in the METADATA table." (Our [~mbertozzi] has been
going on about going this route for ever -- see his design attached to https://issues.apache.org/jira/browse/HBASE-7806...
It has some in common w/ yours).

Thanks [~benlau]

> Hierarchical Layout for Humongous Tables
> ----------------------------------------
>                 Key: HBASE-13991
>                 URL: https://issues.apache.org/jira/browse/HBASE-13991
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Ben Lau
>            Assignee: Ben Lau
>         Attachments: HBASE-13991-master.patch, HumongousTableDoc.pdf
> Add support for humongous tables via a hierarchical layout for regions on filesystem.
> Credit for most of this code goes to Huaiyu Zhu.  
> Latest version of the patch is available on the review board: https://reviews.apache.org/r/36029/

This message was sent by Atlassian JIRA

View raw message