hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-9127) HFileContext
Date Mon, 05 Aug 2013 19:34:48 GMT

     [ https://issues.apache.org/jira/browse/HBASE-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-9127:
----------------------------------

    Attachment: 0003-Move-data-block-encoder-into-HFileContext.patch
                0002-Move-compression-algorithm-into-HFileContext.patch
                0001-Introduce-HFileContext.patch

Attached are three patches as a thought experiment. They are based on 0.94 but taking the
same actions in later versions would be similar.

0001 - Introduces HFileContext and moves CacheConfig into it. 

0002 - Moves compression algorithm selection into HFileContext.

0003 - Moves in cache and on disk data block encoder selection into HFileContext. This one
is not tested, so a few unit tests may need more tweaks.

Next steps could be: 

- Move BloomType into HFileContext

- Move the MVCC read point into HFileContext

- Test if the additional indirection has a performance impact.

- Consider what more of HFileContext state can be made final/immutable to facilitate inlining.
This is interesting wherever a reader wants to reconfigure based on file trailer or other
metadata discovered when opening the HFile.

- CacheConfig could be fully rolled into HFileContext.

- In the HFile V3 work we have a patch that, like in the existing case with memstoreTS, does
whole file optimization if no tags will be stored (just like if memstoreTS would be 0 for
all KVs in an HFile), which breaks encapsulation in the Store and Compactor. Putting this
into HFileContext (at some future time) would fix that.

- HBASE-7544 could be reworked to use this instead of sprinkling additional method parameters
throughout Store and HFile.
                
> HFileContext
> ------------
>
>                 Key: HBASE-9127
>                 URL: https://issues.apache.org/jira/browse/HBASE-9127
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>         Attachments: 0001-Introduce-HFileContext.patch, 0002-Move-compression-algorithm-into-HFileContext.patch,
0003-Move-data-block-encoder-into-HFileContext.patch
>
>
> We can roll up at least some of the state we are leaking between the Store layer and
the HFile layer by introducing an IO context for passing between the two. This idea has come
up in other discussions. Here I am calling it 'HFileContext' because the particulars are regarding
how to configure HFile readers and writers. This will be easier to maintain than various and
sundry method parameters and (duplicated) instance variables sprinkled about, and will make
adding or modifying persistence features easier and less disruptive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message