accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3592) Improvements to RFile
Date Thu, 19 Feb 2015 20:41:12 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328087#comment-14328087
] 

Josh Elser commented on ACCUMULO-3592:
--------------------------------------

bq. If you're suggesting that this change will not require future RFile version number bumps
when we add stuff to this structure, then yeah, let's do it now

Right. Things that we can naturally represent inside this schema-drifting data structure (and,
with proper use by the reader) would allow old readers to read new files. For example, we
add some new value to the header, hypothetically: some density value for key-values in the
file. The reader implementation can use the density score to perform more optimized seeks/reads.
The old readers can still function without this new attribute in the header (and would parse
it without error), while new readers can read it (when present) and leverage it to do things
more efficiently.

It obviously doesn't protect us from any/all changes, but it should reduce the cases in which
we'd otherwise need to bump the RFile version.

> Improvements to RFile
> ---------------------
>
>                 Key: ACCUMULO-3592
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3592
>             Project: Accumulo
>          Issue Type: Sub-task
>            Reporter: Josh Elser
>             Fix For: 1.7.0
>
>
> We have a small "header" at the beginning of the RFile index at the end of the file.
This is currently just a few writable things thrown together -- it would be nice to convert
this into some pb/thrift message that we can add/remove things easily.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message