accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3089) Create a volume chooser that makes decisions based on table attributes
Date Wed, 01 Oct 2014 18:16:34 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155228#comment-14155228
] 

Josh Elser commented on ACCUMULO-3089:
--------------------------------------

I just took the time to re-read the (final) [design document|https://issues.apache.org/jira/secure/attachment/12615761/20131125-HeterogeneousStorage.pdf]
for HSM. This does convince me that I think leveraging HSM is a better long-term solution
than trying to perpetuate our own notion of volume management within Accumulo. The reason
HSM appears relevant to me is because the primary use case provided was giving preference
to certain types of physical storage for HDFS block storage -- one of the primary reasons
that HSM was implemented in the first place. I also agree with Sean in that trying to roll
our own notion of volumes, and thus the management of, is wasted effort given that we already
have resources upstream of us who are giving the specific problem much more thought and consideration
that we can (because we're at such a higher-level). I also agree that we should try to use
new or in-progress features from Hadoop as we're in one of the best positions to contribute
meaningful feedback to shape those features.

This all being said, I don't personally feel the need to look a gift horse in the mouth with
the proposed changes here. While I think trying to leverage HSM is a better goal, I wouldn't
turn away the code that Jenna has written. Reading through her initial implementation did
help solidify what the design would look like which helped a bit. Given what we currently
have in Accumulo, it would work fine in the end. Again, I'm not a huge fan of how the volume
code turned out, but the code does use them in a reasonable fashion.

While I can't talk for Sean, I think I share some of the same opinions with him and hopefully
that helps answer some of the lingering questions.

(reading back before posting...)  Dave did post an additional use case later about separating
index and data tables onto separate volumes which is a use-case that HSM would not solve (unless
you would have enough "fast" storage to store your entire index with slower storage for the
data). It's obvious that there is at least one case that the proposed implementation Jenna
presented would solve that HSM would not address well. I would encourage some more thought
to be put into these use cases and written down as it will help focus what the implementation
actually looks like.

> Create a volume chooser that makes decisions based on table attributes
> ----------------------------------------------------------------------
>
>                 Key: ACCUMULO-3089
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3089
>             Project: Accumulo
>          Issue Type: Improvement
>            Reporter: Christopher Tubbs
>            Assignee: Jenna Huston
>
> Use case:
> User provisions multiple volumes, some with tmpfs drives, some with SSDs, some with traditional
magnetic spindle hard drives. A volume chooser could use attribute information on tables (ACCUMULO-2841)
to decide which volume to choose when creating new tablets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message