accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-4078) Exclude special volumes
Date Mon, 21 Dec 2015 22:09:46 GMT


Josh Elser commented on ACCUMULO-4078:

One last round of responses here on the parent :). Let's move discussion into the appropriate
sub-tasks since Christopher took the time to make them

bq.  What's your intended effect from such a change? Remember that WALs are per-server, not

I'd like to exclude the WAL from the special volume for all tables. Let's assume that the
special instance is appropriately sized for only that one table. The PerTableVolumeChooser
will default to the random chooser for WAL files. The random chooser which will include all
volumes, including the one that I want to exclude.

This would be an extreme overhaul of the WAL system. While it's easy to choose a different
volume for a WAL, the point is that *all* tables hosted on a server share the same WAL file.
There is no means to have a separate WAL for just one file. It's not impossible -- I just
want to make sure you understand that this would be a serious undertaking.

bq. Sorry I realize now that the language was unclear. By "sticky dir" I was referring to
the srv:dir entry for that tablet. The thought was that there is no need to choose a volume
for the temp dir in the same way that Accumulo doesn't choose a volume each time it writes
a new file for a tablet. It chooses a volume when first creating the volume and then continues
to send its files there.

Gotcha, I understand where you were coming from now. Maybe it makes sense to have a srv:temp
for a tablet? What's the net benefit of keeping a static temporary directory for a tablet?

bq. If the PerTableVolumeChooser is used as the general volume chooser and PreferredVolumeChooser
is used as the site-wide table volume chooser, then this use case is nearly satisfied. The
special table would be configured to include only the special volume.

Cool, I think this would be a nice addition.

bq. PreferredVolumeChooser and PerTableVolumeChooser could use a configurable default, other
than the RandomVolumeChooser. So there would be a general.volume.chooser and a general.volume.chooser.fallback.

My first reaction is that it seems unnecessary to provide multiple "general*" volume chooser
properties, but I need to read through your situations some more :). I think it would be best
if we can continue to follow the global/site configuration and per-table/namespace overrides
for VolumeChooser properties.

> Exclude special volumes
> -----------------------
>                 Key: ACCUMULO-4078
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.7.0
>            Reporter: Matt Peterson
>            Assignee: Matt Peterson
>            Priority: Minor
>              Labels: newbie
>             Fix For: 1.8.0
> A few improvements to the {{VolumeChooser}} are desired for a use case that seems general
enough to warrant an update to Accumulo.  
> *Use Case:*
> A new volume is added, with limited capacity, to be dedicated for a specific table. 
All other tables or files should be excluded from this new volume.
> *Suggested Improvements:*
> # Update the signature for {{VolumeManager.choose}} to take a {{VolumeChooserEnvironment}}
instead of {{Optional<String>}}.  This will allow future parameters for volume selection
without repeatedly changing the {{VolumeManager}} interface.
> # It's not currently possible to specify preferred volumes for the write-ahead logs
> # In several places including {{PreferredVolumeChooser}}, {{PerTableVolumeChooser}} and
{{VolumeManagerImpl}}, the failsafe chooser is the {{RandomVolumeChooser}} which will include
the instance volume that needs to be excluded.  It would be useful to have a configurable
failsafe in this situation.
> # The volume chooser is called in {{FileUtils}} for temp directory creation but it could
instead use the sticky dir to create the temp directory, not needing the volume chooser at
> The above suggestions could become sub-tickets.  Note that the improvements listed have
been implemented for my particular instance and I hope to submit them as a patch.

This message was sent by Atlassian JIRA

View raw message