accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4078) Exclude special volumes
Date Fri, 18 Dec 2015 20:06:46 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15064669#comment-15064669
] 

Josh Elser commented on ACCUMULO-4078:
--------------------------------------

Hi [~Matt Peterson]. Thanks for the feedback. I have some thoughts/remarks.

bq. A new volume is added, with limited capacity, to be dedicated for a specific table. All
other tables or files should be excluded from this new volume.

How does the [PerTableVolumeChooser|https://github.com/apache/accumulo/blob/master/server/base/src/main/java/org/apache/accumulo/server/fs/PerTableVolumeChooser.java]
fall short for what you'd like to do? I could see a static VolumeChooser implementation configured
for the table in question that always chooses a specific volume. Every other table would use
the default VolumeChooser (an implementation which ignores that special volume?). Maybe PreferredVolumeChooser
already gives you the ability to limit the volumes you want to use (in a inclusive way, instead
of exclusive).

bq. It's not currently possible to specify preferred volumes for the write-ahead logs

What's your intended effect from such a change? Remember that WALs are per-server, not per-table.

bq. the failsafe chooser is the RandomVolumeChooser which will include the instance volume
that needs to be excluded. It would be useful to have a configurable failsafe in this situation.

By "failsafe" you mean a default chooser? I can see the value in mimicking how we do site-wide
and per-table configuration values. When a per-table implementation doesn't exist (or is not
relevant), it defers to the site-wide chooser. The approach we have now makes the provided
implementations a bit more brittle since you inherit some default functionality (which I think
is what you meant).

bq. The volume chooser is called in FileUtils for temp directory creation but it could instead
use the sticky dir to create the temp directory, not needing the volume chooser at all.

I don't understand what you mean by "use the sticky dir". Why would we not want to use the
volume chooser implementation when choosing a temp directory? This seems like an inconsistency
since we do this for all other file-based operations.

Also, are you planning to work on these changes? If so, let's get this ticket assigned to
you :)

> Exclude special volumes
> -----------------------
>
>                 Key: ACCUMULO-4078
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4078
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.7.0
>            Reporter: Matt Peterson
>            Priority: Minor
>              Labels: newbie
>             Fix For: 1.8.0
>
>
> A few improvements to the VolumeChooser are desired for a use case that seems general
enough to warrant an update to Accumulo.  
> Use Case:
> A new volume is added, with limited capacity, to be dedicated for a specific table. 
All other tables or files should be excluded from this new volume.
> Suggested Improvements:
> 1. Update the signature for VolumeManager.choose to take a VolumeChooserEnvironment instead
of Optional<String>.  This will allow future parameters for volume selection without
repeatedly changing the VolumeManager interface.
> 2. It's not currently possible to specify preferred volumes for the write-ahead logs
> 3. In several places including PreferredVolumeChooser, PerTableVolumeChooser and VolumeManagerImpl,
the failsafe chooser is the RandomVolumeChooser which will include the instance volume that
needs to be excluded.  It would be useful to have a configurable failsafe in this situation.
> 4. The volume chooser is called in FileUtils for temp directory creation but it could
instead use the sticky dir to create the temp directory, not needing the volume chooser at
all.
> The above suggestions could become sub-tickets.  Note that the improvements listed have
been implemented for my particular instance and I hope to submit them as a patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message