accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <mad...@cloudera.com>
Subject Re: [DISCUSS] Removing relative paths from metadata
Date Wed, 23 Jul 2014 07:33:08 GMT
I do not want to see anything get re-written between a 1.6.1 system going
down and a 1.6.2 system coming up. We have a wire compatibility promise
amongst the double-dot releases, and parts moving around really make me
nervous. I think it's just too big of a change.

I have no problem with rewriting anything in the internals between 1.6.x
and 1.7.0 (or 2.0.0). Based on experience, it will be a lot harder to
implement as a stand-alone utility, but I do not have strong preferences on
stand-alone or part of the upgrade process.


On Tue, Jul 22, 2014 at 8:37 PM, Josh Elser <josh.elser@gmail.com> wrote:

> On 7/22/14, 12:51 PM, Keith Turner wrote:
>
>> Had some discussion w/ Dave Marion about the need to drop relatavie paths
>> from internal metadata.  From a user standpoint the requirement to
>> possibly
>> configure instance.dfs.uri and instance.dfs.dir if they might have
>> relative
>> paths is confusing over the long term.  Also it places more of a
>> maintenance burden on us if we need to ensure all bug fixes and new
>> features work properly w/ relative paths.
>>
>
> Assuming that we squash relative paths by 1.7.0, we shouldn't have any
> additional burden on new feature work because there should be no new
> features in 1.6. Bug fixes are still potentially more complex.
>
> I think everyone would agree that 1.6.0 should've nuked relative paths
> (I'm sorry if I squash anyone's opinions, but that was the impression I got
> before 1.6.0 came out). I think trying to eradicate them in 1.6 would just
> add even more confusion to an already sufficiently confusing situation. If
> a sufficiently simple approach came be thought of for a 1.6.x, I would be
> open to hear it.
>
>
>  What are our options and what should the timeline be?  We could require
>> the
>> user to do something to remove all relative paths before before starting
>> 1.7.0 for example.
>>
>> Some of the things we discussed
>>
>>   * Provide a utility to rewrite all relative paths
>>   * Rework the volume replacement code to work w/ relative paths
>>
>> A stand alone utility is tricky.  Don't want to modify tablet metadata if
>> the table is loaded.  Thats why the volume replacement code has the
>> tablets
>> themselves do the replacement.
>>
>
> I think I like the idea of writing a standalone utility as, while the
> "safe" conditions to run such a utility are harder, getting the rewrite
> correct is much easier. Didn't Sean already write some sort of check for an
> "is Accumulo off" environment?
>
>
>  I like the idea of reworking the volume replacement code, but I do not
>> like
>> the idea of it happening automatically (like the first time 1.6.2 is
>> started).   Could possibly have a boolean config
>> instance.volume.replaceRelative.  When this is set, as tablets are loaded
>> and when the GC starts relative paths would be replaced using current
>> instance.dfs.* config or hdfs config.
>>
>> Still uncertain about the best solution.  Looking for the course of least
>> user confusion and least maintenance.  I think
>> instance.volume.replaceRelative is a bit confusing from a user
>> perspective.
>>
>> What other options are there to solve this problem?  Any issue w/ the
>> premise?
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message