Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <53CF11C1.7060607@gmail.com>
References: 
 <CAGUtCHrG+a=JtYZyFZE-txpAHeNiBy+zbmc1fMTmcqkuJF6osA@mail.gmail.com>
	<53CF11C1.7060607@gmail.com>
Date: Wed, 23 Jul 2014 09:55:46 -0400
Message-ID: 
 <CAGUtCHq5syAjeSGF1L7LGx--=fd1cuHeFK+H=QRGH-FxGtjs4Q@mail.gmail.com>
Subject: Re: [DISCUSS] Removing relative paths from metadata
From: Keith Turner <keith@deenlo.com>
To: Accumulo Dev List <dev@accumulo.apache.org>
Content-Type: multipart/alternative; boundary=001a11c1e488703b1f04fedcb2b5

--001a11c1e488703b1f04fedcb2b5
Content-Type: text/plain; charset=UTF-8

On Tue, Jul 22, 2014 at 9:37 PM, Josh Elser <josh.elser@gmail.com> wrote:

> On 7/22/14, 12:51 PM, Keith Turner wrote:
>
>> Had some discussion w/ Dave Marion about the need to drop relatavie paths
>> from internal metadata.  From a user standpoint the requirement to
>> possibly
>> configure instance.dfs.uri and instance.dfs.dir if they might have
>> relative
>> paths is confusing over the long term.  Also it places more of a
>> maintenance burden on us if we need to ensure all bug fixes and new
>> features work properly w/ relative paths.
>>
>
> Assuming that we squash relative paths by 1.7.0, we shouldn't have any
> additional burden on new feature work because there should be no new
> features in 1.6. Bug fixes are still potentially more complex.
>
> I think everyone would agree that 1.6.0 should've nuked relative paths
> (I'm sorry if I squash anyone's opinions, but that was the impression I got
> before 1.6.0 came out). I think trying to eradicate them in 1.6 would just
> add even more confusion to an already sufficiently confusing situation. If
> a sufficiently simple approach came be thought of for a 1.6.x, I would be
> open to hear it.
>
>
>  What are our options and what should the timeline be?  We could require
>> the
>> user to do something to remove all relative paths before before starting
>> 1.7.0 for example.
>>
>> Some of the things we discussed
>>
>>   * Provide a utility to rewrite all relative paths
>>   * Rework the volume replacement code to work w/ relative paths
>>
>> A stand alone utility is tricky.  Don't want to modify tablet metadata if
>> the table is loaded.  Thats why the volume replacement code has the
>> tablets
>> themselves do the replacement.
>>
>
> I think I like the idea of writing a standalone utility as, while the
> "safe" conditions to run such a utility are harder, getting the rewrite
> correct is much easier. Didn't Sean already write some sort of check for an
> "is Accumulo off" environment?


I think to make this utility safe, accumulo would need the ability to take
the metadata table offline inorder to update the root table.  The root
table would need to be taken offline inorder to safely update zookeeper.  I
don't think the tables can be taken offline.   Also need to ensure nothing
brings them online while the operation is running.


>
>
>  I like the idea of reworking the volume replacement code, but I do not
>> like
>> the idea of it happening automatically (like the first time 1.6.2 is
>> started).   Could possibly have a boolean config
>> instance.volume.replaceRelative.  When this is set, as tablets are loaded
>> and when the GC starts relative paths would be replaced using current
>> instance.dfs.* config or hdfs config.
>>
>> Still uncertain about the best solution.  Looking for the course of least
>> user confusion and least maintenance.  I think
>> instance.volume.replaceRelative is a bit confusing from a user
>> perspective.
>>
>> What other options are there to solve this problem?  Any issue w/ the
>> premise?
>>
>>

--001a11c1e488703b1f04fedcb2b5--