accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: [DISCUSS] Removing relative paths from metadata
Date Thu, 24 Jul 2014 19:50:35 GMT
I think you're right, that there's no risk to wire compatibility in this
issue, and the data version wouldn't change in any way, so rollback should
be fine, too. The discussion here is simply a discussion of when/how to
accelerate a metadata upgrade that was already in place in 1.6.0, but did
so gradually in 1.6.0. The question at the heart of the matter here is:
when can we safely write code that no longer has to have special
considerations for relative paths written from older versions?


--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Wed, Jul 23, 2014 at 9:17 PM, Mike Drob <madrob@cloudera.com> wrote:

> As long as you can roll back, then I think it's fine. I also have an
> implicit promise on wire compatibility, but I don't think that is under
> huge risk here.
>
>
> On Wed, Jul 23, 2014 at 8:51 AM, Keith Turner <keith@deenlo.com> wrote:
>
> > On Wed, Jul 23, 2014 at 3:33 AM, Mike Drob <madrob@cloudera.com> wrote:
> >
> > > I do not want to see anything get re-written between a 1.6.1 system
> going
> > > down and a 1.6.2 system coming up. We have a wire compatibility promise
> > > amongst the double-dot releases, and parts moving around really make me
> > > nervous. I think it's just too big of a change.
> > >
> >
> > Are you concerned about the case where someone starts 1.6.2 and then
> needs
> > to roll back to 1.6.1 because of a bug in 1.6.2?  Replacing relative
> paths
> > with absolute paths would not prevent 1.6.1 from running, because 1.6.1
> can
> > understand absolute paths.
> >
> > if we do defer this to 1.7.0, then we can mitigate the risk of relative
> > paths causing unexpected bugs by upgrading from 1.5 to 1.7 and exercising
> > many features.
> >
> >
> > >
> > > I have no problem with rewriting anything in the internals between
> 1.6.x
> > > and 1.7.0 (or 2.0.0). Based on experience, it will be a lot harder to
> > > implement as a stand-alone utility, but I do not have strong
> preferences
> > on
> > > stand-alone or part of the upgrade process.
> > >
> > >
> > > On Tue, Jul 22, 2014 at 8:37 PM, Josh Elser <josh.elser@gmail.com>
> > wrote:
> > >
> > > > On 7/22/14, 12:51 PM, Keith Turner wrote:
> > > >
> > > >> Had some discussion w/ Dave Marion about the need to drop relatavie
> > > paths
> > > >> from internal metadata.  From a user standpoint the requirement to
> > > >> possibly
> > > >> configure instance.dfs.uri and instance.dfs.dir if they might have
> > > >> relative
> > > >> paths is confusing over the long term.  Also it places more of a
> > > >> maintenance burden on us if we need to ensure all bug fixes and new
> > > >> features work properly w/ relative paths.
> > > >>
> > > >
> > > > Assuming that we squash relative paths by 1.7.0, we shouldn't have
> any
> > > > additional burden on new feature work because there should be no new
> > > > features in 1.6. Bug fixes are still potentially more complex.
> > > >
> > > > I think everyone would agree that 1.6.0 should've nuked relative
> paths
> > > > (I'm sorry if I squash anyone's opinions, but that was the
> impression I
> > > got
> > > > before 1.6.0 came out). I think trying to eradicate them in 1.6 would
> > > just
> > > > add even more confusion to an already sufficiently confusing
> situation.
> > > If
> > > > a sufficiently simple approach came be thought of for a 1.6.x, I
> would
> > be
> > > > open to hear it.
> > > >
> > > >
> > > >  What are our options and what should the timeline be?  We could
> > require
> > > >> the
> > > >> user to do something to remove all relative paths before before
> > starting
> > > >> 1.7.0 for example.
> > > >>
> > > >> Some of the things we discussed
> > > >>
> > > >>   * Provide a utility to rewrite all relative paths
> > > >>   * Rework the volume replacement code to work w/ relative paths
> > > >>
> > > >> A stand alone utility is tricky.  Don't want to modify tablet
> metadata
> > > if
> > > >> the table is loaded.  Thats why the volume replacement code has the
> > > >> tablets
> > > >> themselves do the replacement.
> > > >>
> > > >
> > > > I think I like the idea of writing a standalone utility as, while the
> > > > "safe" conditions to run such a utility are harder, getting the
> rewrite
> > > > correct is much easier. Didn't Sean already write some sort of check
> > for
> > > an
> > > > "is Accumulo off" environment?
> > > >
> > > >
> > > >  I like the idea of reworking the volume replacement code, but I do
> not
> > > >> like
> > > >> the idea of it happening automatically (like the first time 1.6.2
is
> > > >> started).   Could possibly have a boolean config
> > > >> instance.volume.replaceRelative.  When this is set, as tablets are
> > > loaded
> > > >> and when the GC starts relative paths would be replaced using
> current
> > > >> instance.dfs.* config or hdfs config.
> > > >>
> > > >> Still uncertain about the best solution.  Looking for the course of
> > > least
> > > >> user confusion and least maintenance.  I think
> > > >> instance.volume.replaceRelative is a bit confusing from a user
> > > >> perspective.
> > > >>
> > > >> What other options are there to solve this problem?  Any issue w/
> the
> > > >> premise?
> > > >>
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message