accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <mad...@cloudera.com>
Subject Re: [DISCUSS] Removing relative paths from metadata
Date Thu, 24 Jul 2014 01:17:53 GMT
As long as you can roll back, then I think it's fine. I also have an
implicit promise on wire compatibility, but I don't think that is under
huge risk here.


On Wed, Jul 23, 2014 at 8:51 AM, Keith Turner <keith@deenlo.com> wrote:

> On Wed, Jul 23, 2014 at 3:33 AM, Mike Drob <madrob@cloudera.com> wrote:
>
> > I do not want to see anything get re-written between a 1.6.1 system going
> > down and a 1.6.2 system coming up. We have a wire compatibility promise
> > amongst the double-dot releases, and parts moving around really make me
> > nervous. I think it's just too big of a change.
> >
>
> Are you concerned about the case where someone starts 1.6.2 and then needs
> to roll back to 1.6.1 because of a bug in 1.6.2?  Replacing relative paths
> with absolute paths would not prevent 1.6.1 from running, because 1.6.1 can
> understand absolute paths.
>
> if we do defer this to 1.7.0, then we can mitigate the risk of relative
> paths causing unexpected bugs by upgrading from 1.5 to 1.7 and exercising
> many features.
>
>
> >
> > I have no problem with rewriting anything in the internals between 1.6.x
> > and 1.7.0 (or 2.0.0). Based on experience, it will be a lot harder to
> > implement as a stand-alone utility, but I do not have strong preferences
> on
> > stand-alone or part of the upgrade process.
> >
> >
> > On Tue, Jul 22, 2014 at 8:37 PM, Josh Elser <josh.elser@gmail.com>
> wrote:
> >
> > > On 7/22/14, 12:51 PM, Keith Turner wrote:
> > >
> > >> Had some discussion w/ Dave Marion about the need to drop relatavie
> > paths
> > >> from internal metadata.  From a user standpoint the requirement to
> > >> possibly
> > >> configure instance.dfs.uri and instance.dfs.dir if they might have
> > >> relative
> > >> paths is confusing over the long term.  Also it places more of a
> > >> maintenance burden on us if we need to ensure all bug fixes and new
> > >> features work properly w/ relative paths.
> > >>
> > >
> > > Assuming that we squash relative paths by 1.7.0, we shouldn't have any
> > > additional burden on new feature work because there should be no new
> > > features in 1.6. Bug fixes are still potentially more complex.
> > >
> > > I think everyone would agree that 1.6.0 should've nuked relative paths
> > > (I'm sorry if I squash anyone's opinions, but that was the impression I
> > got
> > > before 1.6.0 came out). I think trying to eradicate them in 1.6 would
> > just
> > > add even more confusion to an already sufficiently confusing situation.
> > If
> > > a sufficiently simple approach came be thought of for a 1.6.x, I would
> be
> > > open to hear it.
> > >
> > >
> > >  What are our options and what should the timeline be?  We could
> require
> > >> the
> > >> user to do something to remove all relative paths before before
> starting
> > >> 1.7.0 for example.
> > >>
> > >> Some of the things we discussed
> > >>
> > >>   * Provide a utility to rewrite all relative paths
> > >>   * Rework the volume replacement code to work w/ relative paths
> > >>
> > >> A stand alone utility is tricky.  Don't want to modify tablet metadata
> > if
> > >> the table is loaded.  Thats why the volume replacement code has the
> > >> tablets
> > >> themselves do the replacement.
> > >>
> > >
> > > I think I like the idea of writing a standalone utility as, while the
> > > "safe" conditions to run such a utility are harder, getting the rewrite
> > > correct is much easier. Didn't Sean already write some sort of check
> for
> > an
> > > "is Accumulo off" environment?
> > >
> > >
> > >  I like the idea of reworking the volume replacement code, but I do not
> > >> like
> > >> the idea of it happening automatically (like the first time 1.6.2 is
> > >> started).   Could possibly have a boolean config
> > >> instance.volume.replaceRelative.  When this is set, as tablets are
> > loaded
> > >> and when the GC starts relative paths would be replaced using current
> > >> instance.dfs.* config or hdfs config.
> > >>
> > >> Still uncertain about the best solution.  Looking for the course of
> > least
> > >> user confusion and least maintenance.  I think
> > >> instance.volume.replaceRelative is a bit confusing from a user
> > >> perspective.
> > >>
> > >> What other options are there to solve this problem?  Any issue w/ the
> > >> premise?
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message