accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Drob <mad...@cloudera.com>
Subject Re: [DISCUSS] Removing relative paths from metadata
Date Thu, 24 Jul 2014 01:18:58 GMT
I think we've heard of people running 1.4.2 fairly recently (David's
teams?) so I would not count of folk upgrading to the latest bugfix. I
would really like to see a 1.7 that can upgrade from 1.6


On Wed, Jul 23, 2014 at 9:28 AM, Keith Turner <keith@deenlo.com> wrote:

> On Wed, Jul 23, 2014 at 10:24 AM, John Vines <vines@apache.org> wrote:
>
> > That is my issue with Keith. Requiring a double dot upgrade to do a
> single
> > dot upgrade is a pretty big break in our operating procedure up until
> this
> > point.
> >
> > How do we ensure people don't go to 1.7.0 straight from 1.6.1 and earlier
> > versions?
> >
>
> Christopher and I disuccsed this yesterday.  After 1.6.2 does its thing,
> could put a special marker in zookeeper.   1.7.0 upgrade would not start
> until it see this.
>
>
> >
> >
> > On Wed, Jul 23, 2014 at 10:03 AM, Keith Turner <keith@deenlo.com> wrote:
> >
> > > On Wed, Jul 23, 2014 at 3:33 AM, Mike Drob <madrob@cloudera.com>
> wrote:
> > >
> > > > I do not want to see anything get re-written between a 1.6.1 system
> > going
> > > > down and a 1.6.2 system coming up. We have a wire compatibility
> promise
> > > > amongst the double-dot releases, and parts moving around really make
> me
> > > > nervous. I think it's just too big of a change.
> > > >
> > >
> > > One thing thats really screwy about the proposal to do this in 1.6.2
> (and
> > > completely drop support for relative paths in 1.7.0), is that you have
> to
> > > run 1.6.2 before you can upgrade to 1.7.0.     This is something
> > > Christopher pointed out in an offline discussion yesterday.   Is this
> the
> > > concern you had?  This may be the biggest reason not to do it.  I think
> > in
> > > practice most production users will end up on later bug fix versions of
> > > 1.6.0 anyway.   No one runs 1.4.0 or 1.4.1 anymore.  But not sure if we
> > can
> > > count on that.  If 1.6.1 is stable and works for a user, they may just
> > > stick with it.
> > >
> > >
> > > >
> > > > I have no problem with rewriting anything in the internals between
> > 1.6.x
> > > > and 1.7.0 (or 2.0.0). Based on experience, it will be a lot harder to
> > > > implement as a stand-alone utility, but I do not have strong
> > preferences
> > > on
> > > > stand-alone or part of the upgrade process.
> > > >
> > > >
> > > > On Tue, Jul 22, 2014 at 8:37 PM, Josh Elser <josh.elser@gmail.com>
> > > wrote:
> > > >
> > > > > On 7/22/14, 12:51 PM, Keith Turner wrote:
> > > > >
> > > > >> Had some discussion w/ Dave Marion about the need to drop
> relatavie
> > > > paths
> > > > >> from internal metadata.  From a user standpoint the requirement
to
> > > > >> possibly
> > > > >> configure instance.dfs.uri and instance.dfs.dir if they might
have
> > > > >> relative
> > > > >> paths is confusing over the long term.  Also it places more of
a
> > > > >> maintenance burden on us if we need to ensure all bug fixes and
> new
> > > > >> features work properly w/ relative paths.
> > > > >>
> > > > >
> > > > > Assuming that we squash relative paths by 1.7.0, we shouldn't have
> > any
> > > > > additional burden on new feature work because there should be no
> new
> > > > > features in 1.6. Bug fixes are still potentially more complex.
> > > > >
> > > > > I think everyone would agree that 1.6.0 should've nuked relative
> > paths
> > > > > (I'm sorry if I squash anyone's opinions, but that was the
> > impression I
> > > > got
> > > > > before 1.6.0 came out). I think trying to eradicate them in 1.6
> would
> > > > just
> > > > > add even more confusion to an already sufficiently confusing
> > situation.
> > > > If
> > > > > a sufficiently simple approach came be thought of for a 1.6.x, I
> > would
> > > be
> > > > > open to hear it.
> > > > >
> > > > >
> > > > >  What are our options and what should the timeline be?  We could
> > > require
> > > > >> the
> > > > >> user to do something to remove all relative paths before before
> > > starting
> > > > >> 1.7.0 for example.
> > > > >>
> > > > >> Some of the things we discussed
> > > > >>
> > > > >>   * Provide a utility to rewrite all relative paths
> > > > >>   * Rework the volume replacement code to work w/ relative paths
> > > > >>
> > > > >> A stand alone utility is tricky.  Don't want to modify tablet
> > metadata
> > > > if
> > > > >> the table is loaded.  Thats why the volume replacement code has
> the
> > > > >> tablets
> > > > >> themselves do the replacement.
> > > > >>
> > > > >
> > > > > I think I like the idea of writing a standalone utility as, while
> the
> > > > > "safe" conditions to run such a utility are harder, getting the
> > rewrite
> > > > > correct is much easier. Didn't Sean already write some sort of
> check
> > > for
> > > > an
> > > > > "is Accumulo off" environment?
> > > > >
> > > > >
> > > > >  I like the idea of reworking the volume replacement code, but I
do
> > not
> > > > >> like
> > > > >> the idea of it happening automatically (like the first time 1.6.2
> is
> > > > >> started).   Could possibly have a boolean config
> > > > >> instance.volume.replaceRelative.  When this is set, as tablets
are
> > > > loaded
> > > > >> and when the GC starts relative paths would be replaced using
> > current
> > > > >> instance.dfs.* config or hdfs config.
> > > > >>
> > > > >> Still uncertain about the best solution.  Looking for the course
> of
> > > > least
> > > > >> user confusion and least maintenance.  I think
> > > > >> instance.volume.replaceRelative is a bit confusing from a user
> > > > >> perspective.
> > > > >>
> > > > >> What other options are there to solve this problem?  Any issue
w/
> > the
> > > > >> premise?
> > > > >>
> > > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message