accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dlmar...@comcast.net
Subject Re: Planning for (eventual) removal of instance.dfs.{uri,dir}
Date Thu, 11 Dec 2014 16:33:28 GMT
I think we crossed the streams. I'll talk to you offline. 


----- Original Message -----

From: "Christopher" <ctubbsii@apache.org> 
To: "Accumulo Dev List" <dev@accumulo.apache.org> 
Sent: Thursday, December 11, 2014 11:27:48 AM 
Subject: Re: Planning for (eventual) removal of instance.dfs.{uri,dir} 

On Thu, Dec 11, 2014 at 10:52 AM, <dlmarion@comcast.net> wrote: 

> "Making it required is annoying if users don't have relative paths." 
> 
> But this property is used to determine the location of new files..... 
> 
> 
No, it's not. Not if you're using instance.volumes. It should only be used 
for resolving old files in that case. 


> ----- Original Message ----- 
> 
> From: "Christopher" <ctubbsii@apache.org> 
> To: "Accumulo Dev List" <dev@accumulo.apache.org> 
> Sent: Thursday, December 11, 2014 10:03:56 AM 
> Subject: Re: Planning for (eventual) removal of instance.dfs.{uri,dir} 
> 
> Well, no, the database will start if we rely on instance.volumes, but we 
> may be unable to find files that have relative paths, if we incorrectly 
> assume /accumulo. Making it required is annoying if users don't have 
> relative paths. 
> 
> 
> -- 
> Christopher L Tubbs II 
> http://gravatar.com/ctubbsii 
> 
> On Thu, Dec 11, 2014 at 8:15 AM, <dlmarion@comcast.net> wrote: 
> 
> > How so? If someone upgrades from another version and is using a different 
> > dir, doesn't specify it in the configuration, and we assume /accumulo, 
> then 
> > their database won't start. The other option, which may be safer than 
> > making any assumptions, is to make instance.volumes a required parameter 
> > with no defaults. 
> > 
> > ----- Original Message ----- 
> > 
> > From: "Christopher" <ctubbsii@apache.org> 
> > To: "Accumulo Dev List" <dev@accumulo.apache.org> 
> > Sent: Wednesday, December 10, 2014 11:51:39 PM 
> > Subject: Re: Planning for (eventual) removal of instance.dfs.{uri,dir} 
> > 
> > The URI is probably reasonable, but the dir is potentially problematic if 
> > you weren't using the default. 
> > 
> > 
> > -- 
> > Christopher L Tubbs II 
> > http://gravatar.com/ctubbsii 
> > 
> > On Wed, Dec 10, 2014 at 10:03 PM, dlmarion <dlmarion@comcast.net> wrote: 
> > 
> > > Looks like VolumeConfiguration falls back to fs.defaultFS for the uri 
> and 
> > > /accumulo for the dir. You could remove both properties and still keep 
> > this 
> > > as the documented default behavior if instance.volumes is not 
> specified. 
> > > 
> > > 
> > > 
> > > <div>-------- Original message --------</div><div>From: Christopher
< 
> > > ctubbsii@apache.org> </div><div>Date:12/10/2014 9:13 PM (GMT-05:00)

> > > </div><div>To: Accumulo Dev List <dev@accumulo.apache.org>

> > > </div><div>Cc: </div><div>Subject: Re: Planning for
(eventual) removal 
> of 
> > > instance.dfs.{uri,dir} </div><div> 
> > > </div>My ACCUMULO-2589 branch in github ( 
> > > https://github.com/ctubbsii/accumulo/tree/ACCUMULO-2589) does have a 
> > > commit 
> > > that drops a bunch of stuff (which may or may not be accepted as is for 
> > > 2.0). The instance.dfs.{uri,dir} properties aren't so easy and require 
> > more 
> > > planning, because it's not just removing a property... it's also 
> dealing 
> > > with updating internal data by making relative paths absolute. 
> > > 
> > > For what it's worth, I'm also looking at what changes if we drop 
> Hadoop 1 
> > > support. 
> > > 
> > > As for the validation of config, I think we do a sanity check on 
> startup 
> > > already, which we can always extend. Doesn't solve this issue, though. 
> > > 
> > > 
> > > -- 
> > > Christopher L Tubbs II 
> > > http://gravatar.com/ctubbsii 
> > > 
> > > On Wed, Dec 10, 2014 at 8:59 PM, dlmarion <dlmarion@comcast.net> 
> wrote: 
> > > 
> > > > We should schedule a bunch of deprecated things for removal in 2.0 to

> > > ease 
> > > > maintenance. Do we have a way to validate the site.xml and zookeeper 
> > > > settings before startup and fail with appropriate error message. 
> > > > 
> > > > 
> > > > 
> > > > <div>-------- Original message --------</div><div>From:
Christopher < 
> > > > ctubbsii@apache.org> </div><div>Date:12/10/2014 8:44 PM
(GMT-05:00) 
> > > > </div><div>To: Accumulo Dev List <dev@accumulo.apache.org>

> > > > </div><div>Cc: </div><div>Subject: Planning for
(eventual) removal of 
> > > > instance.dfs.{uri,dir} </div><div> 
> > > > </div>So, 
> > > > 
> > > > instance.volumes replaces instance.dfs.uri and instance.dfs.dir in 
> 1.6. 
> > > > But, what's our long-term plan for these? I ask, because we still 
> have 
> > > > internal code that uses instance.dfs.uri to resolve relative paths. 
> > > > 
> > > > Should we force these to be re-written at some point (maybe on 
> upgrade 
> > to 
> > > > 1.7)? Should we continue to support the deprecated properties 
> > > indefinitely 
> > > > and continue the lazy re-write-on-compact? Do we transition by 
> > requiring 
> > > > instance.volumes to specify the volumes, and only use the old 
> > properties 
> > > to 
> > > > resolve relative paths? 
> > > > 
> > > > My personal view is that we should provide an upgrade-prep/check tool

> > > prior 
> > > > to upgrade to 2.0, which checks and/or re-writes paths and verifies 
> > that 
> > > > instance.volumes is set. 
> > > > 
> > > > Does anybody have a different opinion on this? 
> > > > 
> > > > -- 
> > > > Christopher L Tubbs II 
> > > > http://gravatar.com/ctubbsii 
> > > > 
> > > 
> > 
> > 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message