Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (nike.apache.org: domain of josh.elser@gmail.com designates
 209.85.216.46 as permitted sender)
Message-ID: <5489C276.7080009@gmail.com>
Date: Thu, 11 Dec 2014 11:12:38 -0500
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Postbox 3.0.11 (Macintosh/20140602)
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: Planning for (eventual) removal of instance.dfs.{uri,dir}
References: <qhp6c85n4l15wip8m697bj2k.1418267001999@email.android.com>
 <CAL5zq9aYHMwFwF9XcrwiBs65KtoHpM8hgbzpuqVNG10UoUBX4g@mail.gmail.com>
 <999388760.2391882.1418303736974.JavaMail.zimbra@comcast.net>
 <CAL5zq9a=iFv_t2yy-QftiPFSLUjYK7DhVrPOUcqF8-wKNhzQVQ@mail.gmail.com>
 <20141211154421.GB10400@ll.mit.edu>
In-Reply-To: <20141211154421.GB10400@ll.mit.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

You're building functions over the location of files in HDFS? Automated 
configuration of instances?

Jeremy Kepner wrote:
> When we remove functions, do we have any official guidance to our users who may
> have built applications that use those functions?
>
> Right now, the official position is that the Accumulo developers can remove
> as they see fit.  However, this provides no guidance to users as to what
> they are suppose to do?  As it stands, our guidance is that they have the following choices:
>
> (0) Diligently watch the Accumulo e-mail list and aggressively weigh in on
> any vote to remove functions that may impact them.
>
> (1) Find someone to modify the original source code of their applications,
> build it, and *re-verify* the application.  I emphasise the re-verify because
> that is usually the most costly part of the process that often won't get approved
> by management.
>
> (2) Don't upgrade Accumulo.
>
>
>
>
>
> On Thu, Dec 11, 2014 at 10:03:56AM -0500, Christopher wrote:
>> Well, no, the database will start if we rely on instance.volumes, but we
>> may be unable to find files that have relative paths, if we incorrectly
>> assume /accumulo. Making it required is annoying if users don't have
>> relative paths.
>>
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>> On Thu, Dec 11, 2014 at 8:15 AM,<dlmarion@comcast.net>  wrote:
>>
>>> How so? If someone upgrades from another version and is using a different
>>> dir, doesn't specify it in the configuration, and we assume /accumulo, then
>>> their database won't start. The other option, which may be safer than
>>> making any assumptions, is to make instance.volumes a required parameter
>>> with no defaults.
>>>
>>> ----- Original Message -----
>>>
>>> From: "Christopher"<ctubbsii@apache.org>
>>> To: "Accumulo Dev List"<dev@accumulo.apache.org>
>>> Sent: Wednesday, December 10, 2014 11:51:39 PM
>>> Subject: Re: Planning for (eventual) removal of instance.dfs.{uri,dir}
>>>
>>> The URI is probably reasonable, but the dir is potentially problematic if
>>> you weren't using the default.
>>>
>>>
>>> --
>>> Christopher L Tubbs II
>>> http://gravatar.com/ctubbsii
>>>
>>> On Wed, Dec 10, 2014 at 10:03 PM, dlmarion<dlmarion@comcast.net>  wrote:
>>>
>>>> Looks like VolumeConfiguration falls back to fs.defaultFS for the uri and
>>>> /accumulo for the dir. You could remove both properties and still keep
>>> this
>>>> as the documented default behavior if instance.volumes is not specified.
>>>>
>>>>
>>>>
>>>> <div>-------- Original message --------</div><div>From: Christopher<
>>>> ctubbsii@apache.org>  </div><div>Date:12/10/2014 9:13 PM (GMT-05:00)
>>>> </div><div>To: Accumulo Dev List<dev@accumulo.apache.org>
>>>> </div><div>Cc:</div><div>Subject: Re: Planning for (eventual) removal of
>>>> instance.dfs.{uri,dir}</div><div>
>>>> </div>My ACCUMULO-2589 branch in github (
>>>> https://github.com/ctubbsii/accumulo/tree/ACCUMULO-2589) does have a
>>>> commit
>>>> that drops a bunch of stuff (which may or may not be accepted as is for
>>>> 2.0). The instance.dfs.{uri,dir} properties aren't so easy and require
>>> more
>>>> planning, because it's not just removing a property... it's also dealing
>>>> with updating internal data by making relative paths absolute.
>>>>
>>>> For what it's worth, I'm also looking at what changes if we drop Hadoop 1
>>>> support.
>>>>
>>>> As for the validation of config, I think we do a sanity check on startup
>>>> already, which we can always extend. Doesn't solve this issue, though.
>>>>
>>>>
>>>> --
>>>> Christopher L Tubbs II
>>>> http://gravatar.com/ctubbsii
>>>>
>>>> On Wed, Dec 10, 2014 at 8:59 PM, dlmarion<dlmarion@comcast.net>  wrote:
>>>>
>>>>> We should schedule a bunch of deprecated things for removal in 2.0 to
>>>> ease
>>>>> maintenance. Do we have a way to validate the site.xml and zookeeper
>>>>> settings before startup and fail with appropriate error message.
>>>>>
>>>>>
>>>>>
>>>>> <div>-------- Original message --------</div><div>From: Christopher<
>>>>> ctubbsii@apache.org>  </div><div>Date:12/10/2014 8:44 PM (GMT-05:00)
>>>>> </div><div>To: Accumulo Dev List<dev@accumulo.apache.org>
>>>>> </div><div>Cc:</div><div>Subject: Planning for (eventual) removal of
>>>>> instance.dfs.{uri,dir}</div><div>
>>>>> </div>So,
>>>>>
>>>>> instance.volumes replaces instance.dfs.uri and instance.dfs.dir in 1.6.
>>>>> But, what's our long-term plan for these? I ask, because we still have
>>>>> internal code that uses instance.dfs.uri to resolve relative paths.
>>>>>
>>>>> Should we force these to be re-written at some point (maybe on upgrade
>>> to
>>>>> 1.7)? Should we continue to support the deprecated properties
>>>> indefinitely
>>>>> and continue the lazy re-write-on-compact? Do we transition by
>>> requiring
>>>>> instance.volumes to specify the volumes, and only use the old
>>> properties
>>>> to
>>>>> resolve relative paths?
>>>>>
>>>>> My personal view is that we should provide an upgrade-prep/check tool
>>>> prior
>>>>> to upgrade to 2.0, which checks and/or re-writes paths and verifies
>>> that
>>>>> instance.volumes is set.
>>>>>
>>>>> Does anybody have a different opinion on this?
>>>>>
>>>>> --
>>>>> Christopher L Tubbs II
>>>>> http://gravatar.com/ctubbsii
>>>>>
>>>